MikroTik MetaRouter on Multicore PowerPC

|

This is a repost of something I wrote for the MikroTik forums a few months back: a procedure I developed for enabling (with some caveats) the MetaRouter feature to work on MikroTik’s dual-core PowerPC RouterBoard models.

MikroTik is a router manufacturer and router operating system software developer. They make good stuff. Linux runs at the core of most of it, but almost everything else on top of the kernel itself is homegrown. I like to think of them as the Apple to Cisco’s (old-guard) “mainframe”-esque product line. The price-to-performance metric on their hardware products is phenomenal. MetaRouter is a virtual hypervisor that MikroTik built into the MIPS and PowerPC versions of RouterOS, but which currently is only officially supported on single-core systems. (x86 RouterOS uses KVM for virtualization support on that hardware platform, and has no such limitation.)

(I felt the heavy-handed disclaimer at the beginning was necessary given that I was posting this to their official support forum, and I didn’t want anybody getting the wrong idea.)


Just to be clear, by proceeding with this, you acknowledge that you either have an RB1100Hx2 or an RB1100AHx2 which is not currently being used in a production capacity and which you are willing to experiment on, and that neither MikroTik nor I can be held liable for any damage you may cause to yourself, your property, or your business by performing this mod. In theory, this is just a software change that should have no permanent impact on the hardware and which is easy to undo; however, there are no guarantees attached to this procedure, and…

…THIS IS NOT OFFICIALLY SUPPORTED BY MIKROTIK. AT ALL. PERIOD. DO NOT CONTACT MIKROTIK SUPPORT ABOUT THIS PROCEDURE, EVER. YOU ARE CHOOSING TO PROCEED AT YOUR OWN RISK!

The reason that MetaROUTER is not supported on the Hx2 or the AHx2 is because they are dual-processor systems, and MetaROUTER does not support multiprocessor systems at this time. So in a nutshell, what we are going to be doing to get MetaROUTER working on these boards is replacing the multicore/SMP Linux kernel with the uniprocessor Linux kernel. When RouterOS is installed on a device, the installer determines which kernel to install depending on the hardware it is being installed to. The uniprocessor kernel should (in theory) boot and run an (A)Hx2 board just fine, and this kernel contains all of the support needed to host MetaROUTER guests. The downside is that by doing this, you will be limited to 1 CPU core while running this kernel, essentially “downgrading” your 1100AHx2 to an 1100AH. If MetaROUTER support is worth more to you than the extra CPU core, then this procedure will give you the freedom to choose to make that sacrifice.

Here are the materials that you will need to assemble beforehand:

  1. 1 RB1100Hx2 or RB1100AHx2
  2. 1 other RouterOS router (any kind) to act as a Netboot host
  3. 1 MicroSD card
  4. 1 serial cable
  5. The “upgrade package” NPK for the version of RouterOS you wish to run on the (A)Hx2
  6. A computer with a functioning Python interpreter installed
  7. The following files:

And following is a description of how to put the pieces together; I assume here an advanced familiarity on the part of the reader with RouterOS, networking, and Linux/Unix. If any part of these instructions is unclear, let me know and I will try to fill in the gaps for you.

Basically, what we are going to be doing is extracting the 4 kernel files from the PowerPC NPK and replacing the kernel on the 1100AHx2 with the kernel for the 1100AH. Each PowerPC upgrade NPK contains 4 kernels:

  • One for RB333/600 (Freescale MPC83xx)
  • One for RB1200 (APM/AMCC PPC44x)
  • One for RB800/1000/1100/1100AH (Freescale MPC85xx — uniprocessor)
  • One for RB1100Hx2/1100AHx2 (Freescale MPC85xx — multiprocessor)

The one that is already on your (A)Hx2 is the fourth one. The one you want is the third one. Unfortunately, it is impossible to know which kernel is which without trying each one, because they are all named the same thing in the NPK (‘kernel’). The supplied Python script will extract each kernel file as it comes across it in the NPK and add a number to the end of each file (kernel1, kernel2, kernel3, kernel4), but the kernels are not always stored in the same order in each RouterOS upgrade NPK. In general, I have found that the uniprocessor kernel for RB1100 is most often the second-largest of all the kernels in terms of file size, while the multiprocessor kernel (the one you’re already running) is generally the largest of the 4. In RouterOS 6.4, the uniprocessor RB1xxx kernel is kernel4, but it could be different for different RouterOS versions. Keep this in mind; you may have to perform a little trial-and-error testing before you find the right kernel.

Here we go, step-by-step:

  1. It is recommended that you start by performing a clean, fresh Netinstall of RouterOS onto your Hx2/AHx2.
  2. Boot up the router, and try to create a MetaROUTER guest. You should see the error “not enough resources”. This is expected.
  3. On the machine that has Python installed, put the dumpnpk-ppc-kernels.py Python script and the “upgrade” NPK into the same directory.
  4. Run the Python script and supply the path to the NPK as the first argument; for example: “python dumpnpk-ppc-kernels.py ./routeros-powerpc-6.4.npk”
  5. Verify that the script extracted 4 kernel files from the NPK, located in a subdirectory called ‘boot’, named ‘kernel1’ through ‘kernel4’.
  6. Copy the ‘kernel’ files onto the MicroSD card. Insert into the SD slot on the (A)Hx2.
  7. Prepare the other RouterOS router (item #2 in the “materials needed” list; can be of any type, even x86) to be a Netboot host for the (A)Hx2 by doing the following:
    1. Copy the ‘openwrt-rb1100-linux-2.6.35-initrd.elf’ file to the router.
    2. Create a DHCP server on this router; set the “Boot File Name” of the DHCP Network to ‘openwrt-rb1100-linux-2.6.35-initrd.elf’
    3. Enable the TFTP server on RouterOS.
  8. Hook a serial cable up to the (A)Hx2, and plug ether13 on the (A)Hx2 into the other RouterOS router.
  9. Power up the (A)Hx2. Interrupt the RouterBOOT boot process when you see “Press any key”.
  10. Set the (A)Hx2 to boot via ethernet using options o, 1, and then x to exit and resume booting.
  11. You should eventually see “Please press Enter to activate this console.” Press Enter to get to a shell.
  12. Mount the SD card: “mount -t vfat /dev/mmcblk0 /mnt”
  13. Copy the kernel files from the SD card to the RAM disk temporarily: “cp /mnt/kernel? ~”
  14. Unmount the SD card: “umount /mnt”
  15. Prepare the boot partition of the RouterBOARD’s NAND for mounting: “ubiattach /dev/ubi_ctrl -m 0”
  16. Mount the boot partition of the RouterBOARD’s NAND: “mount -t ubifs /dev/ubi0_0 /mnt”
  17. Pick one of the 4 kernels you extracted from the NPK to replace the kernel on the boot partition, and copy it over on top of the existing kernel: e.g., “cp ~/kernel4 /mnt/kernel”
  18. Cleanly unmount the boot partition: “umount /mnt”
  19. Reboot the router: “reboot”

At this point, RouterOS should boot up again. If it DOES NOT, and the router either goes into a reboot cycle or hangs while booting, you probably picked the wrong kernel. Try another one by starting again from step 8, and choosing a different kernel file at step 17.

Once you have found and successfully installed the uniprocessor kernel, you should be able to go to System -> Resources and verify that it only sees 1 CPU core instead of 2. At this point, try creating and booting a MetaROUTER guest again. If all went well, it should work. Congratulations! You can remove the SD card at this point; it does not need to remain in the router after you have finished the procedure.

To undo the change, simply re-Netinstall the RouterOS version of your choice, which will cause the original multiprocessor kernel to be copied back into the boot partition of the NAND.