Motorola and Lenovo, Sitting in a Tree…

|

Last week, right after it broke, I linked to the news that Google was divesting itself of Motorola, and selling it to Lenovo. I wasn’t quite sure what to think about it then. Part of me was sad because I saw some promise in the changes Google had been making at Motorola, and I was eager to see how they would play out. Towards the end of ATP episode 50, the guys talked some about this turn of events, and after listening to this and having some time to chew on the issue a bit more, I’ve decided that I’m largely optimistic about this new combination. What follows is a slightly edited version of the thoughts that I sent to the ATP folks.


Lenovo and I have a history. Before I started buying Mac laptops, I was a huge ThinkPad fanboy. I started using ThinkPads back in 1998, when it was still an IBM institution. So I was there as the Lenovo takeover unfolded. I know that many people who were once (and perhaps still are) ThinkPad users have not been happy with some of the changes that came about under the Lenovo regime to what they consider to be “signature traits” of the brand, such as what was at one time thought to be the absolute best keyboard on a mobile computer of any stripe. I had my own gripes as well: the timing of this change may have been purely coincidental, but I personally was not happy when, in 2006, roughly a year after the Lenovo acquisition had been finalized, they stopped shipping workstation-grade laptops with high-resolution (for the time) in-plane switching LCD displays. That’s right: IBM was one of the few manufacturers doing IPS (which they called “FlexView”) on their laptops back in those days. In fact, they first offered that option in 2001, well before Apple started to get religion about it. In comparison, the Apple laptop displays of that era (least-common-denominator twisted nematic technology, sub-100 DPI) were absolutely terrible. I didn’t see how anybody could stand to use those things, much less call them gorgeous.

That period between 2006 and 2012, when the Retina MacBook Pro finally came out…those were dark days.

But as John and Marco both alluded to, Lenovo overall did a decent job of not screwing things up…which in the world of high-technology is relatively high praise. I’ll stick my neck out and say it: if I had to buy a PC these days, it would still be a Lenovo ThinkPad, hands-down. I’ve used some recent machines, and the engineering, build-quality, and durability are still excellent. And I think that the key to the relative success of the Lenovo transition was that it wasn’t just a brand acquisition, or just an IP acquisition, or just a talent acquisition. Lenovo saw the value in all of it, wanted all of it, and kept all of it. (To this day, I believe that Lenovo still contracts with IBM to use their international support infrastructure; if you call Lenovo support in the U.S. today, it still rings the same call center in Atlanta that it always has.) They didn’t conduct massive layoffs or restructurings afterward. As I understand it, they retained all of the U.S. offices where the IBM Personal Systems Group guys were housed as well as all of the people in those offices, and at the end of the day, it was still the same guys making the decisions as before, and the same engineers working their magic as before. So if anybody is actually guilty of tarnishing the ThinkPad brand post-Lenovo, I think it could be successfully argued that the blame should be laid squarely at the feet of the same people who had been running the place when it still said “IBM” above the door, and not Lenovo management. In retrospect, I think that the IBM guys had more influence on Lenovo after being bought than vice-versa. Regardless of how it looked on the balance sheets, it felt in practice more like a bizarre reverse-acquisition, as if IBM spun off the PSG, had them acquire Lenovo, and then assumed their name.

Now, I personally think that Google threw in the towel on Motorola way too early. Everybody in the press was going ga-ga over the Moto X when it came out last summer, and, yeah, it looks like a decent phone, but in my opinion, the most interesting and exciting thing to come out of that strange union was the Moto G, which was only released last December. I was (and still am) very bullish about the Moto G, and was eager to watch this experiment unfold and see whether it ended up paying off for them or not. (If you watch the Moto G announcement, it’s actually pretty hilarious: it basically consists of a bunch of ex-Googlers that had been installed at Motorola getting up and publicly saying, in effect, “look, you idiot hardware partners: THIS is how you make an Android phone. Are you getting this, Samsung? Are you taking notes, HTC?”) A couple of months is simply not enough time in this industry to accurately gauge the success or failure of a strategic course correction, and Google getting rid of Motorola almost feels in a sense like HP and Palm all over again. At least Google is giving somebody else the chance to make a go of it rather than just pulling the plug the way HP did, and if Lenovo can maintain that same kind of “hands-off” attitude with this freshly-Googlified Motorola that they had with the IBM guys1, getting acquired by Lenovo may end up proving to be the best possible outcome for them.


  1. There are a couple of notable difference between the IBM Personal Systems Group acquisition and the Motorola acquisition. One is that is that the PSG acquisition didn’t involve taking over the old parent company’s brand — IBM kept their own name, obviously — while in the Motorola acquisition, the Motorola name and trademark are (I presume) along for the ride. Now the question is, do they call the phones that they sell Motorola phones, or Lenovo phones?

    The second is that the PSG group was a fairly mature organization that had more or less existed in the same form for decades. Motorola, on the other hand, has had a rocky past few years. Google didn’t buy Motorola per se, they bought Motorola Mobility, which itself had been split from the parent company just 6 months before. While under Google’s thumb, it would seem that Motorola Mobility underwent some pretty stark changes. So it could be that Motorola, in its current form, is more “malleable” (and more fragile) than the IBM PSG was at the time of its acquisition, and thus more at risk of being meddled with by the uppity-ups at Lenovo. 

MikroTik MetaRouter on Multicore PowerPC

|

This is a repost of something I wrote for the MikroTik forums a few months back: a procedure I developed for enabling (with some caveats) the MetaRouter feature to work on MikroTik’s dual-core PowerPC RouterBoard models.

MikroTik is a router manufacturer and router operating system software developer. They make good stuff. Linux runs at the core of most of it, but almost everything else on top of the kernel itself is homegrown. I like to think of them as the Apple to Cisco’s (old-guard) “mainframe”-esque product line. The price-to-performance metric on their hardware products is phenomenal. MetaRouter is a virtual hypervisor that MikroTik built into the MIPS and PowerPC versions of RouterOS, but which currently is only officially supported on single-core systems. (x86 RouterOS uses KVM for virtualization support on that hardware platform, and has no such limitation.)

(I felt the heavy-handed disclaimer at the beginning was necessary given that I was posting this to their official support forum, and I didn’t want anybody getting the wrong idea.)


Just to be clear, by proceeding with this, you acknowledge that you either have an RB1100Hx2 or an RB1100AHx2 which is not currently being used in a production capacity and which you are willing to experiment on, and that neither MikroTik nor I can be held liable for any damage you may cause to yourself, your property, or your business by performing this mod. In theory, this is just a software change that should have no permanent impact on the hardware and which is easy to undo; however, there are no guarantees attached to this procedure, and…

…THIS IS NOT OFFICIALLY SUPPORTED BY MIKROTIK. AT ALL. PERIOD. DO NOT CONTACT MIKROTIK SUPPORT ABOUT THIS PROCEDURE, EVER. YOU ARE CHOOSING TO PROCEED AT YOUR OWN RISK!

The reason that MetaROUTER is not supported on the Hx2 or the AHx2 is because they are dual-processor systems, and MetaROUTER does not support multiprocessor systems at this time. So in a nutshell, what we are going to be doing to get MetaROUTER working on these boards is replacing the multicore/SMP Linux kernel with the uniprocessor Linux kernel. When RouterOS is installed on a device, the installer determines which kernel to install depending on the hardware it is being installed to. The uniprocessor kernel should (in theory) boot and run an (A)Hx2 board just fine, and this kernel contains all of the support needed to host MetaROUTER guests. The downside is that by doing this, you will be limited to 1 CPU core while running this kernel, essentially “downgrading” your 1100AHx2 to an 1100AH. If MetaROUTER support is worth more to you than the extra CPU core, then this procedure will give you the freedom to choose to make that sacrifice.

Here are the materials that you will need to assemble beforehand:

  1. 1 RB1100Hx2 or RB1100AHx2
  2. 1 other RouterOS router (any kind) to act as a Netboot host
  3. 1 MicroSD card
  4. 1 serial cable
  5. The “upgrade package” NPK for the version of RouterOS you wish to run on the (A)Hx2
  6. A computer with a functioning Python interpreter installed
  7. The following files:

And following is a description of how to put the pieces together; I assume here an advanced familiarity on the part of the reader with RouterOS, networking, and Linux/Unix. If any part of these instructions is unclear, let me know and I will try to fill in the gaps for you.

Basically, what we are going to be doing is extracting the 4 kernel files from the PowerPC NPK and replacing the kernel on the 1100AHx2 with the kernel for the 1100AH. Each PowerPC upgrade NPK contains 4 kernels:

  • One for RB333/600 (Freescale MPC83xx)
  • One for RB1200 (APM/AMCC PPC44x)
  • One for RB800/1000/1100/1100AH (Freescale MPC85xx — uniprocessor)
  • One for RB1100Hx2/1100AHx2 (Freescale MPC85xx — multiprocessor)

The one that is already on your (A)Hx2 is the fourth one. The one you want is the third one. Unfortunately, it is impossible to know which kernel is which without trying each one, because they are all named the same thing in the NPK (‘kernel’). The supplied Python script will extract each kernel file as it comes across it in the NPK and add a number to the end of each file (kernel1, kernel2, kernel3, kernel4), but the kernels are not always stored in the same order in each RouterOS upgrade NPK. In general, I have found that the uniprocessor kernel for RB1100 is most often the second-largest of all the kernels in terms of file size, while the multiprocessor kernel (the one you’re already running) is generally the largest of the 4. In RouterOS 6.4, the uniprocessor RB1xxx kernel is kernel4, but it could be different for different RouterOS versions. Keep this in mind; you may have to perform a little trial-and-error testing before you find the right kernel.

Here we go, step-by-step:

  1. It is recommended that you start by performing a clean, fresh Netinstall of RouterOS onto your Hx2/AHx2.
  2. Boot up the router, and try to create a MetaROUTER guest. You should see the error “not enough resources”. This is expected.
  3. On the machine that has Python installed, put the dumpnpk-ppc-kernels.py Python script and the “upgrade” NPK into the same directory.
  4. Run the Python script and supply the path to the NPK as the first argument; for example: “python dumpnpk-ppc-kernels.py ./routeros-powerpc-6.4.npk”
  5. Verify that the script extracted 4 kernel files from the NPK, located in a subdirectory called ‘boot’, named ‘kernel1’ through ‘kernel4’.
  6. Copy the ‘kernel’ files onto the MicroSD card. Insert into the SD slot on the (A)Hx2.
  7. Prepare the other RouterOS router (item #2 in the “materials needed” list; can be of any type, even x86) to be a Netboot host for the (A)Hx2 by doing the following:
    1. Copy the ‘openwrt-rb1100-linux-2.6.35-initrd.elf’ file to the router.
    2. Create a DHCP server on this router; set the “Boot File Name” of the DHCP Network to ‘openwrt-rb1100-linux-2.6.35-initrd.elf’
    3. Enable the TFTP server on RouterOS.
  8. Hook a serial cable up to the (A)Hx2, and plug ether13 on the (A)Hx2 into the other RouterOS router.
  9. Power up the (A)Hx2. Interrupt the RouterBOOT boot process when you see “Press any key”.
  10. Set the (A)Hx2 to boot via ethernet using options o, 1, and then x to exit and resume booting.
  11. You should eventually see “Please press Enter to activate this console.” Press Enter to get to a shell.
  12. Mount the SD card: “mount -t vfat /dev/mmcblk0 /mnt”
  13. Copy the kernel files from the SD card to the RAM disk temporarily: “cp /mnt/kernel? ~”
  14. Unmount the SD card: “umount /mnt”
  15. Prepare the boot partition of the RouterBOARD’s NAND for mounting: “ubiattach /dev/ubi_ctrl -m 0”
  16. Mount the boot partition of the RouterBOARD’s NAND: “mount -t ubifs /dev/ubi0_0 /mnt”
  17. Pick one of the 4 kernels you extracted from the NPK to replace the kernel on the boot partition, and copy it over on top of the existing kernel: e.g., “cp ~/kernel4 /mnt/kernel”
  18. Cleanly unmount the boot partition: “umount /mnt”
  19. Reboot the router: “reboot”

At this point, RouterOS should boot up again. If it DOES NOT, and the router either goes into a reboot cycle or hangs while booting, you probably picked the wrong kernel. Try another one by starting again from step 8, and choosing a different kernel file at step 17.

Once you have found and successfully installed the uniprocessor kernel, you should be able to go to System -> Resources and verify that it only sees 1 CPU core instead of 2. At this point, try creating and booting a MetaROUTER guest again. If all went well, it should work. Congratulations! You can remove the SD card at this point; it does not need to remain in the router after you have finished the procedure.

To undo the change, simply re-Netinstall the RouterOS version of your choice, which will cause the original multiprocessor kernel to be copied back into the boot partition of the NAND.

Google Mail, SPF, Forwarding, and You

|

I have a Gmail address that I don’t typically publish, and a non-Google-hosted e-mail address that I do. For years I have forwarded all mail received at my public e-mail address over to my Gmail account, and for years this has worked flawlessly. That is, it worked flawlessly until sometime during the weekend of January 18-19, when I noticed that my Gmail inbox had an unusually low message count. (Fortunately, I have my server configured to retain a copy of every message it forwards on to my Gmail address, so I didn’t actually lose any mail.)

I started to run some tests. First, I tried to send myself e-mail from work, and Google bounced the message right back to me:

550-5.7.1 [xxx.xxx.xxx.xxx xx] Our system has detected that this message is
550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to Gmail,
550-5.7.1 this message has been blocked. Please visit
550-5.7.1 http://mail.google.com/support/bin/answer.py?hl=en&answer=188131 for
550 5.7.1 more information.

That’s rarely a good sign.” Likely anyone who had tried to send me e-mail during that time period also ended up receiving a similar bounce message back from Google. In a few cases, e-mail lists that I had signed up for decided to unsubscribe me as a result.

The question, of course, was why Gmail now thought that the mail that I regularly forward to myself was spam, and why it had become a problem all of a sudden when they’ve been gleefully accepting my forwarded mail for years (3 months shy of 10 years, to be precise)?

The first thing that immediately occurred to me was that perhaps Google had blacklisted my server’s IP address for some reason. To test that theory, I tried changing the IP address to a different one temporarily (while also making sure to update DNS and reverse DNS entries), but that didn’t make a difference. As I was looking back through past e-mail headers searching for potential clues, I realized that since my server was running dual IP (v4+v6) stacks, and since Google was reachable via IPv6, that it was preferentially exchanging mail with Google by way of IPv6. Could it be that my IPv6 address was what Google had ended up blacklisting? I told Postfix not to use IPv6, tried sending myself another piece of e-mail, and voilĂ : it went through. I then tried giving my server a different IPv6 address (from a completely different /64 prefix, even), re-enabled IPv6 in Postfix, patched up my DNS zone files again, and sent myself another piece of e-mail. That also worked. Finally, I returned everything back to the way it was before, sent myself mail again, and bam: rejected.

This, to my mind, confirmed the theory that Google had decided to block mail specifically from my server’s IPv6 address. But for what reason? Both my v4 and v6 addresses received a clean bill of health; you’d think that if my server had been compromised and was being used as a relay by some unscrupulous individual(s) that Google would not have been the only one to notice. In any case, I reported my issue to Google since I wasn’t sure what else to do, but I wasn’t going to hold my breath.

When I casually mentioned my dilemma to a co-worker, he suggested that if I hadn’t yet published an SPF record for my domain, that I should give that a try. The interwebs backed his suggestion up: others who had encountered similar difficulty delivering to Gmail servers found that the answer lay in implementing SPF on their end.1 Interesting, to be sure, but I had already added an SPF record to my domain’s zone some time ago. That a lack of (valid) SPF could be the problem just didn’t make sense to me either, because I wasn’t the author of the e-mails that were being bounced, I was the intended recipient. The people who had “fixed” their Gmail woes by taking the time to add SPF to their domain were having problems sending to Gmail directly, whereas I was acting as a forwarder. But I nonetheless went through the motions of verifying that my record was syntactically valid and that it was constructed correctly. And it was.

Later on, as I was once again mulling over the problem, and as my thoughts wandered toward what the exact purpose of SPF was, I had my “aha!” moment: if SPF is all about verifying that the sender is genuine, and it accomplishes this by giving the receiving MX a way to ensure that the transmitting MX is authorized to send mail on behalf of the sender, how can e-mail forwarding work within that kind of a scheme? The answer, sadly, is that it doesn’t (and it doesn’t). If my problem was at all related to SPF, it wasn’t that my SPF record was missing or invalid. The problem was that the people sending me mail were doing so from domains that had implemented SPF, and when my mail server would forward those messages to Gmail, Google would see mail that had “originated” from my server’s IP, and my server would not be a permitted sender for the return address that was listed on the envelope. So from Google’s perspective, I was falsely impersonating all of the people who were sending me mail.

As I discovered while further researching this problem, it turns out that the solution to this mess is apparently not to set up your server as a forwarder, but rather as a remailer.2 That is, something which throws away the original envelope, and then generates a new one that bears your own return address instead of the original sender’s address. In such a scenario, the final recipient’s mail server would check the sender (yourself) against your own domain’s SPF record instead of the SPF record for the e-mail author’s domain, and assuming that your SPF record is constructed correctly, that check would pass. (Note that “remailing” in this way would not cause the e-mail to appear to you as if you had sent it to yourself; the value of the envelope MAIL FROM is distinct from the “From header” on the message itself, in much the same way that the return address on a physical envelope doesn’t necessarily have to match the address that the letter-writer decides to put at the top of the letter inside of that envelope.)

Remailing, though, does have some downsides. If the final mail server does need to “return to sender”, the “sender” in this case is going to be the mail server doing the remailing, and not the original sender. Which means that if the mail bounces for some other reason (mailbox is full, account has been closed or suspended, etc.), the person who originally sent the mail will not see the bounce, and so will have no idea that the intended recipient never got their message. The solution to this problem is to implement SRS on the remailing server. SRS munges the return address on the envelope so that in the event a bounce is generated upstream, the return address contains enough clues buried within it that the remailing server can use to determine who sent the original e-mail, and send the bounced message back to their address. It also supposedly does this in a secure enough manner that (in theory) the SRS munging scheme cannot be exploited by others to turn the remailing server into an open relay.

It turns out that Postfix does not natively support SRS, so one can’t just flip on a switch somewhere in the configuration. There are a handful of third-party solutions that exist for Postfix, though.3 I elected to use pfix-srsd from pfixtools. (Incidentally, these guys not only have a brief HOWTO on their site for installing pfix-srsd, but they also wrote an excellent description — complete with illustrations — of this very problem.)

Adding pfix-srsd to my Postfix server cured my forwarding-to-Gmail problem instantly. Immediately after setting it up, I sent myself more test messages, and they all came through. I temporarily reverted the changes and tried to send myself more test messages, and those were bounced by Google in the same manner as before.


Mail that I am forwarding to myself now contains stuff like this in the header, which gives me warm fuzzies:

Received-SPF: pass (google.com: domain of SRS0=XxXx=XX=sender.domain.com=sender-username@remailier.domain.com designates 2001:dead:beef::face as permitted sender) client-ip=2001:dead:beef::face;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of SRS0=XxXx=XX=sender.domain.com=sender-username@remailier.domain.com designates 2001:dead:beef::face as permitted sender) smtp.mail=SRS0=XxXx=XX=sender.domain.com=sender-username@remailier.domain.com

The bit that still puzzles me about this whole thing, though, is why my method of doing e-mail forwarding to Gmail worked just fine for forever and then seemingly — and without warning — became an issue almost overnight. I decided to go back and look again at the headers of e-mail messages that I had successfully forwarded to myself prior to all of this, and here is an example of what I saw:

Received-SPF: fail (google.com: domain of sender-username@sender.domain.com does not designate 2001:dead:beef::face as permitted sender) client-ip=2001:dead:beef::face;
Authentication-Results: mx.google.com;
       spf=hardfail (google.com: domain of sender-username@sender.domain.com does not designate 2001:dead:beef::face as permitted sender) smtp.mail=sender-username@sender.domain.com;

…and yet despite this, Google dutifully delivered these messages to me, didn’t bounce them back, and didn’t apply the Spam label to them. We also know that it can’t be the case that Google decided only within the past couple of weeks to start rejecting mail globally if it failed SPF. How do we know this? Because…

  1. Others who run MXes have had something similar happen to them, and just as suddenly, but they experienced this issue months — sometimes years — ago, well before my IP address began to face similar discrimination.
  2. I proved to my own satisfaction earlier that just by changing the IP address my server uses to speak to Google, Google would resume delivering mail that I forwarded to it even when I didn’t have SRS remailing enabled on my server. So the source IP address is definitely a variable.

This means that Google can’t possibly be applying the same SPF policy to all sources of mail across-the-board. When receiving e-mail from some IPs, Gmail does not necessarily take an SPF fail into account. (Perhaps this is the default policy.) But for other IPs, Google will opt to block any mail that it receives unless SPF explicitly passes (or, presumably, the DKIM signature checks out).

Why does Google treat mail from some IPs one way and mail from other IPs a different way? I haven’t been able to find a single document anywhere in which Google lays out what their exact policies are in a comprehensive manner. If I had found such a document, I could have been spared a lot of grief. It would also be nice to know what specific event caused Google to change how they treat my mail server’s IP specifically, so that if my server has been compromised, or if there is something else that I’ve done wrong that I am overlooking, I can work to correct it. The closest thing I’ve managed to find from the horse’s mouth is this. Perhaps they feel like they need to hold their cards close to their chest so that they don’t accidentally reveal to spammers any potential holes in their policies that might be possible to exploit or work around.


Ultimately, the lessons I took away from my own experience are these:

  1. If you haven’t already created an SPF record for your domain, do so.
  2. If your mail server allows its users to forward mail off-site, then implement SRS remailing on it ASAP.

As a mail server admin, you need to do these things not merely to ensure that your legitimate mail is always being delivered to Google successfully, but because Gmail has demonstrated that other mail server admins can and do enforce SPF policy selectively, and you have no idea how other mail servers might be configured to respond to SPF policy failures.


  1. e.g., http://forums.devshed.com/mail-server-help-111/gmail-blocks-my-messages-what-can-i-do-825511.html 

  2. This is not the only solution; as the OpenSPF site points out, forwarding servers could be whitelisted on mail exchanges that normally check SPF. But from a scalability perspective, it is not realistic — nor would it be terribly secure — to ask every single MX in the world to whitelist your forwarder. 

  3. In addition to the pfix-srsd solution discussed above, I found these alternatives:

    http://dozzie.jogger.pl/2013/11/13/postfix-sender-rewriting-scheme

    http://kmlinux.fjfi.cvut.cz/~vokacpet/activities/srs-milter/README

    http://groups.google.com/forum/#!topic/mailing.postfix.users/x1pMbikyFEw 

I Blog, You Blog, He/She/It Blogs

|

For my grandfather, James Louis Austin, Sr.: April 9, 1921 — December 31, 2013

"Blogs are the new Usenet!"

For many years, this cartoon epitomized my view of most blogs. My goal is to ensure that this one avoids the fate of becoming a caricature of such sites.

About

Hello there. Name’s Nathan. Nice to meet you. I’m a lifetime tech enthusiast and wannabe/amateur engineer. “Wannabe” because even though I’m employed within the computing industry, I have received no formal training, and the more times I’ve been around the block, the more I’ve come to understand just how pitifully little I actually know. This can be discouraging, but mostly it just makes me thirsty for more opportunities to acquire experience and to further my understanding and my craft.

Whenever my knowledge has come up short and been found wanting, I’ve relied heavily on the broader tech community on the Internet to bail me out. And I feel that it’s high-time that I started giving back. This site is my way of saying “thank you” to anyone out there who has taken the time to document the solutions to the technical problems they have run up against. I have directly benefited from the efforts of countless such people. I aim to publish my own discoveries and solutions here to some of my own technical puzzles, and my hope is that other people who have encountered some of the same problems that I have will find at least part of the answer here.

Comments

As you may (or, more likely, may not) have noticed, this site lacks a commenting feature. That’s not because I am actively trying to discourage dialogue and feedback from others about what I write. Far from it. If you have some thoughts about something I’ve posted here, please post it somewhere and contact me so that I can link to it!

Credits

Speaking of benefiting from the work of others, credit where it’s due: this site is powered by Second Crack (with some select patches from Continental folded in), and the theme was modified from and heavily influenced by Marco’s. Graphics were made in Microsoft Image Composer, for which I make no apologies.

New Year, New Phone

|

I am late in posting this update to my original dilemma, but after all talks with Apple fell through, I decided to just let it be and move on; after all, what else could I do?

Talks did indeed cease: right after the first of the year, nearly 3 months after I opened my ticket and after almost a month had gone by with no word from anybody at Apple, my executive relations contact finally got back to me for the last time, and told me in no uncertain terms that the engineers had informed her that they weren’t going to change anything, and as a result there was nothing that Apple could do for me. I thanked her for reaching out to me, but told her that what I was hearing Apple say to me is that they no longer valued me as a customer, and that I should switch to something else. In fact, I told her that they had, in effect, made my decision for me, which she of course denied was their intention. But what was done was done.

These days, I am using a Google/LG Nexus 4, which, despite doubt expressed by some about how honest Google’s claims about “openness” on their platform are, has at least proven to be much more open in the ways that matter to me as an end-user than iOS ever was. And at least when Google said that it was an “unlocked” phone, they used the term in the way that I expected it to be used and consistent with the way that the rest of the industry uses it, and true to their word, it works on every GSM carrier, does not fight me when I try to use it with a specific carrier (even an MVNO), and does not hand over control of parts of the phone to either the MVNO or the carrier that underlies the MVNO and lock me (or my actual carrier/MVNO) out of those same parts.

I am sad that it had to come to this, because even though the Nexus is a very good phone, all things being equal, I do prefer the iOS platform and definitely prefer the iPhone hardware. However, all things are not equal. At this point, I cannot even recommend iPhone in good conscience anymore to friends and family. So long, Apple, and thanks for all the fish.

How Apple Broke My iPhone

|

(and How Their Policies Prevent Them from Fixing It)

Update 12/18/2013: Here is the conclusion to this drama.



Hello. I’ve been an otherwise happy iPhone user since 2008. I use my iPhone with an MVNO carrier in the U.S. called Straight Talk, which runs over AT&T’s network. This is my story.

“Book report” version: iOS prevents customers of MVNOs whose parent carrier restricts on-device APN editing from being able to configure their phones to properly work with the MVNO service, even if it is an officially unlocked iPhone. There were unofficial workarounds to this (ones that did not require rooting/jailbreaking the phone) which Apple ended up patching in iOS 6 without giving customers who relied on them an officially-supported alternative. Thus, when I upgraded my phone to iOS 6, internet data and MMS broke. I was able to get internet data working again using Apple-provided tools, but MMS remains broken, and Apple cannot will not help me.

FIRST, Apple won’t help me because they agreed to block on-device APN editing for certain carriers, including AT&T, even though this negatively impacts people who are not direct customers of AT&T, and even though there is not one single other unlocked GSM phone model on the market besides iPhone that imposes this restriction on users or hands over this kind of control to carriers.

SECOND, Apple won’t help me fix this problem by means of allowing me to downgrade to iOS 5. iOS is specifically engineered to prevent downgrades on a per-device basis; see this Wikipedia article. Apple has refused to whitelist my device’s serial number on their TSS server even though we both know that allowing me to go back to iOS 5 would address my immediate need.

I have interacted with Apple on a number of levels:

  1. Opened a ticket with AppleCare support (October 16); after a week of this, I was advised by my support agent to…
  2. Write a (long) letter to Apple Product Feedback (October 22). I knew I’d probably never see a response, so a couple days later I…
  3. Wrote a letter to Tim Cook, CC: Phil Schiller, Scott Forstall (October 24). This resulted in a call back from…
  4. Apple Executive Relations, whom I have been in constant contact with since.

I also go into more detail about the history of this support case and some of the issues surrounding it in discussion threads I’ve started on a couple of web forums (linked at the top).

Although everyone I have interacted with at Apple has been sympathetic with my plight, and even though my Apple Executive Relations handler has been willing to repeatedly go to bat for me after every time we have talked on the phone, nothing substantive has changed. I don’t know what else to do other than to inform others about this, and to urge others who are in a similar position (unlocked iPhone user, and Straight Talk or other MVNO customer who can’t use MMS) to complain loudly as well. Really, anybody who has been negatively impacted by the draconian downgrade enforcement policy Apple has pushed onto iOS should be complaining.