Noisy Neighbor Perfect is a verb, not a noun.

Learning From Apple

I’m one of many in the industry to note that Apple had a pretty bad time last week… while it’s not the first time they’ve had security flaws in their products (for the record, I’d be more suspicious of products that “never” find any flaws – are they just not used much, or hiding something?), the embarrassment came from the nature of the more recent flaws and, more importantly to me, the interactions between a couple of those flaws.

I want to reflect on this experience to ask a rhetorical question, and promote a particular practice I’ve been diving into over the last couple years. However, before going there, I want to state a few things so I don’t just come off like an Apple basher.

  • I’ve used Apple in some form for over a decade, having owned a Mac Classic, Mac Pro workstation, a handful of MacBooks, and converting from Android to iPhone (mostly because of the fractured marketplace and inconsistent update model for Android devices).
  • I’m a UNIX geek at heart (many years saw me running BSD variants on PC hardware), and the love child of BSD, Mach and NeXT producing what we now lovingly call Darwin, OS X or macOS combined with Apple’s expertise at hardware engineering produced what I consider the almost-perfect developer laptop (the only way it could be better is with fully open hardware, but this is an industry-wide vs Apple problem).
  • In general, I trust and respect Apple’s commitment to security and privacy. Anyone not living under a rock in recent years has seen this in action. They are usually great at fixing issues quickly, and doing so consistently across a diverse product line.

With that out of the way, there have been several incidents which are out of character for a company of Apple’s size… These are even more embarrassing given their commitment to security, and the fact their increasing market share clearly implies any issues they encounter have a broad impact on the global community. Here’s a brief tour:

  • While typically fast, Apple has occasionally taken years (yes, YEARS) to fix highly-publicized, “critical” vulnerabilities reported by researches. Consider the 3+ year MTTR for the (in)famous finfisher trojan.
  • More than an anomaly, careful study of Apple’s patch history reveals a startling trend… while there is a seemingly constant flow of “updates” to my iDevices, Krebs on Security found that on average 91 days elapsed between the date that a security researcher alerted Apple to an unpatched flaw and the date Apple shipped a patch!
  • Timing aside, the very nature of the most recent vulnerabilities has been shocking. The most surprising was the root login vulnerability which was the easiest root exploit I’ve ever seen.
  • OK, OK…but apple only took 18 hours vs 3+ years to fix that one. Phew. Unfortunately, anyone who applied the fix without already having installed macOS 10.13.1 had the fix reverted when they upgraded. Others reported, even when running 10.13.1, the fix didn’t work until reboot. Oh, and there was no indication a reboot was required… and the update broke file sharing. Yeesh.
  • Other notable vulnerabilities this year include nasty password stealing and giving away your passwords for free.
  • Beyond this short list, the increasing reliance on biometrics (despite security expert’s consistent warnings about the down-sides) raises privacy concerns. While Apple has assured us the technology is mature enough to be trusted and our immutable biometric signatures are safe by investing in Secure Enclaves and Differential Privacy, recent media coverage says otherwise.

With a few minutes in your favorite search engine, you’ll find a lot more to be worried about. I’ve been trying very hard to write blog posts rather than academic theses so I’ll wrap up with a statement (because hey, this is my blog where I get to be opinionated) and the promised question…

While the above is certainly worrisome, having vulnerabilities isn’t the problem. I expect any innovative and heavily used product to have lots of vulnerabilities found. The issue is response time and effectiveness of patches. If either of these suffer, users are at serious risk. If Apple is truly committed to security and privacy, they need to do better on both fronts. I think it’s time Apple pause feature development for a moment, do a quick retrospective on recent events, and devote more time to improving security and stability. Think Snow Leopard (a OS X release dedicated to stability and optimization). While it’s easy to say features trump everything else, and technical debt should just be paid down as you go… security is a much-touted feature (not just an after-thought!), and it’s clear the debt has been accumulating. We need more Snow Leopards.

My question is… does Apple practice TDD (Test Driven Development)? Unfortunately, I’ve fallen out of regular contact with the few folks I know working for Apple today so my direct questions have gone unanswered. I wasn’t as thorough as I’d like, but some comments around the Internet imply use of TDD is quite limited if practiced at all. I highly respect Terry, and the fact that post is as much his opinion as Apple’s, but also disagree with the comments “TDD is more applicable when there are standards bodies driving adherence to international standards” and “BDD is pretty useless.” I think both of these show the need for better understanding of TDD.

In his seminal work on TDD, Test Driven Development By Example, Kent Beck talks about the “no time for testing death spiral” when pressured to deliver features – the more stress you feel, the less testing you do. The less testing you do, the more errors you make. The more errors you make, the more stress you feel.

Some have mixed feelings over TDD (debates often get more emotional than fact based), and while nothing is a panacea, I honestly don’t see how a company of Apple’s size, trying to innovate at an increasingly frantic pace, with a ballooning user base ensuring greater impact from software defects, can operate sanely without strict adherence to TDD principles.

As a loyal customer and security advocate concerned over recent events, I sincerely hope best practices like TDD and pair programming are embraced across Apple’s engineering department. With most of my professional life and PII housed on Apple devices in one form or another, my future quality of life depends on it.