Are Android upgrades actually improving? It's complicated

There's a popular narrative in tech media right now that Android upgrades are really, truly getting better — that for all the time device-makers take to send out updated software, they're nevertheless improving and making strides toward a more reasonable standard.

I've been tracking Android upgrade performance closely for years now, and I'm hesitant to buy into such a narrative — especially when it seems to be based mostly on limited-scope, anecdotal perception and maybe the occasional marketing department nudge. So now that my Android 10 Upgrade Report Card is complete, I thought it'd be interesting to take a careful look at the actual data surrounding Android upgrades to see what the cold, hard numbers reveal.

The answer, in short: It's complicated. Let's break it down, shall we?

Android upgrades: All flagships are not equal

One thing folks frequently forget when talking about the state of Android upgrades is that there are two separate fronts we have to consider — the current-generation flagship phones, at the time of an upgrade's arrival, and then the previous-gen flagship phones alongside 'em. As a general rule, Android device-makers commit to providing updates for two years to top-tier devices, which means those year-old phones are every bit as important as their more recent siblings. But since they tend to be less top of mind in terms of mainstream media attention, they also tend to be where companies cut some disconcerting corners.

We'll start with the relatively positive news, though, specific to those current-gen flagships: On that front, with Android 10, we saw some genuine, heartening improvements from a few different sources.

Take a look for yourself. This first chart shows the total number of days from the time of an Android OS update's release to its first appearance on a company's current-gen flagship — from this current cycle all the way back to the Android Lollipop launch in 2014. (As a reminder, my data focuses specifically on U.S.-based versions of devices. That gives us a consistent metric for comparison and allows us to assess how quickly a typical U.S. device-owner could get the software in a typical situation. You can read more about my methodology here.)

JR

So what do we see there? Well, the company that deserves the most credit in terms of sheer improvement is OnePlus. It went from a 138-day delivery day with 2017's Oreo release down to 47 days with last year's Android Pie software and then a mere 18 days with this year's Android 10 update. That's an exemplary arc showing exactly how Google wanted update delivery processing to improve as a result of its Project Treble optimizations for Android — a series of steps implemented in 2018 to separate the software into multiple layers and make the updating process easier to pull off. 

(2017 was the first year OnePlus was included in my Report Card, by the way, which is why you don't see any data for it in the previous cycles.)

We'll talk more about Treble in a moment, but first, two other players show noteworthy improvements in their current-gen flagship upgrade performance: Samsung, which shaved its delivery time down from 213 days with Oreo to 177 days with Pie and then 106 days with Android 10 — and LG, which for the first time ever got a bit better this year by going from 298 days for last year's upgrade to 129 days for this one.

But, well, when you look at the big picture, you quickly realize that neither of those is actually that grand of an evolution. First of all, taking 100-plus days to get a major operating system update into the hands of a company's highest-paying customers is in and of itself nothing to celebrate. But even relatively speaking, Samsung is simply returning to the level of ho-hum performance it showed back in 2014, with Lollipop. And LG isn't even quite touching its 2014 level of mediocrity.

Neither company is achieving new ground, in other words; it's just coming down from an embarrassingly bad four-year run and going back to the same underwhelming performance it pulled off five years ago. So does that really qualify as "getting better"? I guess it depends on your perspective.

As for HTC and Motorola, neither has delivered an Android 10 update to any current-gen flagship yet, so it's impossible to say anything concrete — but even if one or the other does manage to do a bit better than last year, the standard for improvement is so laughably low that it won't mean much. And taking more than six months to send software to users is certainly nothing to write home about, regardless.

Google, meanwhile, has basically been consistent, with little real room for improvement.

Again, though: That's only half the story.

The previous-gen flagship complication

When we zoom out further to look at the time it takes Android device-makers to get new, major OS releases onto their previous-generation flagships — phones that frequently cost close to a thousand dollars and are only a single year old — an even messier reality emerges.

Here's the data:

JR

In this scenario, OnePlus again saw some significant improvements — jumping from 142 days with last year's Pie update to 93 days with this year's Android 10 delivery. But complicating matters are two things: First, in 2017, OnePlus also took 90-some-odd days to deliver the year's Android update to its previous-gen flagship. That makes this year's improvement seem less like a new accomplishment and more like a bounce back from a random off-year. And second, a quarter of a year is still simply too damn long to keep people waiting for current software, even on a year-old device.

Samsung's story, meanwhile, is basically the same as in the current-gen flagship domain: The company managed to lower its delivery time from last year's 233-day window down to 147 days this year. But that's not even quite matching its performance from 2014, with Lollipop — and, I mean, c'mon: Nearly five months for a major OS update on a year-old top-tier device? There's just no way that can be seen as acceptable, let alone commendable.

As for everyone else, LG, Motorola, and HTC have all yet to send Android 10 to any previous-gen flagships, while Google again has been relatively consistent across the years, albeit with a little more fluctuation along the way.

So what to make of all of this? There's one more factor to consider, and then we'll pull the various pieces together and sum it all up.

Time to talk Treble

So back to Project Treble. Treble, at its core, is all about making it faster, easier, and cheaper for phone makers to process Android software updates and get them out to users. And 2018's Android Pie release was the first time its advancements were complete and operational — with chipset vendors ready to support the system and a significant number of Treble-ready devices out in the wild. So 2018 should have been the year we saw significant upgrade delivery improvements across the board as a result.

We actually have a metric to measure that success, too: Ahead of Android Pie's release, I had the chance to interview Iliyan Malchev, a principal software engineer at Google and a key architect of the Project Treble program. One of my most pressing questions for him was exactly how much of an impact Treble ought to have on the Android upgrade delivery process, from a real-world perspective. And his answer was unambiguous.

The various improvements connected to Treble were expected to shave about a quarter of a year off the upgrade process, Malchev told me. That's how long the lower-level code updating typically took, in his estimation — the part of the process Treble made unnecessary — and so without that work in the picture, device-makers were essentially getting a three-month jump.

By and large, though, we didn't see a three-month improvement in update delivery times with the Pie update — not when it comes to the companies and products most relevant to U.S. users, at least.

So what about with Android 10? Well, things are absolutely a bit better — but can we really attribute that progress to Treble and consider it a result of Google's structural improvements? I'm not entirely convinced. On the one hand, from 2017's Oreo rollout to this year's Android 10 process, Samsung did shave just over a hundred days off its current-gen flagship delivery, which is indeed close to a quarter of a year (a hair over it, in fact). But at the same time, it seems slightly disingenuous to credit that change to Treble when Samsung managed that same 100-ish-day delivery time way back in 2014, before Treble even existed.

The same can be said for LG, only that company still isn't quite back to its 2014 level of mediocrity. So can we really say Treble is responsible for allowing LG to do almost as poorly as it did six years ago? It seems like a stretch.

And then there's the previous-gen flagship front, where Samsung again almost (but not quite) matched its underwhelming 2014-level performance — and LG is currently 42 days worse than its 2014 effort and counting. Again, it's hard to square those realities with any sort of "Treble is saving the day" narrative. And that's to say nothing of the disastrous downward spirals HTC and Motorola are exhibiting, though one could certainly contend other factors are at play on those fronts.

Ultimately, the only conclusion I can reach is the same one I've been saying for years now: The companies responsible for the expensive, professional devices we're buying and relying on can and should do better at keeping them up to date. But much as we may like to, we can't control how those corporations prioritize post-sales software support — and clearly, Google can't, either.

The best we can do is make ourselves aware of their tendencies, decide how much their successes or failings in that department matter to us, and then use that knowledge as ammo to power our future phone-purchasing decisions. (And if you think OS updates are relevant only for surface-level polish and front-facing features, by the way, you're missing the bigger picture.)

The info is officially in your hands, my compadre. Now it's up to you to decide what to do with it.

Sign up for my weekly newsletter to get more practical tips, personal recommendations, and plain-English perspective on the news that matters.

[Android Intelligence videos at Computerworld]