The question is somewhat academic, since you have to work hard to get to these versions, i.e. we're well into geek territory. Buy hey, we're all interested in the results, aren't we? And yes, I promise I'll get back to current devices very shortly after a week of 'retro'(!)
Based on my previous feature, and using some data from it, for this comparison, I used the S4-based, three year old Lumia 1020 and Lumia 920 (with the same core chipset):
- on Windows 10 Mobile 10586.586 (yes, a slightly new build exists, but it's no longer accessible to my Insider-updated Lumia 920)
- on Windows Phone 8.1, the official latest production OS for the device
- on Windows 10 Mobile 14393.67, the latest Fast Ring Insiders build, currently on the Redstone branch
Some notes:
- As usual, I've concentrated mainly on real world operations, I don't believe that processor and GPU-bound benchmarking utilities tell a complete story - what matters is how fast the phones are when being used for real.
- The timings also assume that each app wasn't previously open or 'tombstoned', i.e. we're looking at cold start timings on the whole.
- I do include a typical benchmark figure at the bottom of the table.
- In each case, I timed the operation several times and picked the fastest.
- All applications were latest versions, updated in the Store, of course.
See the asterisked notes for clarifications to exactly what I was testing and see the comments below for what all this means, etc.
|
Benchmarking as at 10 Aug 2016 |
Lumia 920 with Windows 10 Mobile Build 10586.586 |
Lumia 1020 with Windows Phone 8.1 Update 1 |
Lumia 1020 with Windows 10 Mobile Build 14393.67 |
| Booting up to fully populated Start screen | 48 | 32 | 54 |
| Starting the Store to see all front screen content |
10 | 4 | 10 |
| Opening up Outlook Calendar | 3 | 1.5* | 3 |
| Launching MSN News to fully populated opening graphics |
7 | 2.5 | 8 |
| Opening up Outlook Mail to my Gmail a/c | 4 | 1** | 6 |
| Opening up Skype i.e. signing in after boot |
10 | 5 | 17+ |
| Opening up Cortana to the 'look at the day' vertical panorama, after boot |
7 | 5*** | 8 |
| Open Lumia Camera | 3 | 3 | 9 |
| Planning a route from Reading to Aberdeen in Maps |
9 | 5**** | 17 |
| Starting up Weather to the full opening summary |
7 | 3 | 7 |
| Starting a new Excel sheet with a template |
4++ | 4 | 3++ |
| Total time | 112 | 66 | 142 |
| Benchmark: PeaceKeeper universal browser test (higher score better, of course) |
279 | 271 | 268 |
* Just 'Calendar', of course, under 8.1
** Just named 'Mail', under 8.1
*** A more limited view under 8.1
**** HERE Maps, under 8.1
+Skype Preview UWP under W10M Redstone
++ 'Office Mobile' under Windows 10, of course
As with my original conclusion, slightly edited here:
...there's a very significant difference between Windows Phone 8.1 and Windows 10 Mobile 'Threshold' - a 70% increase in time taken if you add it all up. And if you take away the boot time (assuming that people don't restart their phone every morning) then it's much closer, for the day to day operations, to a ratio of 2:1. Now, it's true that many operations on a smartphone will run at the same speed, such as when an application is on-screen and the user is interacting with the UI, but I don't think the operations listed above are that outrageous, and are intended to be real world tasks.
For you and I, riding the wave of Windows 10 Mobile and the Insiders Preview programme, we're just happy that everything (mostly) works, are enjoying some grown up Office, mapping and communications applications - so what if the phone runs 63% slower? Perfectly acceptable, and hey, maybe it'll get tuned and optimised in time (yeah, I know, the eternal optimist)? But what about the man in the street?
I then launched into an editorial about how the latter would be utterly unimpressed by a phone which had effectively slowed to half speed after the update - and Microsoft obviously agreed, since the S4-based x20 Lumias all got omitted from the official Windows 10 Mobile upgrade list.
Had Microsoft allowed these older phones to upgrade (officially) then we'd have been in even more trouble when the time came for this 'Anniversary Update' (i.e. Redstone), which is rolling out to all Windows 10 Mobile devices. Look at the timings above, after taking away boot timings, the overall application experience is nigh on three times slower than under Windows Phone 8.1.
Wow. Not that it's stopping me playing with the older hardware on the various platform versions, thanks to a little Insider and registry hacking, which is why I'm able to report on all this.
Threshold to Redstone?
However, the most commonly asked question following my recent adventures into Redstone territory was "How does performance improve or suffer compared to 'Threshold'"? And it's this that the table above answers. Look at the timing totals, with the Redstone-powered Lumia being significantly slower overall than under Threshold. What's interesting though is that in general use the two OS versions feel about the same. In fact, if anything parts of the UI feel smoother under Redstone, as you might expect given the extra work put into it by Microsoft.
As you can see for yourself, the main slowcoaches under Redstone for the older phone are Maps and Skype. The latter I can understand - this is the new 'Preview' UWP version, probably still has debug and unoptimised code inside, and so on, but route planning in Windows 10 Maps is a mystery. As far as I can see it's the same UWP app on the same processor - so perhaps there are some Redstone-specific optimisations designed for the newer phones and which just don't work as well on the old S4 processor? All very curious!
PS. I included one benchmark, the oldish but respected PeaceKeeper, which runs on the web, even in ye olde Internet Explorer under Windows Phone 8.1. It wasn't very conclusive though, aside from perhaps showing that Microsoft Edge under Redstone is now pretty well optimised and fastest by a small margin.