Benchmarking the Creators Update (Redstone 2)

Published by at

As usual with my benchmarking features, I'm focussing here on real world timings, not on artificial software-based utilities. My observations at the end of 2016 were that the early builds of Redstone 2 were showing performance gains compared to the existing Anniversary Update (Redstone) on the same (or similar) hardware. What about the latest, release version of the former, i.e. the Creators Update? Is it still faster?

For the purposes of the test, I used a Lumia 950, taking it back to the Anniversary Update on a fresh install and then running it up to the Creators Uodate via the Release Preview Insiders ring (since the CU hasn't been rolled out to my 950 yet in my region (UK). In each case the times were after booting, to eliminate oddities caused by the application already being in RAM, etc.

Now, while there are minor variations in Wi-fi/Internet speed, not to mention my own data and configuration having an impact, it's still very useful to look at speed differences between the two OS branches, looking at the same real world speed tests on the Lumia 950, showing the fastest from two trial runs at the operation:

Benchmarking
(most times
to nearest half second) 

Windows 10 Mobile
Build 14393.1066
(Anniversary Update) 
Windows 10 Mobile
Build 15063.251
(Creators Update)

Booting up to Start screen 38 36
Starting the Store to see all
front screen content
3.5 4.5
Opening up Outlook Calendar 2.5 2
Launching MSN News to
opening graphics
4 5.5
Opening up Outlook Mail 1.5 1.5
Opening up Skype (UWP)
i.e. signing in after boot and
updating messages from server
4 3
Opening up Cortana
to the 'look at the day'
vertical panorama, after boot
3.5 6
Opening the full New York
Times desktop web site to
the filling in of the
right side bar content
16 12
Open Lumia Camera 1.5 1.5
Planning a route
from Reading
to Aberdeen in Maps
(with map offline and available) 
12 10
Starting up Weather to the
full opening summary
4 4
Starting a new Excel sheet 
with a template
4 2.5
Total time 94.5 88.5

So an overall speed increase of around 5% for the Creators Update (Redstone 2) over and above Anniversary Update (Redstone), though don't get too excited - 5% is also my estimated margin of error in the tests above.

Still, anecdotal evidence from my own hands-on with Redstone 2 on both the Lumia 950 range and older devices, upgraded through the Insider rings, is that the Creators Update is noticeably 'smoother' than its predecessor, and on the same hardware. I wouldn't expect users to notice a 5% speed bump, but it seems that it's not all about raw speed - the Creators Update leaves less applications hanging or loading, on average. And is helped here by the newer UWP applications and by a more capable Edge browser version.

It's true that 5% isn't the 10% or even 20% that I was sometimes seeing on earlier builds of Redstone 2, so I'm guessing that the (necessary) code added in the last few months of Insider updates has counteracted the super-speedy earlier builds. But 5% faster is still a notch up from most OS updates on most platforms, where the newer OS is almost always slower.

Lumia 930 updating

I don't expect any AAWP readers to go through the whole OS update cycles that I've endured while creating this feature, but I would welcome data points from you if your phone is now on the Creators Update (via production or Release Preview Insiders ring) - is it noticeably faster or 'about the same'? Could you verify that 'smoother' is appropriate?