In the mix here were the Nokia Lumia 830, 930, 1020 and 920, the first two representing the newest x00 series of processors and the latter two the older S4 units. See the table below for more. I did try GFXBench on lower-specced devices like the 512MB RAM Lumia 530 but it wouldn't run beyond its first warning message about resources needed. Ah well - there's little point in benchmarking at this low end anyway - there, it's all about price point.
Here's blurb on GFXBench, to start with:
GFXBench is a free, cross-platform and cross-API 3D graphics benchmark that measures graphics performance, long-term performance stability, render quality and power consumption with a single, easy-to-use application. Using console-quality 3D content, GFXBench provides tools for cross-platform 3D graphics benchmarking on all major desktop and mobile platforms (Windows Phone, Windows, iOS, OS X, Android).
GFXBench DirectX is a suite of high-level and low-level 3D graphics test that measures the performance of DirectX 11 (feature level 9 and 10) implementations on various Windows devices (Windows Phone 8.0/8.1, Windows RT, Windows 8.1).
The main UI, in which you can run any or all of the graphics tests, plus compare scores for any test to those from hundreds of other devices. (right) There's a hefty 50MB download of resource and data files - this seems to be mandatory as you can't continue without this 'sync'...
Two of the graphics demos, screenshotted.... they're mostly quite pretty to watch and give a good indication of device speed for games, for example...
So there's a heavy focus on graphics speed, though of course processor speed and RAM also come into the equation here. Regardless, I thought the utility might be very interesting to use to answer questions about the relative performance of the various recent generations of Windows Phone. The restriction to only working on devices with 1GB or more of RAM (and preferably 2GB) is a bit of a pain, but there's still plenty of data that's worth highlighting.
As usual with tables on AAWP and issues viewing them on Windows Phones because of the 'responsive design' and quirks in IE, the table is also available here as an image, gfxtable.jpg.
NB. With the GFXBench stats, more is better, if you see what I mean, e.g. successful frames rendered. With the Sunspider javascript benchmark, included here as an extra (more processor-centric) data point, less is better (representing time in milliseconds to complete the requested operations).
| Lumia 920/925 | Lumia 1020 | Lumia 930/1520 | Lumia 830/735 | |
| Snapdragon processor/chipset | S4 (dual core) 1.5GHz | S4 (dual core) 1.5GHz |
800 (quad core) 2.2GHz | 400 (quad core) 1.2GHz |
| Graphics chipset | Adreno 225 | Adreno 225 | Adreno 330 | Adreno 305 |
| RAM | 1GB | 2GB | 2GB | 1GB |
| Display resolution | 768 x 1280 | 768 x 1280 | 1080 x 1920 | 720 x 1280 |
| GFXBench T-Rex rendering (frames) | 391 | 380 | 823 | 427 |
| GFXBench Alpha Blending (MB/s) | 1470 | 1414 | 5896 | 1283 |
| GFXBench Fill Texturing (MTexels/s) | 496 | 461 | 2363 | 549 |
| Sunspider javascript (ms) | 909 | 942 | 569 | 1244 |
Interesting results then, with a few baseline conclusions.
- The Lumia 920/925 and the 1020 should be almost identical, but there's a significant difference in favour of the older devices, perhaps because of the slightly non-standard configuration for the graphics chipset on the 1020? Or could it be that the extra Gigabyte of RAM is being used, but with a slight performance penalty while all the bits are shuffled around? The difference isn't huge, but I thought it worth noting.
- The most interesting comparison here is for the older S4 devices against the newer but mid-range Snapdraon 400 devices, the Lumia 830 (and 735). Anecdotal user experience is of a pretty smooth ride on the 830, though the Sunspider browser test shows that the raw processing power is actually around 30% down on the S4 devices like the 920, 925 and 1020. However, the GFXBench metrics are, on average, slightly higher, probably a result of the newer graphics chip, so it's swings and roundabouts overall. I still equate the general interface and app speed (including games) of the Lumia 830 (and 735) to those of the previous year's flagships. Progress, eh? - such is life!
- Unsurprisingly, the Lumia 930 (and, by extension, the 1520, which should produce identical results) produces a graphics performance which is dramatically better, armed with its much more capable graphics chip and faster processor. When watching the animated mini-movie with the motorbike and the T-Rex, only the 930 was watchable - all the other test devices here produced very jerky, stumbling footage. The use of the Snapdragon 800 also means a browser speed that's almost twice as fast. As you'd expect.
However, it's worth also noting that GFXBench uses 1080p test creations explicitly - in reality, i.e. in day to day use, the 720p and 768p devices here have to drive far less pixels and compute far less vectors than when running the utility. So playing a game on a Lumia 830, for example, shouldn't be too different an experience than playing it at higher resolution on the 930. (Not that every game title is optimised for 1080p, in fact far from it!)
Using the 'Compare' feature in GFXBench is a little misleading, since there are a vast number of Android smartphones listed that use the Snapdragon 800, 801 and equivalents from other chip makers, most of which which come in with better scores than most of the Lumias listed. However, it has been evident from the beginning that Windows Phone makes better use of processors and chipsets than Android. For example, the Windows Phones based on the Snapdragon 200 are still pretty fluid in their interface and animations, where the Android equivalents are starting to lag noticeably. So take the cross-platform comparisons with a pinch of salt.
My aim here was to use an available tool to provide data comparing some of the best known Windows Phone options. It's a pity that the lower specced 512MB (RAM) devices couldn't be included, but there's hopefully still plenty of interest above.