Comparing phone performance with Rightware's Basemark OS II

Published by at

Rightware has released Basemark OS II, a benchmarking tool that aims to measure the overall performance of a smartphone. The benchmarking app is available across multiple smartphone platforms (Android, iOS, and Windows Phone), thus allowing you to compare the benchmark results of devices from different manufacturers. 

Benchmarks should be treated with some caution as they show generic theoretical performance and the results can be effected by numerous factors, including how well the benchmarking software is optimised for a given platform. In addition the results, and more importantly comparative results, are rarely directly proportional to real world usage performance differences (i.e. a benchmark may indicate a device is twice as fast, but the difference is real world usage is usually much smaller).

Here's how Rightware describe the benchmark:

The benchmark features a comprehensive suite of tests and produces an objective overall score as well as a breakdown of each tested area, including system, internal and external memory, graphics, web browsing, camera, battery and CPU consumption.

The chart below shows the results for several Windows Phone devices: Lumia 1520, 925, and 620. The results are in line with the core specifications of the respective devices.

Basemark results

While benchmarks, including the Basemark results above, show a substantial advantage for the Lumia 1520 that does necessarily translate proportionally into real world usage, as we noted in our in depth review of the device:

In terms of real world usage, the new hardware doesn’t have a big as immediate impact on Windows Phone as you might think. There’s no discernible difference in opening or resuming the built in apps (e.g. Messaging) and general UI interactions don’t feel any faster. On the other hand, the Lumia 1520 does boot up from a cold start about 5 seconds faster the Lumia 925. Similarly, third party apps do open a little quicker, when compared to the Lumia 925, but the difference is usually less than half a second.

To be fair, the performance differences are sometimes more apparent for resource intensive apps, such as PhotoSynth, but it says something that we had to go to some effort to find examples of this. 

Basemark OS II is a good example of a resource intensive apps that does show a performance difference. The extra capability of the Lumia 1520 also makes it a more future proof device, as befits a hardware family (Snapdragon 800) that will be used in many of the high end Lumia models released this year.

The chart below shows the overall Basemark OS results for a range of leading smartphones. As with the Windows Phone results above there's a clear distinction between the different generation of chipsets, with the quad-core equipped LG Nexus 5, Sony Z1, and Lumia 1520 all scoring similar results. For Windows Phone users the interest here will be that the new hardware generation, as  represented by the Lumia 1520, is comparable to the leading smartphones from other platforms.

Basemark 2

Windows Phone Store description:

Basemark OS II is a cross-platform "all-in-one" benchmark that evaluates the overall performance of any Android, iOS and Windows Phone 8 smartphones and tablets for easy comparison. The benchmark covers a comprehensive suite of tests and produces an objective overall score as well as breakdown of each tested area, including system, memory, graphics, web browsing and camera.

Basemark OS II Free is a free download from the Windows Phone Store