'SteveMark', the Lumia 950-normalised scores!

Published by at

Having used the classic Lumia 950/XL phone camera as the baseline for almost forty (count 'em) imaging articles here on AAWP/AAM in the last half decade, with pixel by pixel comparisons against a wide spectrum of smartphones from all quarters, it occurred to me that it would be fun to do the maths and generate some deltas and some averages. With my article-by-articles scores, which are/have been the best smartphone camera systems of the last five years? 

smartphone imaging

Now, there are some huge caveats here, of course. Not least:

  • test shots/lighting will have varied from comparison feature to feature.
  • the number of cameras on a smartphone increased during the period - I typically included the zoom elements, but rarely any ultra-wide lenses, since the baseline Lumia 950 only had the one lens (as was normal back in 2015).
  • zoom testing will have varied throughout, depending on the zoom capabilities of the featured phone(s).
  • some phone cameras have appeared in multiple features (e.g. Pixels or iPhones), while others only had the one comparison, so one data point.
  • where phones have featured in multiple features, it's often because later testing was with newer, improved camera software.
  • some features were larger than others in terms of number of test scenes, so the 'points deltas' would have been more or less significant.

Having said all of that, the general format of my imaging comparisons here on the 'All About' sites has been fairly consistent, with test scenes typically including landscapes in sunlight and at night, zoomed and unzoomed, and with plentiful low light and close-up shots as well. So I think the methodology below is appropriate.

For each test scene in each feature, I award points out of 10 for each phone camera shot, taking into account colour accuracy, digital noise, and pixel purity. And at the end of each imaging comparison, I add up the totals to produce scores out of (typically) 100 (or so). Now, while I can't include actual scores, since these will vary according to the exact number of test scenes, I can include the 'delta' of each to the baseline, the titular Lumia 950(/XL). i.e. How much better (or, more often, worse) were each of the test phone cameras than the Lumia 950(/XL) in that feature's set of test scenes?

Where I tested a phone camera system more than once, I've been able to average its delta over the Lumia 950, leading to (hopefully) a more accurate score. Note that I've excluded any 'ultra wide bonus points' in the tallies, something that I've occasionally added in to feature tallies.

Here then are the results - there are a few scoring surprises, but I'll comment on these below:

+13
  • Huawei P40 Pro
+10
  • Huawei P30 Pro
  • RealMe X3 SuperZoom
+9
  • Apple iPhone 11 Pro Max 
+7
  • Apple iPhone 12 Pro Max
  • Google Pixel 4/XL
  • Samsung Galaxy S20 Ultra
 +6
  • Xiaomi Mi 11 5G 
 +5
  • Samsung Galaxy S21 Ultra 
 +4
  • Samsung Galaxy S20 FE 
  • OnePlus 7 Pro
 +3
  • Sony Xperia 1 ii
  • Redmi Note 9 Pro
  • Google Pixel 3/XL 
 +2
  • Xiaomi Mi Note 10 
  • Samsung Galaxy S9+
 +1
  • Apple iPhone 12 
  • Poco X3
 Level
  • Sony XZ2 Premium 
  • OPPO Find X2 Pro
  • Huawei P20 Pro
  • Samsung Galaxy S9
 -1
  • Samsung Galaxy Note 9 
 -2
  • Poco F2 Pro
  • Apple iPhone SE 2020
  • Samsung Galaxy S20
  • OnePlus 6
  • Huawei Mate 10 Pro 
  • Huawei Mate 20 Pro
  • Samsung Galaxy S10e
  • Google Pixel 4a
  • Google Pixel 3a/XL
  • Google Pixel 2/2XL
 -3
  • LG G7
  • Google Pixel 5/4a 5G
  • Apple iPhone Xs/Max
  • Nokia 7 Plus
  • Sony Xperia X5 ii
 -4
  • Asus Zenfone 5 
  • Redmi Note 10 Pro
 -6
  • BlackBerry Key2 
 -7
  • LG V50 
  • Honor 10
 -10
  • Xiaomi Mi 9T Pro 
 -12
  • Nokia 7.2 
  • Samsung Galaxy Note 10+
-13
  • Motorola One Vision
  • Sony XZ1 Compact
-14
  • Redmi Note 9T
 -15
  • Nokia 9 PureView 

Do also note that there are a few smartphone heavyweights not included in the table because I've not been able to test them, hands-on. During the last 18 pandemic months, review hardware to UK-based journalists has somewhat dried up - which is odd, since you'd have thought that mailing over devices would have been easier than before? The bottom line is perhaps that with no physical events it's been harder to make or maintain good press contacts. Oh well.

Some surprising conclusions then:

  • I'd forgotten just how many smartphone camera systems had beaten the mighty Lumia 950 - in my mind it's still the champion phone camera from four years ago and, as at June 2021, no less than 18 phone cameras that I've tested in detail have scored more highly. Admittedly this is often partly because of better zoom, but the number still shocked me. Maybe I need a new baseline phone camera from 2021 onwards? The iPhone 12 Pro (/Max) in ProRAW (i.e. ultra-pure) mode would seem to be the logical choice.
  • The top three are there because of huge periscope zoom lenses - great if you need high zoom factors (e.g. to spot wildlife) but not necessarily useable in everyday life, as I've mentioned many times, 2x and 3x are actually more useful.
  • Google's Pixels peaked with the 4 series, with the last year's mid-range 4a and 5 series offering worse image quality. Which is odd, since specs are the same, but my gut feel is still that Google went for cheaper lenses. A shame, but rumour has is that this year's Pixel 6 will rectify all this and restore Google to near the top of the phone imaging tree.
  • The iPhones at the top are in reverse order, with the 12 Pro Max lower than the 11 Pro because these are average deltas and the former was tested in both default and ProRAW modes. Default processing is populist, but ProRAW mode is much purer. 
  • Right down at the bottom of the list, in ignominious last place, across all prices and tiers, is the Nokia 9 PureView - the imaging flagship that HMD Global point blank refused to send me for review because they knew I would tear it apart. So I borrowed one and did this anyway. All very sad.

Your comments welcome, on the SteveMark deltas. I contend that imaging on any phone marked in green will be decent and worth considering if you're hunting around for a new or second hand smartphone. Perhaps you own one of the devices above - do you think the scoring looks fair?