SteveMark? Two completely different smartphone imaging Top 10s!

Published by at

By popular request, and in particular response to the arguably incomplete results coming out DxOMark in the last few years, I immodestly present what has come to be called 'SteveMark', as in a compilation of results of the real world tests I've been doing with a wide range of smartphone flagship cameras. Is there a top dog? Unsurprisingly, it depends on what you want from a phone camera, and also unsurprisingly, I present to you not one, but two Top 10 lists. Take your pick!

There are, in fact, several criteria by which a phone camera could be judged:

  • Absolute image quality and resolution
  • Zoom facilities (or wide angle)
  • Speed of focus and shot to shot capture times, important if trying to shoot people or pets
  • Video capture quality, stability and resolution
  • Extra features (e.g. 'portrait' mode, panoramas)

For 'SteveMark' here, I'm just going to be looking at the first three - video capture is almost another feature in its own right, while extra features are often gimmicks to anyone who doesn't actually use them day to day.

I'll be awarding marks and scores in the tables below, based partly on the points scores in all the phone camera comparisons done on this site over the last decade, and partly on review material for my cross-platform Phones Show.

If you want to check out some of the higher profile phone camera comparisons here, see (taking a deep breath, and in reverse chronological order):

Go back through the AAWP archives and you'll find another 30 or so such comparisons. Phew.

Notes:

  • The very nature of assigning scores and numbers to phones and their functions means that a table will be needed below. Hopefully most people will be able to see the whole thing!
  • All scores are for automatic/default software and settings. Of course most of these devices have a 'Pro' mode, but any amount of fiddling with this (and sometimes a tripod) are ways to achieve almost anything, so there's no way to really compare these.
  • I've parenthesised the Nokia 808 PureView below - despite my fondness for it, it can't be used as a smartphone in any practical way in 2018. But I knew I'd get asked what it scored on my scales, so it's here in brackets(!)
Phone
(all scores out of 10)
Year of launch Image
quality/
purity 
Zoom
capability
Dynamic
range
Speed
of focus
Shot to
shot time
Special features? 
(Nokia 808) (2012) (10) (9) (3) (3) (3) (PureView zoom)
Lumia 1020  2013  10 9 5 2 1 Reframing after the fact, PureView zoom
Lumia 950/XL 2015  9 3 8 4 5 Altering flash balance after the fact
Apple iPhone 8 Plus 2017 7 7 7 8 10 Extra 2x telephoto lens
HTC U11 2017 7 4 5 8 9  
Google Pixel 2/XL 2017  8 4 9 8 9
Apple iPhone X 2017 8 7 7 8 10 Extra 2x telephoto lens
Samsung Galaxy
Note 8
2017 7 7 9 10 Extra 2x telephoto lens
Nokia 8 (2017 model) 2017 6 4 4 9 9 Zoom hybrid from two sensors
Sony XZ1, XZ2 range 2017/8 3 4 3 10 10  
Huawei Mate 10 Pro 2017 8 3 7 10 10 Zoom hybrid from two sensors
Samsung Galaxy
S9
2018 7 4 7 9 10  
Samsung Galaxy
S9+
2018  7 7 7 9 10 Extra 2x telephoto lens
Huawei P20 Pro 2018 5 10 8 9 10 Extra 3x telephoto lens, 5x 'hybrid' zoom

With the raw, amalgamated relative scores in, I'd like to present two lists for consideration under 'SteveMark'.

The absolute best, most flexible, and purest...

Firstly, image quality - how good are the photos taken? Ignoring any usability and speed/performance issues, how good are the images themselves? For example, when looking back on favourite images, cropping in, printing them, sharing them on, does it matter later if the capture process took a little longer? And I factor in zoom here because you can't always get close to your subject physically and there are many times when zoom can help you get shots you wouldn't normally get at all.

So, with the 'count' ignoring the Nokia 808, which remains parenthesised, here's my first 'purest' SteveMark Top 10:

1 Lumia 1020: 24 pts
2 Huawei P20 Pro: 23 pts
3 Apple iPhone X: 22 pts
(Nokia 808): 22 pts
4= Samsung Galaxy S9+: 21 pts
Google Pixel 2 XL: 21 pts
Apple iPhone 8 Plus: 21 pts
Samsung Galaxy Note 8: 21 pts
8 Lumia 950/XL: 20 pts
9=
Samsung Galaxy S9: 18 pts
Huawei Mate 10 Pro: 18 pts


Now that's a Top 10 you won't see at DxOMark etc. And it confirms once again that I still rate the Lumia 1020 as tops overall - if you discount performance. 

Lumia 1020

...and now factoring in performance and usability

Secondly, and this time factoring in what happens at capture time, let's try and give a more complete picture (pun intended!). Note that faster focus and faster shots partly saves you time in your day, but also partly lets you take more photos, with more accurate focus, more quickly - which, for sports, pets, kids, etc. can be vital.

For this second and 'overall' SteveMark Top 10, the one that perhaps matters for day to day photos by those (ahem) less patient than me(!), I'll add up the points for all five columns:

1 Huawei P20 Pro: 42 pts
2= Apple iPhone X: 40 pts
Samsung Galaxy S9+: 40 pts
Samsung Galaxy Note 8: 40 pts
Apple iPhone 8 Plus: 39 pts
6= Google Pixel 2/XL: 38 pts
Huawei Mate 10 Pro: 38 pts
8 Samsung Galaxy S9: 37 pts
9 HTC U11: 33 pts
10 Nokia 8: 32 pts

P20 Pro cams

At which point, fans of PureView, of AAWP will be crying foul, since this is an all 'Android and iOS' Top 10. That the Huawei P20 Pro comes out ahead (thanks mainly to the 5x hybrid zoom and sheer speed), despite its flawed image processing is notable - the algorithms will apparently be fixed in a firmware update in June, at which point it will pull further ahead.

That the Lumia 1020 and 950/XL aren't in the Top 10 at all shows how far Windows-powered hardware has slipped behind the cutting edge in terms of the user experience for imaging. 2018 flagships have:

  • ISPs and imaging electronics which can routinely shoot multiple exposures and then combine them, incredibly fast
  • amazingly fast auto-focus, either 'dual pixel' or laser, typically. Complete with motion tracking of selected objects (e.g. a pet or child), keeping them in focus.
  • shot to shot times which are so fast that they can't be measured - 0.05 second is common (c.f. 3 seconds on the Lumia 1020!!!)
  • apertures as wide as f/1.5
  • physical telephoto lenses
  • multi-camera software effects (e.g. portrait modes)

In short, I really appreciate the Lumia 1020 and 950 range for the amazing photos they still produce, world class in every way. But you do need a little patience and/or skill. Once you include a time factor then the modern competition runs away as 'better' for the man in the street. 

The Lumia 1020 was 2013, the 950 and 950 XL were late 2015. Two and a half years on and one has to wonder 'what might have been' if Nokia and/or Microsoft had persevered with the platform and hardware design.

So - comments welcome - which 'SteveMark Top 10' do you identify with more? Is speed an issue for you, or can you live with the camera experiences in the Lumia 1020, 930, 950, and so on?