Again, see part 1 of this feature for all the detailed imaging specifications of each camera phone. Note that I'll be using the Lumia 950 XL in 8MP oversampled 'PureView' mode, since that most closely matches the Mate 20 Pro's output. And, for zoomed shots, that 8MP mode does allow a little (1.5x) lossless digital zoom, though it's going to fall some way short of the Mate 20 Pro's zoom options.
Test 1. Bright sunlight, simple snap
A typical sunny snap, here is the scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
A complaint you'll hear several times on this page, the Mate 20 Pro image is good but looks almost like a colour photocopy of the Lumia one, i.e. the latter is, genuinely, purer and more natural. It's all down to sharpening (and lack thereof) - why does every manufacturer mess this up. So I'm being super picky - and I'll dock the Lumia a point as well, for making the white boat cream coloured.
Lumia 950 XL: 9 pts; Mate 20 Pro: 8 pts
Test 2. Bright sunlight, masses of detail
A typical sunny snap, here is the scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
A perfect photo from the Lumia 950 XL and a very good effort by the Mate 20 Pro. However look at the small details, for example leaf edges, and you can see enhancement where there should be none. Given the sensor size, why on earth is Huawei insisting on such (relatively) heavy handed extra processing? It's perhaps not quite as bad as on the original P20 Pro, but your eyes can hopefully see what I mean above?
Lumia 950 XL: 10 pts; Mate 20 Pro: 9 pts
Test 3. Overcast conditions, and introducing zoom!
Testing the phone cameras under more modest lighting, here is the unzoomed scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
Hmm... it's the same story again, as with the first few tests. The Mate 20 Pro makes something of a mushy mess of greenery and, looking at the plane's fuselage, makes texture and patterns that aren't really much more than dirt on the metal. And there doesn't seem to be a way to turn all this edge enhancement off, even in the 'Pro' mode. The photo still looks good, especially on the phone screen, but the Lumia's photo is richer and more real - as is my common phrase, "this crop could be the view out of a window and you'd believe it". Whereas the Mate 20 Pro's photo looks just like... a photo!
Note that I'm not hating on the Huawei phone here - I really wanted results from its large main sensor to be better. Interestingly, the processing can be sidestepped by shooting and saving a RAW file, but that's an article for another day - and perhaps another site - since there's really no way regular users can be expected to have to mess around with 40MB RAW files!
Lumia 950 XL: 10 pts; Mate 20 Pro: 8 pts
And now we start testing zooming in. The Lumia 950 XL has about 1.5x lossless zoom, so we'll use that and then apply 0.5x lossy digital zoom, for a (roughly) 2x zoom overall. While the Mate 20 Pro has a dedicated 3x telephoto lens, so that's what we'll use for the comparison.
In case you want to grab the original zoomed images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
With the aforementioned caveat over edge enhancement making up texture that might not really be there, the 3x telephoto lens does a cracking job here under gloomy conditions. The Lumia 950 XL's 2x hybrid zoom just isn't in the same ballpark, being noisy and with artefacts, plus it's not 3x, so....
Lumia 950 XL: 7 pts; Mate 20 Pro: 10 pts
Test 4. Zooming to 5x - is it really possible?
Having got the taste for zooming and with the sun now out again, here is an unzoomed river scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
At 1:1 and 1x zoom here, the Lumia 950 XL again has the edge - there's something not right about the processing from the Mate 20 Pro's main sensor - it's as if the software's trying too hard. And yes, AI processing is definitely turned off. All very odd, and I expect better from one of PureView's creators.
Lumia 950 XL: 10 pts; Mate 20 Pro: 8 pts
But let's zoom again (like we did last summer...) and this time take the Mate 20 Pro's camera up to its maximum 5x hybrid zoom, using output from the 40MP and 8MP telephoto sensors.
In case you want to grab the original zoomed images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
Now, don't freak out about the amount of artefacts in the 5x hybrid zoomed crop from the Mate 20 Pro - this is at the pixel level in a 10MP image and you really do have to look this closely to see the understandable imperfections. For a handheld shot from a phone camera, I'm impressed by the results - grab the original, using the link above and see what you think.
The Lumia's crop is from a 2x zoomed photo and it too looks great when seen on a phone or computer screen, but it too has artefacts. But the win by a sizeable margin has to go to the Mate 20 Pro here.
Lumia 950 XL: 6 pts; Mate 20 Pro: 9 pts
Test 5. Sunny macro
Moving away from zoom, down to the close-up, here I'm shooting some small flowers in the sun, at about 10cm, here is the scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
Both photos are remarkably good - I think the Lumia has the edge because of the richer and smoother colours, while the Mate 20 Pro again tries too hard to pick out texture with its edge enhancement. Your comments welcome, but I'm giving the 950 XL an extra point here.
Lumia 950 XL: 10 pts; Mate 20 Pro: 9 pts
Test 6. Rainy morning gloom
Starting to look now at low light (and beyond), here is an early morning gloomy shot in light rain, with loads of detail of a garage across the road. Here is the scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
The edge enhancement strikes again for the Mate 20 Pro here - look at the blue smartphone advert bollards (or poster behind), look at the goods in the bays outside the garage - in each case details are lost because of over-zealous enhancement. Note that this is nothing to do with the Quad-Bayer sensor - this is a 10MP shot and as such is using 'super pixels'. The errors at the pixel level here are PURELY down to choices Huawei has made, ill-advised in my opinion, in their image processing (for JPG, again I'll come to RAW versus the 950 XL in another article).
Meanwhile the Lumia 950 XL shows how it should be done - there's more detail, less edge artefacts - and better dynamic range, preserving most of the green in the M&S lit up logo.
Lumia 950 XL: 10 pts; Mate 20 Pro: 8 pts
Test 7, Low light, indoors
Lit only by the light from a side window in the church - and a lot darker than the phones make out here(!), so a real challenge in terms of colours, detail and light gathering, here is the scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
The images are very, very close. The Mate 20 Pro's sharpening adds a little more detail, at the expense of looking a little more artificial, while the Lumia gets a natural image but with a slight yellow cast (as often happens). Honours even overall, I think.
Lumia 950 XL: 9 pts; Mate 20 Pro: 9 pts
Test 8. Night time
One of my standard suburban night tests - a boring scene, but hey, where are you going to go at midnight? Here is the scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
Huawei's usual heavy handed sharpening and noise reduction rather spoils this shot - I have a feeling that the sensor is capable of so much more. As it is, the three year old Lumia 950 Xl camera smokes it, with more - and crisper - detail, as you can see above.
Yes, the Mate 20 Pro has a 'Night' mode that uses multiple shots taken over a few seconds, but I tried this and - although much brighter - the detail was still smudged out by the image processing. Which makes me sad. And, at this stage getting a bit cross - I had such high hopes. I test night mode below, if you want to see what it can do...
Lumia 950 XL: 9 pts; Mate 20 Pro: 7 pts
Test 9. Crazy stupid night mode
Now this was insane, but we have a sunflower growing at random in our front garden (maybe a bird dropped the seed there!) And, as I was out shooting the night test above, of the lit street, I wondered how far the acclaimed 'night mode' of the Mate 20 Pro could go. The flower was in complete darkness, I could hardly see it with my eyes. Here is the scene as presented by the Lumia 950, though it was FAR darker in reality. I'm just seeing how much light these devices can gather if pushed!
I haven't provided original JPGs for privacy reasons (my car number plate is in the background). But to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
It's quite astonishing that both phone cameras managed to extract yellow and green from the monochrome blackness that my eyes saw! The Mate 20 Pro's special 'night mode' does indeed win this test in terms of lower noise and purer colours, though the Lumia 950 XL, in full auto and without going into any special mode, really wasn't far behind.
Lumia 950 XL: 8 pts; Mate 20 Pro: 9 pts
Test 10. Bright sunlight, human face!
By popular request, an actual human being in good light, just me as I couldn't find a volunteer! Here is the scene as presented by the Mate 20 Pro (not the Lumia this time, as noted below):
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central 1:1 crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
I'm scoring this one as a draw, since the Lumia clearly has the more natural detail, it's almost perfect, but then it goes a spoils it with a typical 'yellow cast' - even in full sun, I'm not that tanned or (almost) orange! While the Mate 20 Pro shot is far paler - in fact, slightly too pale, but it's closer to my real skin colour than the Lumia. But then the detail looks artificial and sharpened.
It's at this point that I start to wonder if there's ANY camera phone that can nail shots like this with no complaints. Which gives me the idea for another article....
Lumia 950 XL: 9 pts; Mate 20 Pro: 9 pts
Test 11. Party time
My typical party mock-up flash test, with moving subject and very poor lighting, with LED flash forced on, here is the scene as presented by the Lumia 950:
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
This is one of those 'WTF' moments. I retook the shot SIX times with the Mate 20 Pro and every single time the LED fired but the actual shot captured was stupidly dark. I restarted the phone and tried again. Same result.
Then my camera-woman/wife suggested turning off the small lamp in the corner of the frame. And hey presto, the Mate 20 Pro's photo-taking returned to normal, as shown below. This is clearly a bizarre bug in the Mate 20 Pro, one that hopefully will get fixed - there's no way a small change in ambient lighting should completely kill the final photo in this way.
In case you want to grab the original images to do your own analysis, here they are, from the Lumia 950 XL and Mate 20 Pro, click the links to download. And to look at the images in more detail here, here are fairly central crops, just wait to make sure the page has fully loaded and then use your mouse or trackpad pointer to compare the images:
Don't worry about differences in framing, I never go right down to 1:1 in party mock-up shots, no one's expecting pixel perfection for this use case. It's about snapping a moment. And, now that the 'lamp' bug had been taken out of the equation, the Mate 20 Pro did just as well as the Lumia 950 and I'd normally score the snaps about the same. But I have to take off a few points for failing me the first half dozen times. Sorry, Huawei, this is a nasty bug and will catch people out.
Party short overall: Lumia 950 XL: 9 pts; Mate 20 Pro: 7 pts (allowing for the bug)
Verdict
Despite the zoom superiority, you will have noticed a theme running through all these tests - the image processing in the Mate 20 Pro is messed up. What Huawei has done is, essentially, transplant the same sharpening, edge enhancement and noise reduction algorithms from other Huawei phones with lesser cameras (on which such drastic measures were needed), when there really isn't any need for them here. A 1/1.7" 40MP sensor with f/1.8 aperture needs no help at all. And as a result the big sensor never gets to shine. Look at the pixel level, as I have done (and as you will in the real world every time you decide to crop a photo down a bit), and the result is usually something of a mess.
Producing, as with the P20 Pro before it, the same verdict - if you need to do lots of zooming then this is a good camera phone bet - otherwise avoid it and await news of updates. Given the ex-PureView heritage of Eero Salmelin, Huawei's imaging lead, I'd have expected a lot better.
For the record, the final scores were:
- Lumia 950 XL: 116 pts
- Huawei Mate 20 Pro: 110 pts
Obviously the Mate 20 Pro is far more capable as a general smartphone at the end of 2018, but the Lumia 950 XL has been THE benchmark for imaging, at least for me, since the end of 2015. And the comparison clearly shows how far some of the competition still has to go before it can deliver photos anywhere near as good.
PS. Of course there are some follow-up pieces needed. A RAW image comparison. Video tests. Picture purity tests across multiple phones. And yes, a re-test if there are significant updates. Watch this space!
