The wildcard is, of course, and no matter what the blinkered folk at DxO Mark say, the current best camera phone in the world, the Nokia Lumia 1020. Last year's flagship, it's true, but still sold in most places and still top gun by a huge margin in terms of specifications. 1/1.5" sensor, 41MP resolution, proper Xenon flash, and so on.
Actually tested here were the 1020, plus the Lumia 1520 (also representing the 930/Icon, of course) and the Sony Xperia Z3 Compact (I have the Z3 here as well, but the cameras are again identical and the Compact is so much easier to use as an imaging device), though I use just 'Z3' below, for convenience. The test cases are designed to bridge a variety of light conditions and use cases. Each of the smartphone cameras uses oversampling, so it's completely fair to use the noise-reduced, oversampled output in the 1:1 crops, though note that the Sony's was from a 8MP original rather than the Nokia's 5MP in each case.
And, so as to make use of the high resolution sensors as-is at least a couple of times, I included some lossless zoom tests, in which each high resolution sensor is taken down to the 1:1 level (i.e. no oversampling). Or at least, I tried to.... You'll see what I mean below.
As usual, all shots were taken with camera applications on 100% default settings. Yes, each system allowed for infinite variations of exposure, white balance, and so on, but I wanted to look at capabilities out of the box, in the hands of normal users, not photo-geeks(!)
Test 1: Bright sun, medium distance subject
An interesting subject at FAST, an old BAC Lightning, with plenty of detail, not trivial to capture given the amount of silver! Here's the overall scene:
The 1:1 crops are from detail just off centre, looking at the cockpit - a little sneaky of me to look away from the centre of focus, but in these light conditions the whole plane is well within the depth of field. From top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
The 1020 just edges this photo, with super colours and detail, while the 1520(/930) camera is very slightly 'off' in terms of the greens and plane colour, and the Z3 has something of an unnatural/artificial quality, a product of over zealous noise reduction and then subsequent over-sharpening.
Scores: Lumia 1020: 10, Lumia 1520: 9, Z3: 8
Test 2: Sunlight, landscape
A pretty easy subject, a clean, new office at around 80m. Here's the overall scene:
And some 1:1 crops from the centre of the frame. From top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
Pretty good photos all round, as you'd expect. Starting a trend that you'll see throughout the following 2500 words, there's evidence of over-sharpening on the Lumia 1520 shot and lots more on the Z3 one, while the 1020 keeps things natural and, as a result, looks more real.
Scores: Lumia 1020: 10, Lumia 1520: 9, Z3: 8
Test 3: Bright indoors, flowers
What a great display, and masses of colour. Here's the overall scene:
Pretty challenging, in terms of amount of detail and the sunlit (through a window) highlights on the yellow flowers in particular. From top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
All three photos are pretty good, but I'd give the edge, in terms of detail and colour, to the Lumia 1020, despite the blown out yellow highlights, something which all three phone cameras couldn't cope with on default settings. In a real world setting, you'd hopefully spot the problem on each camera phone and try another shot with one less stop on the exposure (and then correct later in a photo editor). The central detail wasn't focussed quite as well on the Z3, but I'm not going to knock it down for this, because there was so much else in the full frame and it was hard to get an exact focus for every flower.
Scores: Lumia 1020: 9, Lumia 1520:8, Z3: 8
Test 4: Challenging contrasts of light
With shade and sunlight streaming through trees, plus extremes of detail, this made for an interesting test scene. And, rather than crop in immediately (though see below), I wanted to show differences in how each smartphone camera handled the scene on default settings.
From top to bottom, (roughly 50%) scaled crops from the Lumia 1020, Lumia 1520, Sony Z3 Compact:
All three shots are perfectly fine, though there are small exposure and colour differences. The 1020 gets the colours most accurately (look at the path, for example, which should be a totally neutral grey), while the 1520 does well in terms of exposure and contrast, but has a slight red cast (as usual), with the Z3 just bringing up the rear, with a slight over exposure and a slight colour cast.
But, with so much detail on offer, it's also worth cropping in to 1:1. From top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
The over-sharpening on the Z3 camera becomes quite unpleasant when you look closely, as you can see here, the algorithms just don't know what do to with so many edges, so much detail. The 1520 does a lot better, with better oversampling and far less artificial enhancement, while the Lumia 1020 lives up to the 'PureView' monicker well, with great and natural handling of detail.
Overall scores for both colour and 1:1 detail: Lumia 1020: 10, Lumia 1520:8, Z3: 6
Test 5: Bright light
Another interesting subject at FAST, with plenty of colour and detail. Here's the overall scene:
The 1:1 crops from central detail are below, from top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
The 1020 just edges this photo, as you might expect, with perfect colours and detail, while the 1520(/930) camera produces its characteristic (slight) red tint and the Z3 sensor loses some detail and contrast with the blown out whites.
Scores: Lumia 1020: 10, Lumia 1520: 8, Z3: 8
Test 6: Bright light, losslessly zoomed
One of the points of high resolution sensors is that, if needed, you can losslessly zoom in, effectively cropping in on the sensor while maintaining the normal output resolution. The same subject helicopter as in the previous scene, but this time centred on the rotor head and using the built-in zoom facility.
And it's at this point that we encounter our first real issue with the Sony Z3 cameras - it's in the interface. The on-screen zoom display indicator doesn't show accurately when lossless zoom is over and when lossy zoom is being used. There's a bar with a clear marking halfway along, zooming stops temporarily at this point and it's evident that Sony means this to mark the region you should stay within, when zooming with either the volume buttons or by pinching and zooming. However, the actual end of lossless zoom comes way before this marking, and there are no appropriate end-stops or limits in the controls, so it's pure guesswork as to how to stay within the realms of decent quality.
The 1:1 crops below show what happens if you rely on Sony's markings in the UI. From top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
I know the whole point of this article is to compare the 1520 and Z3 class phone cameras, but take a look at the top crop - the quality from the Lumia 1020 is staggering. In contrast, the 1520 seemed to have difficulty focussing accurately when zoomed in and the Z3 has clearly moved massively into lossy and blocky digital zoom territory, producing ugly results. Hence the next test scene, deliberately using less Z3 zoom....
Scores: Lumia 1020: 10, Lumia 1520: 5, Z3: 3
Test 7: Bright light, zoom manually limited on the Z3
Using my own judgement, and ignoring Sony's on-screen indicator, I tried another zoomed scene, desperately trying to stay within lossless zoom territory. It becomes quickly apparently how useful Nokia's system is, wherein you're physically stopped from making any incursion into lossy digital zoom. Here's the overall scene, this is already zoomed in, at the 1020's limit:
The 1:1 crops from central detail are below, from top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
Another resounding win for the 1020, the colours and detail, even zoomed right in on the limit of the sensor, are quite extraordinary. The 1520's photo isn't bad, considering what I was asking of the unit, while the Z3's result looks like a photocopy of an original photo. What I think is happening is that even being incredibly light on the zoom controls, it's still far, far too easy to slip into digital zoom. In fact, on the basis of these tests, I think you can effectively eliminate even the possibility of lossless zoom on the Z3 and Z3 Compact - it's a system that's just unusable as-is. And Sony could fix it with a software update, with a detent at the right point in the UI or by changing the limits of zooming completely.
Scores: Lumia 1020: 9, Lumia 1520: 6, Z3: 4
Test 8: Low light, static subject
The interior of a Concorde cockpit mock-up, artily lit, also at FAST. The video screens usually work as well, but were broken here, making for a better low light test for me! Here's the overall scene:
Good results from all test devices here, let's look at noise and detail in the 1:1 crops. From top to bottom, Lumia 1020, Lumia 1520, Sony Z3 Compact, and click any of the crops to download the full original image, if you so wish:
There's not that much to choose between the phone cameras here - the Sony's extra default 8MP output produces very slightly more detail (and manages to keep noise in check), but makes a mess of the blue lighting. In fairness, my eyes don't remember the scene as having as much blue as the Lumias portray, so I suspect reality is somewhere in between.
Scores: Lumia 1020: 8, Lumia 1520: 8, Z3: 7
Test 9: Party/flash test
My traditional 'party mock-up' test, where I actively try to dance/move/laugh (thank goodness this isn't video!) and see how well the phone cameras can capture low light movement. In each case, I made sure I was moving significantly when the flash fired. No overall scene here, since there's little point in going down to 1:1 detail. What we're interested in here is how well the subject is captured/frozen under typical 'event' lighting.
From top to bottom, (roughly 50%) scaled crops from the Lumia 1020, Lumia 1520, Sony Z3 Compact:
There's not that much to choose between the Lumia 1520 and Z3 results here - typical LED-lit people shots indoors with blurred subject. As seen in gazillions of phone camera uploads world wide. The Lumia 1020's Xenon flash makes a far better fist, as expected, at freezing movement, plus it even gets exposure of the overall scene and the light perfectly. It's not perfectly crisp on my face, but it's more than acceptable and gives a great impression of what the human eye would see/remember.
Scores: Lumia 1020: 9, Lumia 1520: 4, Z3: 3
Test 10: Dead of night, handheld
Just street lamps, around midnight, shot handheld. Again, light levels were so low that I'm not going to expect miracles by dropping down right to 1:1 crops. Here's the overall scene, and note the doubled lampposts - the council put in new ones and haven't yet got round to taking out the old ones!
And. from top to bottom, (roughly 50%) scaled crops from the Lumia 1020, Lumia 1520, Sony Z3 Compact:
To be honest, in terms of noise and detail, the Z3 camera doesn't do too badly here, considering that it doesn't have OIS - a testament to how easy it was to hold still. But it gets the exposure wrong, with the scene looking like it's floodlit rather than eco-friendly council street lighting! The 1520 gets somewhat closer to how my eye saw the scene, but with characteristic low light noise detracting from the image. Meanwhile, the Lumia 1020 sensor, around three times larger, gathers so much light and uses so much oversampling that noise is kept to an absolute minimum.
Scores: Lumia 1020: 8, Lumia 1520: 5, Z3: 4
Verdict
Overall then, ten test scenes, with scores out of 100, giving a ranking of:
- Nokia Lumia 1020 - 93 pts
- Nokia Lumia 1520/930/Icon - 70 pts
- Sony Z3/Z3 Compact - 59 pts
The main aim of this head to head was to compare the latest from Sony with the current Nokia/Microsoft Lumia flagships, and in terms of imaging the Lumia 1520/930/Icon are significantly ahead of Sony's Z3 line, as they were ahead of the Z2 before it. Which isn't surprising, since Sony's imaging has basically been standing still for two years, all a bit sad given Sony Ericsson's arguable camera-phone expertise back in the early 2000s....
The only areas where Sony's camera here had an edge was in speed - focussing was faster, as was shot to shot time and camera launch time. But it's an argument that keeps getting repeated in these pages - would you rather have four sub-standard images (Z3), two slightly disappointing images (1520/930) or one really excellent one, all in the same five second period? Surely - surely - most people would go for the latter?
What's worrying, from a Windows Phone perspective, is the sheer gulf between 2013's Windows Phone flagship, the Lumia 1020, and the next generation devices. And with the latest Lumia 830 having a much smaller sensor again, it seems like Nokia (now Microsoft)'s phone camera prowess is going backwards, at least in terms of image quality. There really is only so much you can do with software algorithms, guys - the 1020's sensor/size/flash were perfect - just stick it all in a device with the latest processors and we're done. Pretty please?
PS. Thanks to Clove for the review Z3s.... And see the imminent Phones Show 235 for full reviews.