Demo - why we don't only test smartphone cameras by looking at full photos

Published by at

I've done a lot of smartphone camera shootouts over the last five years on All About Symbian and All About Windows Phone, each revolving around taking the same shot with a number of different test units and then (at some point) cropping in to look at pixel-level detail. And each time I get called out for doing this: "Real users don't crop in to the level where they can see pixels". Here's my defence, aided by some rather nice example photos from a mystery device...

Sample photo at web resolution

Nice snap, eh? Taken on a top 2013 smartphone, of course? Read on...

It drives me mad when I see another reviewer on the Web looking at a smartphone camera and taking a dozen snaps, all of which are reproduced at typical Web resolution, so usually less than 800 pixels wide. "Look at the detail, look at the contrast and exposure" they say, never minding that two of the three things can't be seen when images are downsampled.

Or I put up a photo comparison between shots on the Nokia 808 and Lumia 920 and I get criticised for cropping in to the stage where you can see individual pixels. Yes, I know full well that most users don't crop in this far or enlarge snaps to the stage where pixel detail would matter. In fact, most users rarely do more than view photos on their phone screen or upload them to a sub one megapixel social sharing service.

Sample photo at web resolution

All of which is a huge problem for myself and for anyone else keen to establish, once and for all, the pecking order for phone cameras. At the resolutions shown in most reviews, and deliberately included here, the differences in results from almost all cameras are negligible. You simply can't tell whether any more useful detail is captured unless you look closer - a lot closer.

But, you'll splutter, "If real world users never crop in or print out at huge sizes then does it really matter whether the camera in one phone is better than in another?"

Great point and great question. In a sense, no, it doesn't matter. Not to most people.

I was talking to a friend the other day who's been using a 2 year old budget HTC Android smartphone (I didn't recognise it) - she was about to upgrade to an iPhone 4 and I remarked that it had a far better camera than the one in her old HTC. "Oh no", she said, "it's fine" and she proudly showed me some snaps of relatives taken on it. And, at screen resolution, the snaps were fine. I multi-touch zoomed in a little and showed her how fuzzy the snaps were in terms of lack of detail on faces and clothes, but it didn't bother her - she was fine looking at photos on-screen.

Sample photo at web resolution

However, anyone who cares enough about taking good photos that they've got this far through this feature will know that there's more to a good photo than looking great on a web page or phone screen:

  • If a photo is detailed enough and clear enough then you can often pick out/crop a part of it to create a whole new image. Don't like the shot of one friend, but love the expression on the face of another? Crop in and you've got a fabulous shot of the latter. IF the original image was good enough. IF.
      
  • If a photo (as in great lighting here) is really detailed and clear at the pixel level then it gives you confidence that there's a good sensor and good optics in the device - photos taken in lesser light conditions are more likely to turn out well than on inferior devices.

And so to the photos shown above, three pretty good photos taken in good lighting. Were they snapped on a Nokia 808 PureView perhaps? Or an iPhone 4S? Or a Lumia 720? Impossible, absolutely impossible to tell at this size. I could have snapped the same shot on all three and got a near identical result.

Ditto for something with a really average camera like the Nokia 5800 or the Motorola RAZR i or the Nokia Lumia 710 or iPhone 4 or 3GS. The results would look identical apart from a few variations in colour saturation. Comparing them like this would be a complete waste of time.

Does that mean that these phone cameras are interchangeable? Not at all. The Nokia 808 PureView is clearly a light year better, across all subjects and all conditions, than the old and budget Lumia 710, in terms of camera performance. You just can't tell at this size. You absolutely can when cropped in:

Photo scene, crop detail

(taken from the 808 vs Xperia Z ve Lumia 920 shootout, by the way)

At which point I should stop teasing and reveal that the shots above were taken today on...

Lumia 710 with budget 5mp camera

...yes, indeed, the Nokia Lumia 710, almost two years old and budget hardware even then. Did I fool you?

So the next time I do a camera comparison and crop in fairly savagely in order to more clearly see differences in the photos captured, let's have no more of the 'real users don't crop' talk. By all means I'll judge the full image, looking at colour balance and exposure. But I'll also crop in savagely, to show how good or bad the sensor and optics are in that particular device.

A popular cliché is that "the devil is in the details". In my version of the saying, it's in the "pixels"!