The PureView concept
The central idea behind Nokia's original 'PureView' system was to have a large sensor with a high megapixel count - not for producing high megapixel images (though that's possible too), but for gathering enough light information that 'computational photography' became a reality. In addition to a degree of genuinely lossless digital zoom, the highlight for many people was 'oversampling', pulling information from multiple pixels on the sensor to produce 'perfect' pixels in a lower resolution output image.
This 'perfection' is manifested, in theory, by the elimination of digital (random) noise, since random pixel readouts at the pixel level get neatly averaged out as part of the oversampling process. In addition, there's better colour and illumination accuracy, again because values from underlying physical pixels can be cleverly averaged.
Implementations
In the Nokia 808, launched in 2012, the device's main processor (single core, 1.3GHz) and RAM (512MB) was nowhere near fast enough to handle all this oversampling of (up to) 41 million pixels in real time, so custom work was done to integrate the special Toshiba imaging chip/sensor and a dedicated Broadcom BCM2763 image processing chip, together handling this side of things. So we had the raw sensor image data being processed in real time, spitting out the required oversampled data to the screen and to the JPG and MP4 encoding routines in software. It's a system that worked terrifically well, though was somewhat expensive to have implemented in the first place (in both money and time).
For the Lumia 1020, the main processor (dual core 1.5GHz) and RAM (2GB) was deemed up to the job of doing the 'oversampling' in software, so in theory ending up cheaper and simpler to build and maintain, but there had to be a few compromises made. Rather than process the raw data from the 41MP sensor in real time, when under the control of the Nokia Pro Camera application, the camera unit spits out a 34 or 38MP JPG file (stored in the normally invisible Windows Phone file system with a special extension) and this is then downsampled using traditional software techniques (e.g. bicubic interpolation, though the exact algorithm being used by Nokia isn't known in detail) for screen use and for storing as a 'shareable' 5 megapixel JPG.
[update, after penning this addendum article] Although there is still real time oversampling of the sensor data to produce the initial 5MP 'shareable' image, this is in many cases something of a temporary construct. A 'dual capture' system also constructs a 34 or 38MP JPG file, stored in the normally invisible Windows Phone file system with a special extension and this is used as the base for all 'after the event' manipulation.
This latter approach, despite any criticism which you may think is levelled at it below, does have the immense advantage that for users the 'oversampling' isn't a one time thing, performed when the photo is captured. In fact, you can reframe, zoom and rotate any 5MP viewport onto the underlying full resolution JPG at any later point (hence Nokia's 'Zoom later' slogan for this device).
Problems and questions
The saving and processing of the full resolution image takes a few seconds, even on the Lumia 1020's fairly powerful internals, meaning around a four second shot to shot time in real world use, which might be inconvenient, depending on what you're shooting. Begging the question of what happens when you use the default Windows Phone Camera application on the same hardware - shot to shot time is around two seconds (including an enforced captured image preview), the same as on any other Windows Phone device. Now, obviously, shot this way, there's no PureView zoom or reframing ability, since there's no underlying full resolution image. But is there still any oversampling/downsampling going on?
Moreover, how does the software-based 'oversampling' on the Lumia 1020 compare with the GPU-based oversampling on the Nokia 808 PureView - is it possible to tell the difference?
_______________
In order to answer the questions above, I set up a test environment, a darkened hallway with almost no natural light. There's no point in shooting something in bright conditions, since the images would be virtually indistinguishable. As I've remarked before, testing camera phones is all about pushing them at the limits of their light gathering abilities, not keeping them in 'safe' territory.
The subject was a poster of my surname artily drawn by a student of my wife's - there were consistent colours and clearly drawn outlines that should prove very revealing to try and photograph when there was extremely little ambient light. Here's the whole photo/scene:
Note that the photo above makes the poster look far, far brighter than the scene was in real life. No flash was used for any of these comparisons, I was purely interested in raw sensor and oversampling performance. To my human eyes, the colours on the poster were almost grey, the hallway was so dark.
Here's the photo, cropped slightly and in slices, taken on the Lumia 1020 with the default Windows Phone Camera application first and then the Nokia Pro Camera application. Below these is the same crop taken from the photo as captured on the Nokia 808 (of which more below):
Apart from the slight colour/exposure difference in the bottom crop, there's little obvious difference between the photos at this modest crop level. So, here's conclusion no.1:
You can use the default Windows Phone Camera application for taking quicker (less than 2s shot to shot in real life) 5MP photos on the Lumia 1020. For events and action settings, you'll still get decent shots (with or without flash).
As usual, it's necessary to crop in more tightly, to 1:1, in order to see what's actually going on. The same order applies below, i.e. output from Camera, then Nokia Pro Camera and finally the same section of the Nokia 808's image:
Never mind the slightly different colours and characteristics in the 808's version, we're going to address them in a moment. Look at the top two crops and pick a few details for yourself. You can see that the Camera-shot photo is full of typical camera phone sensor noise, 'fuzziness' in the image, with significant artefacts in blocks of solid colour, while the photo shot by Nokia Pro Camera has cleaner lines and less noise in every detail.
Bringing us to my second conclusion:
The 'oversampling' used in the Nokia Pro Camera application does have significant benefits in terms of reducing noise and depicting fine details, when compared to the default Windows Phone Camera app*.
* a subsidiary question is whether there are other camera apps, i.e. from third parties, which also make use of the bigger, higher resolution sensor. Watch this space for more on this!
Let's come to the main event, comparing the software oversampling (from 34MP JPG) on the Lumia 1020 with the hardware oversampling (from the raw camera output) on the Nokia 808. But first we need to equalise a few things.
For the test shot above, the Nokia 808 was tripod mounted (since it has no OIS) and was on full auto apart from me upping the ISO to 800 to get it in the same ballpark as the Lumia 1020's camera software. In this configuration, the Nokia 808 left its mechanical shutter open for 0.8 seconds, over three times longer than picked by the 1020's camera apps on full auto. So, to compare like with like (and yes, I know that ISO will still be slightly different), I use the manual control over shutter speed on the Lumia 1020 to leave its (also mechanical) shutter open for the same duration, i.e. 0.8s.
So, retaking the shot with the 1020 and then comparing its result side by side with the 808s, we get:
There's still an exposure difference between the two results (curiously, since the 808 was at ISO 800 and the 1020 was at ISO 1000), but it's less significant now that each smartphone camera has had an equal amount of time gathering light. The 808's photo still looks cleaner overall, but let's crop in, beyond 1:1, magnifying the pixels to 4:1, in order to see why:
What you're seeing here, and yes, we have had to go to somewhat artificial lengths to make things clear on-screen, is the difference between the JPG downsampling [updated text because of this addendum] software-based, processor-hosted oversampling on the 1020 and the 'pure' hardware oversampling from raw camera output on the 808. The latter's image is unbelievably clean, with blocks of almost solid colour, clean edges and almost zero random noise.
I do want to set this all in context, of course, but in fairness to the Nokia 808, conclusion number 3 has to be:
Real time hardware oversampling on the Nokia 808, in terms of the absolute purity of output, is significantly more effective than the software-based used in the 1020.
____________________
How pure is pure enough?
The Nokia 808 was something of a freak device, in that it was the last of the Symbian line and contained significant custom hardware which couldn't be transplanted as-is into another, more current smartphone platform. This may even be the last that AAWP readers get to hear of it (for which some will be thankful!), but this focus on oversampling allows the original PureView technology a last hurrah, at least.
Which leaves the Lumia 1020 as the current implementation of the PureView concept in the 2013 smartphone world:
- Despite the relative lack of 'purity' concluded above, the images produced by the 1020 are still significantly higher in quality than the smaller sensored competition. Moreover, as referred to at the start, there's the extra flexibility of being able to reframe images and 'zoom' in after the fact - or, famously, zoom out again after the fact (albeit with a very slight loss in absolute quality).
- In addition to the reframing, there's also the presence of Optical Image Stabilisation (OIS), which has the potential to let normal phone users take far crisper shots than they might otherwise have done with lesser hardware. OIS is just starting to creep into other top end smartphones (e.g. HTC One, LG G2), but the implementation there is either inferior (HTC) or unproven (LG).
- There's the flexibility of no less than three built-in Camera applications (default Windows Phone, Nokia Pro Camera, Smart Camera), plus a growing number of third party alternatives. Nokia Pro Camera, in particular, allows a control of focus, ISO and shutter speed which is unparalleled on smartphones.
Given the advantages listed here, it's tempting to ask how 'pure' you need a photo to be in the first place? Camera phone geeks will demand that every last pixel be perfect (and perhaps I fall into this category at times), but for most people the Lumia 1020's 5MP JPG output will be 'easily good enough' and more focus will fall on how users can manipulate and share the images, tasks for which Nokia Pro Camera and Windows Phone are well suited.
And in view of this final conclusion, perhaps some tweaking of the definition of oversampling, from Nokia's original PureView definitions, can hopefully be forgiven.
Your thoughts welcome.
