The workflow - and hidden cost - of Rich Capture under Lumia Camera 5

Published by at

So Lumia Camera 5 has arrived, along with Lumia Denim for your Nokia Lumia 930 or 1520? What exactly is going on under the hood when you turn on 'Rich Capture'? Is there a downside? What about limitations? And why can't you go back to edit a rich capture photo from the Photos app later on? In this in-depth feature I answer all these questions and much more...

I guess I should also say that much of what follows will also apply to the Lumia 830, Icon and any future devices of this class, though in some cases without the X/X_highres image pairs, since there may not always be oversampling going on in terms of resolution.

To recap, the Lumia 930 and 1520 both have 20MP cameras, with the express intent by the designers that this raw resolution be oversampled to produce more sensible (5MP) output, hopefully with 'purer' pixels and less digital noise. The system works well on the whole.

And this system is largely kept in place for Lumia Camera 5 under Denim on the 930 and 1520 - but with a few twists. One being the moving of the reframing (i.e. from the underlying 20MP image) out of the camera app itself and into Lumia Creative Studio, and the other being the introduction of a whole new multi-shot image capture system, 'Rich Capture', in which multiple shots can be automatically or manually combined.

The point of the former is to simplify the workflow for the latter, as I'll try to explain in a reverse-engineered chart below. It's worth getting to grips with though, as it will help you understand some of the benefits and trade-offs of Rich Capture.

Note that, right up front, I'll state that on the Lumia 930 I don't keep Rich Capture turned on all the time - I'll explain why below!

Workflow for Rich Capture

Let's unpack that chart slightly. With Rich Capture off, everything's as it was before Lumia Camera 5, except that the shot to shot time is much faster because the oversampling to produce the 5MP image (from the 20MP one) is done by one of the phone's other CPU cores, in the background, while you get on with your next photo. And if you did want to 'reframe' a photo you've just taken, there's the extra step of going via the 'Edit' toolbar option and choosing 'Lumia Creative Studio'. There are benefits, of course, since, the latter has all kinds of other useful effects and adjustments.

Now turn on Rich Capture. As intimated in the chart, the phone now takes up to three fast 20MP snaps, with different parameters (exposure, flash) and then combines them in two or three suggested ways, along with the option to set the blending of these manually. Although there only seem to be three different auto schemes at the moment (HDR, Low light and Flash), I can't emphasise enough that all this is just software and a future update to Lumia Camera 5 will almost certainly extend this scheme. 

ScreenshotScreenshot

Either way, oversampling is done as part of the combination of underlying 20MP images you end up with a similar pair of JPGs, one at 5MP and one at 20MP, keeping things nicely compatible for editing in Lumia Creative Studio and other tools. 

File and space matters

Which is all well and good and I - and others - have been impressed by the interface and results. Under challenging light conditions you can shoot and worry about sorting out the shooting parameters later, effectively. But as I say, I don't recommend keeping Rich Capture on all the time, principally because there's a lot of extra file space needed.

As it is, taking a photo on a Lumia 930 or 1520 involves a Image.jpg and Image_highres.jpg pair, at around 1MB and 6MB respectively. So 7MB in total, plus maybe 1MB more if you also have 'Living Images' turned on.

As intimated in the chart, the (up to) three quick 20MP images are grabbed and combined in various ways, if you look in the phone's file system you'll see a '.nar' file, usually around 20MB, actually a ZIP archive with these 'combined' JPGs and various .xml files describing how they were arrived at. The very fact that this archive/XML scheme is used gives me confidence that Microsoft will extend it to other Rich Capture combination options in the future:

Screengrab

Contents of the .nar file/archive associated with a typical Rich Capture photo...

Screengrab

Contents of one of a typical content XML file...

From these 20MP, 6MB combination images, one is chosen in the UI (e.g. 'Artistic HDR') or several are blended manually and then oversampled, ending up with the usual 5MP/20MP image pair. But it's worth noting the byte size cost:

  • Typical 5MP image - 1MB
  • Typical 5MP/20MP image pair on a Lumia 930/1520 - 7MB
  • Total for a photo taken with Rich Capture on the 930/1520 - 27MB

In other words, your phone's storage will disappear four times as quickly when taking photos if you leave Rich Capture on all the time! Not that this is necessarily a problem - you might have a large microSD in a 1520 and be saving to this, but on the space-challenged and unexpandable Lumia 930 it's a serious consideration.

And compared to typical 1MB photos, space will disappear 27 times faster. The price of progress? In fairness, other multi-shot system on other platforms also use up far more space (e.g. HTC Zoes on the One M7), so it's somewhat endemic to computational photography, it seems.

The reframing workflow

Also notable from the chart above is how, by necessity, the reframing process is now handled by a separate application/module. This is partly because reframing works on the Image/Image_highres pair, something which isn't created until after any Rich Capture combinations, and partly for reasons of simplicity, both in the UI and in the underlying code. Having reframing be part of the other 'after the fact' image editing process feels natural and intuitive, and means that the Lumia Camera core UI can be kept cleaner.

In theory, reframing could be added again to Lumia Camera 5, but with the amount that's now in this application it's probably simpler all round to have it just part of Lumia Creative Studio - which is, after all, only a tap or so away at any point. Plus putting it back in the application would confuse things for the non-oversampling cameras, as on the 830/735, for which 'reframing' is literally just going to be cropping.

Photos, UI simplicity and universality(!)

Another valid question is to ask why you can't 'Edit Rich Capture' from the Photos application? After all, in the past, you could 'Open in Nokia Camera'. This could still be an option if Microsoft chose to implement it, i.e. to open an image for which there's 'rich capture' information in Lumia Camera 5. It's messy though - to have to fire up the camera hardware and UI, just to dive out of it into the Rich Capture editor system again. Then back out through the fired up camera to where you started out from.

What about copying the rich capture blending code into Photos itself? Quite apart from the duplication of functionality, Photos is part of Microsoft's new 'universal app' scheme and the same application is going to have to exist and work across phone, tablet and desktop - and it's already complicated enough, without having to take on extra high-end-phone-specific editing functions.

And it's not as if you can't still 'Edit Rich Capture' after the fact - all you have to do is open up Lumia Camera 5, tap the top-left gallery thumbnail and then swipe backwards until you get to the photo you want to change (the chances are that it will be a fairly recent one).

Photographic limitations

There's one other downside of Rich Capture which hasn't been mentioned by Microsoft/Nokia at all - and it's rather obvious. Because you're dealing with multiple photos, taken a discrete interval apart (even if only 1/20s of a second or so), anything that's moving in the shot, any noticeable wobble in your grip that's beyond the OIS's capabilities to cancel, either will cause there to be ghosting issues when combining different images in the Rich Capture editing system. Typically these would be people laughing or turning their heads or walking by in the background. Or cars behind, that sort of thing. 

Of course, this isn't actually a total showstopper, because you have at least one option (and usually two) in terms of picking a version of the image which doesn't contain any combined information from the other images. So the situation's salvageable. But it's worth noting, at least. And if you want to snap people then it still pays them to at least try to pose or otherwise stay still!

Strategy

In view of all of the above, I believe the best strategy is to leave Rich Capture 'off' in Lumia Camera most of the time, saving space, battery, processor time, and more. Instead, you can save it for situations where the light is challenging:

  • shooting into a bright sky
  • using fill-in flash
  • night shots

The rest of the time, use the traditional 5MP PureView oversampling (and reframing) system, happy that with Lumia Camera 5 even this gets improved, since the oversampling itself is now in the background and you get to snap your next shot much, much faster.