Merging purity, intelligence and flexibility - ProRAW on iOS 14.3

Published by at

For the last 20 years of smartphone cameras, from the earliest Symbian handsets (Nokia Nseries, mainly, then the 808 PureView) through the Lumias (1020, 950, mainly), and with iPhones and Android handsets also providing highlights here and there, users have had two main options in terms of phone imaging, both compromised. That changes this week, do please read on.

Traditionally, phones either output a JPG file and/or a RAW file (usually in DNG format). Both have big disadvantages:

  • Most phone camera software adds loads of sharpening and edge enhancement before the final JPG file is produced. The idea is to look good on the phone screen, but looking at the JPG in detail shows horrible liberties usually being taken. Typically, you can't crop part of the imageto see more or to reframe because then the pixel-level artefacts are just too ugly. In addition, JPG output is fairly small, a few Megabytes, meaning that plenty of lossy compression is also applied, resulting in even more artefacts. And, once encoded, the JPG file and its less desirable properties are set in stone, so you've lost any pretence at going back to the image later and reworking it.*
     
  • RAW output has been possible in top end phone cameras since about 2013. The two big disadvantages are the image size (not such an issue in 2020, thanks to storage capacities and chip speeds) and the very 'unprocessed' nature of the output. Because a RAW file contains data more or less straight from the sensor (typically it's colour and luminance data, with loads of noise, and only the basic Bayer filtering from the RGB matrix having been done) and all grabbed from a single exposure, you've lost all the advantages of the image being taken on a smartphone, i.e. you've lost all the intelligence that normally handles HDR, white balance, noise reduction, multi-frame combination, and so on. So what you end up with is pale and uninspiring until you bring it into a photo editor that supports RAW and get to work experimenting with a dozen parameters. And hopefully a wealth of image tweaking experience - you'll need it.

[* the 2013 Lumia 1020 was an obvious exception here, being able to capture a super-hi-res (38MP) JPG AND a PureView over-sampled JPG at the same time. Well, in a few seconds, at least. As a result, you could reframe an output image from the underlying hi-res original as much as you like. Pretty cool, even if super slow by today's standards.]

So, this week saw the release of Apple's ProRAW system and tweaked onboard software on the latest iPhone 12 Pro series. My iPhone 12 Pro Max got the iOS 14.3 update and I've been playing with what Apple has implemented, and the good news is that it seems to offer the best of all worlds.

Screenshot

Note the 'RAW' toggle, top left...

Essentially, at capture time, you decide whether you want to shoot in normal mode or RAW mode. There's a visible toggle in the Camera interface. For most casual snaps - your food, a sign, a document, something casual, or a trendy 'Portrait' blurred background snap (ProRAW can't work with this system, as it involves combining results from multiple cameras), normal mode is fine and who cares if things are a bit jagged, a bit enhanced, a bit artificial, down at the pixel level - after all, no one, even you, are going to examine these shots in extreme detail.

But when you're faced with something special? A sunset. A picturesque garden. A river scene. A seaside panorama. A posed shot of a loved one. Shots like these - especially on a 'Pro' iPhone - are ones that you're almost certainly going to do things with later. Social sharing, of course, but also cropping in for arty effect, printing out at large size for a poster, perhaps. This is where you want the absolute maximum natural detail, with no populist edge enhancement or over-sharpening. After all, you can always enhance all these things later, under your own control. But you can't go the other way around. So, for these special/arty photos, you can toggle ProRAW on.

But the cleverness of Apple's new system is that ProRAW images aren't just the image data from the sensor, from a single exposure, as is the case with traditional RAW files (e.g. on the disappointing Nokia 9, famously). Apple is selecting image data from much later in the imaging workflow. So you can use any of the phone's three cameras (ultra-wide, main, telephoto), multi-frame HDR and Night mode shots can still be composited, Apple's Deep Fusion and Smart HDR algorithms can still work, recognising what you're shooting and optimising the image processing. All of this happens, creating the very best image possible, and only THEN does the ProRAW .DNG file get created. 

As far as I can tell, the ProRAW DNG file still has the original, traditional RAW parameters, for maximum compatibility with existing editing packages. Yet it also has the results of Apple's composition algorithms and calculations. Hence 'ProRAW', i.e. RAW and then a bit more added on. In practice, this means that viewing a captured ProRAW DNG file in the iPhone 12 Pro's Photos application will show a very similar result to a regular JPG shot - at phone screen size, at least. All the HDR and multi-frame processing, all the colour correction, all the noise reduction, it all shows up in Photos, and you can treat it like any other 'smart' image when sharing it around.

Yet... Crucially... the ProRAW file is saved without any of the (to me, objectionable) sharpening and edge enhancement that normal JPG output is treated to. And, obviously, with no JPG compression artefacts. So, apart from the sheer byte size of the files (25MB ish), what we have is a way of capturing ultra-detailed, ultra-natural, 'smart' photos, often multi-frame, often with HDR benefits, yet without any objectionable artefacts.

Add in the usual way (on iOS, at least) that cropping into photos doesn't destroy the original image, so you can - as on the Lumia 1020 - reframe away to your heart's content, and you can see that ProRAW on the iPhone 12 Pro range is potentially as close to phone imaging nirvana as we've ever been.

Night church scene

Typical low light scene, captured here in ProRAW (and also standard JPG, see below for crops)

1:1 crops from the same scene captured in standard (JPG) and ProRAW modes - the most obvious difference in these crops is the way texture and imperfections on the church structure have been thickened for the straight JPG capture, while the building detail is beautifully clean in the ProRAW capture. More examples below, of course!

Now, at this point someone is going to pop up and say 'But you're missing the whole point of RAW - you can edit these files in Lightroom/Photoshop/whatever and do outrageous edits!' That's true, but then that's always been the case for RAW files from phone cameras and Apple's solution only steps the game up in this regard if you have a ProRAW-compatible photo editor (usually on a desktop, though there are now RAW editors on phones too), in which case Apple's enhanced photos become the starting point for your edits rather than a bland washed out affair that needs a lot of manual work.

Scaled crop

For illustration only, the framing isn't exact, but here are scaled crops from three different image files: from left to right: the default JPG capture (all HDR and processing, including sharpening and edges); the plain DNG file, as analysed in a non-ProRAW compatible editor; and the ProRAW-interpreted DNG file on a compatible device/OS version. Apple's processing does well for the JPG here, while ProRAW initially looks duller - but remember that you have full quality to work from in tweaking this, lifting shadows, and so on. While the plain DNG, seen on my (as yet ProRAW-incompatible) Mac's Preview utility is initially almost unusable, but - with skill - you can rescue it, of course. If you have what it takes!

And that's all well and good if you want to be supremely creative in the edit suite, as it were, and if you know your shadows from your blacks from your highlights. That's all available with ProRAW and good luck to you. This is the aspect of ProRAW that most other sites are talking about, but I see the benefits of Apple's new system from a very different standpoint.

I'm coming from years of Nokia PureView workflows, in particular. What we have here on the iPhone 12 Pro range with iOS 14.3 and ProRAW as an actual capture mode is the Lumia 1020's iconic workflow brought bang up to date:

  • Shoot a super photo (in ProRAW mode), taking advantage of modern benefits like multi-frame HDR and Night mode multi-exposure combinations - it's all fully automatic, so you can just point and click in the Camera application.
  • Share the photo through various apps and services - in many cases the source ProRAW DNG file is converted to a 'pure' JPG, i.e. without sharpening or edge enhancement. So people see natural images, not hyper-real. In theory anyway - see the emboldened paragraph below! It's early days...
  • Perhaps crop the photo down to a component part, e.g. one person or one part of a building - you're still limited by the 12MP resolution, but at least a (e.g.) 3MP output crop to JPG will still look great because every pixel is as pure as it can be, with no populist enhancement artefacts.
  • Later, you decide to share a different crop (i.e. 'reframing'), or perhaps go back to a full frame, and in each case the source ProRAW DNG file is still available and used as the source, so there's no loss of data.

You get the idea. It's exactly the sort of thing we Lumia 1020 owners used to do in the glory days of Windows Phone, except that now everything's at least a hundred times faster and on a platform that's fully stocked with services and fully up to date.

This all being new, things are still in flux. Apple's own Photos app on my older 'Catalina' iMac can't open ProRAW files properly ('Big Sur' is needed). Microsoft OneDrive backs up a JPG version (similar to if you'd opted for standard JPG/HEIC), AirDrop to a Mac shares the ProRAW DNG file, but the Mac then doesn't yet know what to do with it in terms of imaging smarts, so you get a vanilla DNG interpretation. Sharing to Google Drive results in a PNG file(!!!!) with lowered quality! Adobe Lightroom on the phone is supposed to be compatible with ProRAW but actually isn't... yet, you get vanilla RAW with no ProRAW tweaks. Thankfully, Windows 10's very latest builds (20H2) can import ProRAW DNGs and Photos can display all the Apple edits.

In short, different apps and services behave differently and few are fully compatible yet. In most cases the lowest common denominator is JPG, but if you've, for example, done some cropping and tweaking work (before sharing) then at least the JPG is being generated from the undamaged ProRAW DNG master. This all needs revisiting in a few months when tools and services have updated and settled down.

I'll experiment with all this more when I do a big four way phone shootout over Christmas - watch this space. I'm thinking Lumia 950, Pixel 5, iPhone 12 Pro Max (standard and ProRAW). My (updated) intended workflow for getting ProRAW images in JPG form for analysis and sharing with readers here is to make a trivial Edit ('Saturation' up a trivial 1% is enough!!) to ProRAW images and then share via AirDrop - the edited status forces a JPG share, it seems. A little bizarre, but... Phew. Or I could try doing it all via Windows 20H2. Watch this space.

Zoomed scene overview

Another low light example, this time at dusk, but of my familiar plane test subject, using the iPhone 12 Pro's 2.5x telephoto camera at about 40m. It was a lot darker than modern phone cameras make it seem! So small aperture, low light, a tough test. Here are the crops below. Firstly the standard JPG capture and then, below it, the ProRAW capture:

1:1 crop for comparison

1:1 crop for comparison

The lack of sharpening and edge enhancement artefacts is astonishing. The bottom crop could have come from one of my PureView devices, the Lumia 1020 or 950, or even from the 2012 Nokia 808 (on a tripod!) - the plane's nose cone is flawless and the chain fence and its concrete posts all look real rather than a photocopy. Amazing.

Overall, I'm really happy with how Apple has implemented this. I feared that its RAW implementation would need third party applications and a lot of faffing about, but the reality is that you just turn the toggle on and take photos. And then what you see in the Photos application is immediately editable (in a non-destructive way), is free from over-sharpening and edge enhancement, and requires no extra software at all.

Example scene

Another low light example scene (low light is when Apple's 2020 JPG 'enhancements' are arguably most egregious). And then crops from JPG and ProRAW capture below:

1:1 crop for comparison1:1 crop for comparison

Yes, I'm pixel-peeping here, but look at the small Santa figure, look at the low light textures, my eyes definitely prefer the ProRAW version, it seems more natural. Though I accept that casual observers will find the JPG leaps off the screen more - which is fine, they can use JPG and I'll use ProRAW and we're both happy. Nice to have the choice on the same device!!

Will any of this - either the ProRAW format itself or the general idea (outputting image data after the 'smarts' but before the sharpening) - make its way to other platforms and devices? The jury's out on this, though I wouldn't be altogether surprised. Google's own RAW implementation is halfway there already in the Pixels, so I guess that's the next one for me to dig into and report back from. For now though, Apple has at least a year's lead on the rest of the phone camera world in much the same way as Nokia had with its PureView devices in 2012 and 2013.

PS. There will also now have to be some imaging retests - I'm looking forward to it. Apple just moved the goalposts for phone imaging and I'm chuffed to be among the first to see what it can do.

PPS. See also this short page on Apple's site, which summarises the best case ProRAW sharing scenario when everything's updated.