As ever, writing about audio isn't trivial, since your ears can't be here to try things and comment with me. So I'll have to run through some of my methodology before launching into actual tests.
A few basics though. I'm testing wired audio here, usually through a 3.5mm socket in the phone but I also threw in a dongle or two, since that's a valid 'last resort' way to go.
In each case, the smartphone's music application (or video, etc.) takes a digital music source (whether local or streaming) and uses the Digital to Analog Conversion (DAC) capabilities of the phone's hardware (either in the main chipset or as a standalone sub-system or perhaps in a DAC dongle) to produce 'analog' voltages, for left and right channels and with a common ground. Leaving aside microphone complications, these analog voltages are turned into tiny speaker movements in your chosen headphones and - hopefully - you hear music in your ears.
And of that seems complicated enough, there are plenty of other variables:
- The digital source might be compromised, e.g. low bitrate or a poor codec, or simply a poor original (e.g. off vinyl) when the source was created.
- The headphones used are very important, since 99% of the wired headphones I see used out in public are pretty ropey. For example, use some cheap outer-ear headphones (e.g. Apple wired in-box white headphones, or clones thereof) or even cheap in-ear headphones (again, usually provided in the box) and you'll be missing many audio frequencies (bass and high end), in addition to also hearing noise 'pollution' from the world around you.
- Your own hearing may be compromised, either through age or over-exposure to loud sounds or music(!)
Now, there's not much that can be done for the last point, but I can certainly address the first two (and you should too), in selecting my test material and equipment. I tried to pick from a range of music genres, settling on, all at 320kbps MP3:
- Kari Jobe's Majestic live album, track "Hands to the Heavens" - the production on this is superb and there's plenty of bass and high end to test.
- Take That's "Reach out", from their Beautiful World album - again exemplary production and a full range of frequencies to listen for.
- "Vierne: Messe solenelle op. 16: Kyrie" from Cathedrale Notre-Dame De Paris - I was there and know how a full choir and organ in that amazing place should sound!
I did my listening through my much-loved and much-used Veho Z-8 over-ear studio headphones - they sound fantastic and as I write this (by coincidence!) are currently "75% off" (yeah, I know, take these discounts with a pinch of salt!) on Amazon UK, at £25. Just saying. These were originally over £50. Here's my Z-8 review from back in the day. I know, I know, audiophiles are going to say that I needed to use £200+ headphones as my reference, but I trust these Z-8 and have lived with them for several years.
And so on with the testing. Obviously, you're not here with me to listen yourselves (and doing a video or audio sample wouldn't help, since you'd then be at the mercy of the audio capabilities of however you watched/listened), so you'll have to rely on my scoring. Hey, it's all only a guide and I'm more interested in the relative differences, i.e which phones/DACs are better and by what margin?
Finally, I can only report on what I have to hand. You'll have to infer audio quality from related hardware. I do give some idea below - manufacturers usually re-use the same chipsets and (of course) Music applications across their ranges. The list just constitutes data points. All marks are out of 10.
And, this being AAWP, I'll start with a load of Windows-powered phones:
| 3.5mm-equipped phone (or DAC) |
Similar to§ (if any) | Perceived volume+ |
Bass++ | High end* | Separation /Fidelity** |
Total |
| Lumia 950 XL | Lumia 950 | 9 | 9 | 10 | 9 | 37 |
| Alcatel IDOL 4 Pro | 10 | 10 | 10 | 10 | 40 | |
| Wileyfox Pro | 7 | 8 | 8 | 9 | 32 | |
| Lumia 930 | 1520 | 10 | 10 | 9 | 9 | 38 |
| Samsung Galaxy S9+ | S9, Note 8/9, S10, S10+, S10e | 9 | 9 | 9 | 9 | 36 |
| Moto G7 Plus | G7 Play, G7 | 7 | 9 | 8 | 8 | 32 |
| Apple iPhone SE | iPhone 6, 6s, 6s Plus | 9 | 8 | 8 | 9 | 34 |
| Sony Xperia XZ1 Compact | Xperia XZ1, XZ2 | 10 | 9 | 10 | 10 | 39 |
| Marshall London | 10 | 10 | 10 | 10 | 40 | |
| Google Nexus 5X | Nexus 6P, Pixel, Pixel XL | 8 | 9 | 9 | 8 | 34 |
| Google DAC dongle | as used on Pixel 2 and 3 series and other Type C devices |
9 | 10 | 9 | 10 | 38 |
| HIDIZS Sonata HD II | third party DAC dongle | 11(!) | 10 | 10 | 10 | 41 |
Notes:
All phones were tested with 'flat' EQ, i.e. testing default output. Most smartphones (including the Lumias here) have an extra software Equaliser than can be applied to some (or all) output, but these are just software EQ effects and muddy the testing waters too much. This is analagous to me testing phone cameras on 'auto' and not in a 'pro' mode, where things get exponentially more difficult to compare.
§ By which I mean other phone models which will sound the same, usually because they're sister devices in a range.
+ Different DACs put out different amounts of 'oomph'. A decent audio amplifier will easily drive headphones with 'volume to spare', while on lesser chipsets you'll be at maximum volume on the phone and no headroom to increase it further. For example, the Alcatel IDOL 4 Pro puts out so much volume in my headphones at 23/30 while
++ I'm listening for rendering bass right down to 20Hz and lower - I like to hear bass guitar notes in their natural entirety, as if I'm standing by the bass amp (I play guitar and bass) and hearing the subtleties. Most headsets fall down here, providing insufficient (or software-boosted) bass.
* I'm listening for details in voices and percussion, with no compression artefacts - again, as if I was in front of the singers or percussion, hearing them 'live'.
** Poor smartphone audio often results in weak stereo separation, with too much common ground between channels. I'm also rating here for an overall impression of fidelity. And, yes, enjoyment.
Verdict
What became apparent, somewhat surprisingly, is that music output and enjoyment on smartphones is less about absolute audio rendering, i.e. less about how well the full range of frequencies could be output for a given audio source (though there are differences), and more about sheer volume/voltage. We've all been there - in a noisy city street and trying to really enjoy our music and even at a phone's maximum output it's only just loud enough. This is where the best music phones have volume to spare. (And no, I'm not going down the 'loud music will damage your hearing' route - I'm assuming that you're being sensible and not overdoing it or for too long a sustained period.)
I was expecting a few under-performers in my test here, though only the Wileyfox Pro really disappointed. It's audio output is just about good enough. Just about. Which is excusable given the price, I think. My Moto G7 Plus, running Android, matched it score-wise, but this also has a range of Dolby Atmos enhancers and equaliser presets that can be enabled, and this easily raises the phone on the music front, in terms of volume at least, by a couple of points.
I was impressed then, across the board, by how good most phones tested were. The IDOL 4 Pro, my favourite Windows 10 Mobile phone, and the music-specialist (and now sadly obsolete) Marshall London, running Android, came top, both with dedicated music chips/amplifiers and some serious output. (Update: now matched by the Sonata HD II DAC, added April 2019.) But the pack wasn't far behind.
I was also surprised that the Type C DAC-dongle tested, £12 or so from Google, also performed so well. It would be easy to dismiss such tiny electronic components, but they can work very well. Obviously, all such dongles aren't created equal (the HTC one is awful), but it's good to know that if we do end up going down a Type C-only route in years to come, that wired audio quality doesn't have to suffer, even if we do have to put up with extra bits of wire and plastic hanging off our phones. (In this article update (April 2019), the third party HIDIZS DAC/dongle is clearly louder, worth knowing if you really need your in-ear volume.)
Given a phone with decent volume output then, the bottle necks in terms of wired quality will be the capabilities of the headphones being used, which is why I've reviewed many such over the years, plus the quality of the source material - low bitrate 'pop-grade' MP3s definitely don't 'cut it'. Still, interesting data points, I think, and your comments most definitely welcome.
PS. With my Bluetooth experiments continuing in parallel, on this and other sites, you may now be wondering what's involved in testing Bluetooth audio in the same depth. The variables are slightly different here, with the DAC being moved to the wireless headphones and with a range of audio codecs that have to be matched at both 'ends'. It's complicated. But if there's enough interest, I'll do a deep dive into these in an upcoming feature.
PPS. You won't want to be wearing over-ears when you're out and about. The closest thing to 'studio quality sound' at a reasonable price is the Triple XXX headset here.