Neil Young's PonoPlayer: The Emperor Has No Clothes

It was one of Kickstarter’s most successful campaigns. Its inventors sought $800,000 in funding from the public — but raised a gigantic $6.2 million.

The project: the PonoPlayer, “a revolution in music listening.” It was designed to play back music files that use up to 20 times more data than the MP3 files that gave the first pocket music players a bad name.

“Everyone who’s ever heard PonoMusic will tell you that the difference is surprising and dramatic,” Pono wrote on Kickstarter. “They tell us that not only do they hear the difference; they feel it in their body, in their soul.”

In Pono’s Kickstarter pitch video, famous musicians react to the Pono sound they’ve just heard. “That music made me feel good. Much better than I’ve felt in a long time listening to music,” says Norah Jones. “This gives it to you as good as you can get it,” says Tom Petty. “MP3 [the old format] is like seeing a Xerox of the Mona Lisa,” says Elvis Costello.

Neil Young, celebrity founder and driving force for Pono, points out that MP3 is a compression scheme. It was developed in the era of music players with limited storage capacity; the idea was to shrink the music files by discarding music data from the original recordings. But these days, storage is copious and cheap. So why are we still compressing our music? Why can’t we listen to our music the way it was recorded in the studio, according to the musician’s original intentions?

The Pono Player, once just a Kickstarter prototype, is now a product that anyone can buy, for $400. To hear the magic, you’re supposed to buy all new music—high-resolution audio files—from Pono’s new music store (ponomusic.force.com), and load them onto your Pono using a new Mac or PC loading-dock program (Pono World). Albums cost about $25 each.

You’ve got to admit it: The argument for the Pono Player sure is appealing — that we don’t know what we’ve been missing in our music.

Unfortunately, it isn’t true.

I’m 51 and a former professional musician. I know how to listen. But when I bought Pono’s expensive remastered songs and compared them with the identical songs on my phone, I couldn’t hear any difference whatsoever.

I got worried. Is the Pono story a modern-day “Emperor’s New Clothes” fable? Were those famous rock stars just imagining things?

There was only one way to find out: conduct a blind trial, using identical songs on identical headphones, comparing the Pono with a standard audio player — an iPhone. So that’s what I did. You can watch the process in the video above.

The player
The Pono Player is big (5x2x1 inches), an awkward chunky soft-touch plastic prism, like an oversized Toblerone chocolate bar. The company says this shape permitted its designers to choose larger, higher-quality audio components and to separate them enough to prevent electrical interference — but you can’t carry it comfortably in a pocket.

There’s an eight-hour battery, 64 gigabytes of storage (enough for 400 of Pono’s highest-resolution songs), a memory-card slot for another 64 gigabytes, and a low-resolution touchscreen; you use dainty finger swipes to scroll through your music or adjust settings. On my player, the lower edge of the screen stopped responding to taps after one day, which made navigation difficult.

The test
How does it sound? I found 15 volunteers, ages 17 through 55. Each subject put on nice headphones — Sony MDR 7506 — and listened to three songs of different styles (“Saturday in the Park” by Chicago, “Raised on Robbery” by Joni Mitchell, and “There’s a World” by Mr. Pono himself, Neil Young). I bought these songs twice: once from the Pono store, in high resolution, and once from the iTunes store.

Each subject then listened to the same songs again, using standard Apple earbuds.

I connected both the Pono Player and an iPhone to an A/B switch; I instructed my test subjects to flip back and forth between the two at will.

The subjects would not know which player was A and which was B. In fact, after each song I disconnected both players and reattached them to the A/B box, sometimes the same way and sometimes the opposite way, so that the subject could never get lazy and keep proclaiming, for example, that B sounded better.

During playback, the subjects were free to compare the two playback sources with as much scrutiny as they wished. After each listen, the subject announced a sound preference: A, B, or neither.

The results
The results surprised even me. Whether wearing earbuds or expensive headphones, my test subjects usually thought that the iPhone playback sounded better than the Pono Player.

Among those who could hear any difference, I asked how much difference there seemed to be. I wanted to see whether this was some astounding, worthwhile improvement or perhaps partly imagination.

On average, my participants said that when they heard a difference at all, it was about 10 percent.

Fourteen of my 15 participants said they would not spend $400 for a new player (not to mention buying all new songs) to attain that difference.

Explaining the results
The results baffled me. The Pono video shows dozens of professionals reacting very differently to their A/B tests. How could my results be so different?

Well, first of all, they listened to the Pono in a car, instead of on headphones in a controlled environment. (What the heck?) Anyway, just to make sure I wasn’t missing something, I also conducted a couple of A/B tests in my car. If anything, there was even less audible difference.

So I wrote to Pono — and heard back from Neil Young himself.

“Of approximately 100 top-seed artists who compared Pono to low resolution MP3s,” he wrote, “all of them heard and felt the Pono difference, rewarding to the human senses, and is what Pono thinks you deserve to hear.”

Aha — there’s a key phrase in there: low-resolution MP3s.

My test compared Pono files against Apple’s iTunes files, which come in 16-bit/256Kbps AAC format (more on formats below). That’s much better than the radically compressed MP3 files of 1998.

There’s another factor at play here, too: Pono is going to extraordinary lengths to acquire remastered versions of the songs in its catalog. “If we are looking for a popular master and find it has not been sampled at the highest rate, we try to access it and, with the cooperation of labels and artists, maximize the recapture at the highest resolution,” Neil Young wrote to me. “We reach out to the creators, if they are still with us, to include their knowledge in the mastering. Sometimes they will even supervise it. This is a long process, but we are providing the absolute best available and pushing for improvement in resolution for maximizing the labels/creators’ art whenever possible.”

Clearly, if Pono’s testing involved a remastered, high-resolution audio file going head-to-head with an original, crummy MP3 of the same song, you’d hear a difference.

The thing is, that’s not a fair test. The music we buy today from iTunes, Amazon, Google Play, and similar online stores has much higher quality than low-res MP3 files. Why would you spend $400 on a new player and re-buy all of your music files if the result sounds no better than what’s on your phone already?

About high-resolution audio
When geeks talk about music-file quality, they bandy about two numbers: the bit rate and the sampling rate. In simple terms, these measurements specify the volume and frequency ranges of a music file. This article explains them well — but the short version is that, up to a point, higher numbers are better.

Music CDs, and the downloadable songs you buy, are sampled at 16 bit/44.1kHz. The songs you buy from Pono, on the other hand, go as high as 24 bit/192kHz. That means more bits of data per instant of sound, and more (smaller) instants per time period: higher resolution. It’s like having more color data and more pixels per inch in a photo.

In theory, the higher these numbers are for a music file, the closer it sounds to what the musician recorded.

In practice, though, there’s an infinitude of footnotes, exceptions, and variables. What headphones you’re using. How and when the recording was made. Whether the recording was originally recorded in analog, or in high resolution, or just converted. Whether the discarded frequencies are, in fact, audible to a human being.

Now, Pono is absolutely clear on one thing: “You will hear the difference.”

But scientific studies say you won’t. Here’s one written up in the journal of the Audio Engineering Society; here’s one from The Guardian. They say that the human ear can’t even detect audio quality beyond what’s on a CD (16 bit/44.1kHz). High-resolution audio includes more music data, yes — but it’s sound you can’t hear.

The store
The Pono store is very new, and it shows.

The company says it has 2 million songs for sale, but 90 percent of it is in 44.1kHz format — no better than what’s on a CD. The remaining 10 percent, the good stuff, the remastered high-resolution songs, is hard to find.

There’s no way to view only albums available in high resolution. In fact, when you search for a band, a list of its albums appears — but you can’t see what resolution they’re available in until you click a button below the album art. Over and over for each album.

For example, there’s nothing in high resolution from Bruno Mars, Taylor Swift, Katy Perry, Iggy Azalea, Lady Gaga, Sam Smith, Keith Urban, Tina Turner, Celine Dion, Loretta Lynn, or (heh) The Pogues.

At 96kHz (which is still not 192), there’s one album each from John Mayer, Kid Rock, Harry Connick Jr., David Bowie, Sting, Carole King, and Blake Shelton. Out of Tony Bennett’s 68 live and studio albums, only two are available at 96kHz.

The Pono store is almost completely devoid of high-res classical music, which is baffling — wouldn’t classical fans cherish high audio quality as much as rock fans?

There are other sources of high-res music online, but it’s of widely varying quality.

The Pono landscape
You may remember that 14 of my test subjects said they didn’t hear enough difference to justify buying a Pono. The 15th guy, however, said, “I would and I did!”

That is, he was already the owner of a Pono.

I pointed out to him that in my test, even he had preferred the sound of the iPhone. His reply: The Pono may not actually sound better, but it delivers more emotion.

This, I think, is what this controversy boils down to. “OK, OK, you can’t hear any difference. But you can feel it,” the high-res-audio fans say. This, too, is what Neil Young says about the Pono — that it restores the music’s “soul.”

Well, OK. But now we’re getting into squishy territory. It’s like saying that wearing a crystal or a magnet makes you healthier: There’s no scientific or measurable basis to the statement, but then again, if it works for you, nobody can argue with you. (They try, though. The battles rage on in every comments section of every article written about high-resolution audio. This one, from Rolling Stone, is less vicious than most.)

To me, the Pono Player story is a modern retelling of “The Emperor’s New Clothes.”

But maybe you’re different. Maybe you believe that high-res music has more “soul” or is less “tiring” to listen to, as fans claim, even if you can’t actually hear a difference.

In that case, you should note that Pono is not the only high-res player. It has many rivals, like the Astell & Kern AK100 Mk II, iBasso DX50, and the new Sony NWZ-ZX1 Walkman. In fact, Android phones can play the same FLAC-format files, if you get right down to it.

Neil Young and the believers in high-res audio aren’t fools, and their hearts are in the right place. But Pono’s statement that “Everyone who’s ever heard PonoMusic will tell you that the difference is surprising and dramatic” is baloney. When conducting the test with today’s modern music files, I couldn’t find even one person who heard a dramatic difference.

Furthermore, there’s another elephant in this room: Most people listen to music while driving, running, walking, working, talking, or hanging out — not sitting in the easy chair, eyes closed, focused solely on the music. In those situations, there are enough distractions and background noises to make any differences even less noticeable.

My advice: If you want a better, richer, better balanced, less tiring, more comfortable listening experience, you don’t have to spend $400 on a new player and throw away your existing music collection.

Just spend a couple of hundred bucks on a nice pair of headphones.