How AI helped us hear the "final" Beatles song: "It's the closest we'll ever come to having John back in the room"

Today, The Beatles release their final song. 45 years in the making, Now And Then features vocals recorded by John Lennon in the '70s and extracted from a demo cassette using AI-powered stem separation technology. 

Thanks to advances in stem separation made over the past few years, Peter Jackson (maker of the Get Back documentary series) and a team of engineers were able to extract Lennon's vocal from the demo to be mixed into a new version of the song, featuring newly-recorded bass and drums from Paul and Ringo, and guitar parts George Harrison recorded in 1995.

Stem separation software separates recordings containing multiple instruments into their constituent parts, or stems; this made it possible for the producers to integrate Lennon's vocal with other musical elements when mixing the song's new version. When the remaining Beatles first considered recording the demo in 1995, this software didn't yet exist, so the song was shelved.

Stem separation tools utilize a form of machine learning wherein the software is trained using thousands of existing songs to understand and recognize the frequency bands that individual elements of a mix, such as vocals, guitars and drums, typically tend to occupy.

READ MORE

We tested 5 of the best stem separation software tools (and the best one was free)

Using a process called time-frequency (TF) masking, the combination of frequencies that make up a piece of music is filtered, enabling us to pick and choose which frequencies to keep or to get rid of. This is how a relatively clean vocal acapella was lifted from a low-quality demo tape in which Lennon sings over the piano. You can hear the demo recording and Lennon's separated vocal in the short film embedded below.

While the Get Back team reportedly developed custom-made stem separation software, nicknamed "Mal", during the making of the documentary, tools of this kind can also be found in a number of apps, plugins and browser-based platforms, and have recently been integrated into the popular DAW FL Studio. (If you'd like to try it out, we'd recommend using Gaudio Studio, a free stem separation platform.)

"The sound restoration is the most exciting thing", Jackson said in an interview with Variety in 2021. "We made some huge breakthroughs in audio. We developed a machine learning system that we taught what a guitar sounds like, what a bass sounds like, what a voice sounds like. In fact we taught the computer what John sounds like and what Paul sounds like.

"So we can take these mono tracks and split up all the instruments we can just hear the vocals, the guitars. You see Ringo thumping the drums in the background but you don’t hear the drums at all. That that allows us to remix it really cleanly.”

Though Jackson utilized stem separation in the making of Get Back, it was McCartney who was the first to wonder if the same technology might be able to help rescue Lennon's voice from a demo cassette left to him by his late bandmate. He was overjoyed to find out that it could.

"We could lift John's voice without lifting the piano, which had always been one of the problems. Now we could mix it and make a proper record of it." McCartney said in the short film made to celebrate the song's release. "There it was, John's voice, crystal clear. It's quite emotional."

"It was the closest we'll ever come to having him back in the room, so it was very emotional for all of us," Ringo continued. "It was like John was there, you know. It's far out."

"My dad would have loved that. He was never shy to use recording technology - I think it's really beautiful." added John's son, Sean Lennon.

The use of artificial intelligence in the song's production initially provoked some controversy, as some fans weren't clear on exactly how the AI would be utilized, or whether it would be used to generate fake vocals that weren't originally recorded by Lennon through AI voice modelling.

McCartney was quick to reassure fans, however, telling Rolling Stone that The Beatles would "never" fake Lennon's voice using AI. "Nothing has been artificially or synthetically created", he continued in a Twitter/X post. "It’s all real and we all play on it. We cleaned up some existing recordings - a process which has gone on for years."

The Beatles' Now And Then is out now.