I just got early access to ElevenLabs' AI sound effect generator — here's what it can do
ElevenLabs is the most prominent and one of the best generative artificial intelligence sound platforms available, creating synthetic voices and cloned voices from tiny samples.
Now it is moving into sound effects, allowing creators to generate a range of noises from a simple prompt that could be used in presentations, movies or podcasts.
I’ve tried out the tool and it can create some complex, multilayered sound effects from a relatively simple prompt including animal, atmosphere and technical noises.
It isn’t perfect yet, it struggles with certain noises and couldn’t get the sound of a mid-90s modern at all, giving it too much musical tone and missing the harsh clicks but overall it is impressive considering the sounds are AI generated and not recorded.
How do ElevenLabs sound effects work?
Like any generative AI tool ElevenLabs sound effects model was trained on a vast amount of data including record sounds of real-world objects.
Each object, animal or scene used in the training dataset would have been well labelled and described, allowing the AI to learn to replicate the noise.
To use the tool you get a standard text box that will be familiar to anyone that has used more or less any generative AI tool, and a button to generate.
ElevenLabs sound effects generator gives you five sounds from the same prompt and unlike AI image and video, each version can vary significantly from one another.
How well does ElevenLabs sound effects work?
It is particularly good at weather sounds and those of the natural world including animals, but it seemed to struggle more with electronic sounds.
I gave it seven prompts to see how well it handled a range of sounds. This included a crackling campfire, ocean waves, thunderstorm, wind chimes and an old-fashioned train whistle.
Over all it did well on most prompts, although of the five versions it generates in response to a prompt, usually only one or two were good and longer than a few seconds.
The tool is also built in such a way that you could double up on sounds. For a video of waves crashing on the beach I found I needed two sounds, one of waves and one of seagulls as neither captured the full tapestry shown in the video.
Sound is a rapidly growing area of AI, from music tools to synthetic speech and now on to sound effects and custom noises. It gives a new dimension to generated video, a new form of storytelling for slides and makes podcast creation more immersive.
