元Decides Not to Release AI That Can Mimic the Voices of Everyone You Know元has announced a powerful AI model that can replicate voices. What could possibly go wrong?

Last week, Meta announced its latest AI innovation, Voicebox, a powerful tool that can learn to speak and sound like whoever you want. According to Meta'spress release, Voicebox's uses will include:

• Producing high-quality audio clips and editing pre-recorded audio — like removing car horns or a dog barking
• Producing speech in six languages
• Giving natural-sounding voices to virtual assistants and non-player characters in the metaverse
• Allowing visually impaired people to hear written messages from friends read by AI in their voices

If that sounds cool and also terrifying to you, you're not alone. Even the developers at Meta were like,Whoa, let's pump the brakes here. The company decided not to release Voicebox to the public, basically because the world is full of terrible people who will most certainly do terrible things with it.

Related:元Wants 10-Year-Olds in Metaverse, Raising Major Red Flags

"As with other powerful new AI innovations, we recognize that this technology brings the potential for misuse and unintended harm," Meta researchersposted. "While we believe it is important to be open with the AI community and to share our research to advance the state of the art in AI, it's also necessary to strike the right balance between openness with responsibility."

元's developers say that, armed with an audio sample that is just two seconds long, "Voicebox can match the sample's audio style and use it for text-to-speech generation."

Meaning that someone who has a recording of you speaking for a few seconds will be able to type in any words they want you to say, and it will sound like you are saying it. The potential for criminal, political and personal harm is insanely high. Ifscammerscan convince people that they are from the IRS and get them to buy gift cards for them, imagine what kind of damage they could do if they sound just like a loved one or your boss.

Related:元Commands Workers Back to Office, Says It's 'Easier to Build Trust in Person'

元has released apaper about Voiceboxdetailing how it works, its potential applications and ways to mitigate its risks. And the company says that it "looks forward to continuing our exploration in the audio domain and seeing how other researchers build on our work."

Looking forward or dreading? If you've ever seen theBlack Mirrorepisode "Be Right Back" where a grieving wife uses AI tech to power a very lifelike android of her dead husband, you know that this is going to get very weird very fast.

Just realized the@blackmirrorepisode "Be Right Back" is happening now with Ai.pic.twitter.com/5x0nZCnKkh

— Domenic | hackingpain.eth (@theonlydomenic)June 13, 2023