A voice know-how firm which makes use of synthetic intelligence (AI) to generate real looking speech says it would introduce further safeguards after its free device was used to generate superstar voices studying extremely inappropriate statements.
ElevenLabs launched a so-called voice cloning suite earlier this month.
It permits customers to add clips of somebody talking, that are used to generate a synthetic voice.
This can then be utilized to the agency’s text-to-speech speech synthesis characteristic, which by default presents an inventory of characters with numerous accents that may learn as much as 2,500 characters of textual content directly.
Read extra:
Ukraine conflict: Deepfake video of Zelenskyy telling Ukrainians to ‘lay down arms’ debunked
‘Google it’ no extra? How AI may change the way in which we search the online
It did not take lengthy for the web at massive to start out experimenting with the know-how, together with on the notorious nameless picture board website 4chan, the place generated clips included Harry Potter actress Emma Watson studying a passage from Adolf Hitler’s Mein Kampf.
Other recordsdata discovered by Sky News included what feels like Joe Biden asserting that US troops will go into Ukraine, and a potty-mouthed David Attenborough boasting a couple of profession within the Navy Seals.
Film director James Cameron, Top Gun star Tom Cruise, and podcaster Joe Rogan have been focused, and there are additionally clips of fictional characters, usually studying deeply offensive, racist, or misogynistic messages.
‘Crazy weekend’
In a press release on Twitter, ElevenLabs – which was based final 12 months by ex-Google engineer Piotr Dabkowski and former Palantir strategist Mati Staniszewski – requested for suggestions on methods to stop misuse of its know-how.
“Crazy weekend – thank you to everyone for trying out our Beta platform,” it stated.
“While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases. We want to reach out to Twitter community for thoughts and feedback!”
The firm stated that whereas it may “trace back any generated audio” to the person who made it, it additionally wished to introduce “additional safeguards”.
It prompt requiring extra account checks, reminiscent of asking for cost particulars or ID; verifying somebody’s copyright to the clips they add; or dropping the device altogether to manually confirm every voice cloning request.
But as of Tuesday morning, the device remained on-line in the identical state.
The firm’s web site suggests its know-how may someday be used to provide voice to articles, newsletters, books, academic materials, video video games, and movies.
Sky News has contacted ElevenLabs for additional remark.
Dangers of AI generated media
The deluge of inappropriate voice clips is a reminder of the perils of releasing AI instruments into the general public sphere with out enough safeguards in place – earlier examples embrace a Microsoft chatbot which needed to be taken down after shortly being taught to say offensive issues.
Earlier this month, researchers on the tech big introduced that they had made a text-to-speech AI known as VALL-E that would simulate an individual’s voice primarily based on simply three seconds of audio.
They stated they might not be releasing the device to the general public as a result of “it may carry potential risks”, together with folks “spoofing voice identification or impersonating a specific speaker”.
The know-how presents lots of the identical challenges as deepfake movies, which have turn out to be more and more widespread on the web.
Last 12 months, a deepfake video of Volodymyr Zelenskyy telling Ukrainians to “lay down arms” was shared on-line.
It got here after the creator of a collection of real looking Tom Cruise deepfakes, albeit extra light-hearted clips purporting to indicate the actor doing magic tips and enjoying golf, warned viewers concerning the know-how’s potential.
Source: information.sky.com”