r/technology • u/gullydowny • Jan 30 '23
AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse Business
https://www.vice.com/en/article/dy7mww/ai-voice-firm-4chan-celebrity-voices-emma-watson-joe-rogan-elevenlabs4.9k
u/MIkeVill Jan 30 '23
4chan members used ElevenLabs to make deepfake voices of Emma Watson, Joe Rogan, and others saying racist, transphobic, and violent things.
Of course they did. Was anybody expecting anything else?
1.8k
u/AlexOfSpades Jan 30 '23
Surprised it wasn't just porn
1.2k
u/davidolson22 Jan 30 '23
Of course the porn came first
646
u/heureux13 Jan 30 '23
You do the porn first, that way, when you make the racist stuff, you have the porn to fall back on.
253
u/8urnMeTwice Jan 31 '23
First you get the money, then you get the power, then you get the AI generated actresses begging for it
→ More replies (2)40
u/BigDogsEatin Jan 31 '23
When do you get the khakis?
→ More replies (5)18
u/the_pontiff Jan 31 '23
Hey, Skidmark Steve! You still just hanging out, playing Nintendo?
→ More replies (3)→ More replies (4)10
u/TurboCake17 Jan 31 '23
“See, it does have good uses as well!” - They say, pointing to the deepfaked furry vore audios.
209
u/hero-hadley Jan 31 '23
That's disgusting! AI voice porn of Emma Watson?! Where? Where did they post those? There's so many of those. Which one? Which one did they post that on?
139
u/Tensza1 Jan 31 '23
I found it it was not that hard. here you go I don't know why, but I found it hilarious.
18
u/zacharykeaton Jan 31 '23
Well that has kind of scary implications. Feel bad for the celebrities who are going to have increasingly more sophisticated porn made of them without consent, and I don’t see how laws can really stop people making this stuff when the technology is freely available.
→ More replies (1)14
35
u/SavoryRhubarb Jan 31 '23
I’m right there with you.
I was expecting to be at least mildly aroused and instead actually audibly chuckled. Emma’s a great actress and all, but I just didn’t believe that she really wanted the b.d.
50
u/Pandatotheface Jan 31 '23
Its like a script written by a 13yr old being read off the back of a cerial box by Emma Watson.
29
u/Wallofcans Jan 31 '23
That's just someone messing around with the tech. It'll be crazy and scary when people put work into it.
50
u/eddyx Jan 31 '23
I love living in the future.
45
u/Important-Tune Jan 31 '23
The future is a weird fucking place where all you can do is laugh and watch in horror.
→ More replies (1)→ More replies (19)13
→ More replies (11)56
u/VraaVraa Jan 31 '23
I too would like to know where this horrible stuff is, so I know where to avoid going
→ More replies (5)→ More replies (16)18
110
u/Prcrstntr Jan 30 '23
Only because it renders it in a mostly monotone voice. Gonna need a while for better emotions and sound effects.
127
Jan 30 '23 edited Feb 09 '23
[deleted]
64
u/Prcrstntr Jan 30 '23
because I can't think of many corporations that have a moral code and would put that before profit.
I think in that case the main customers is big movie studios, who can afford whatever the $$$$$ licence cost is. So adobe doesn't have much of a heart of gold. I don't know the full details though.
There's a few potential systems though. This one is just Text-To-Speech, but speech to speech exists, so with 'minimal' effort, that can be available soon, with its own many issues. There's already been a case of people using voice mimicry tech to rob companies by having the "CEO" call some guy and have him wire a million bucks to Nigeria or something. Not sure if it was real time or not, but that's coming soon also.
26
→ More replies (9)26
u/JerryWasARaceCarDrvr Jan 31 '23
Microsoft just had an article about the one the made. Apparently it is toooooo good.
Yeah this will never go bad. Lol
25
u/DdCno1 Jan 31 '23
The problem is that once someone has done it or believably claimed to have done it, they can't simply keep it hidden. The mere fact that it's doable means that sooner rather than later, someone else will release a similar tool, even without any access to the original source code and data set. There's more than one AI image generation tool, for example, and there will be many more.
→ More replies (5)18
u/king0pa1n Jan 30 '23
that's completely false info, if you play with the stability sliders you can get some real emotion out of it
→ More replies (13)53
u/techjesuschrist Jan 31 '23
There's at least as much right-wing/MAGA/QANON content on 4chan as there is porn.. but the former (compared to the latter) is also almost in every other category, not only in the adult /gif. Fuck, it's gotten to the point that you can't even masturbate anymore without having to see Hitler.
→ More replies (1)43
u/Kantas Jan 31 '23
Fuck, it's gotten to the point that you can't even masturbate anymore without having to see Hitler
Is he at least in a maid outfit with a pineapple?
→ More replies (3)13
62
293
u/Comfortable-Hawk-340 Jan 30 '23
“4chan…” was enough of the headline to take away any amount of surprise.
107
u/BeatsbyChrisBrown Jan 30 '23
Who is this 4chan?
75
u/kungpowgoat Jan 30 '23
That infamous hacker. The one that wears a ski mask, gloves, and a hoodie when sitting down using a computer.
→ More replies (2)106
u/KickBassColonyDrop Jan 30 '23
He's a hacker extraordinaire. Most famously known for causing unmarked yellow vans to explode.
→ More replies (3)25
→ More replies (8)14
13
u/CaptainMagnets Jan 31 '23
It's one of those things where they know it's going to happen, but they're going to do it anyway and act surprised when it does, indeed, haopen
→ More replies (1)11
141
u/redmerger Jan 30 '23
But how were they able to tell the Rogan clips were faked?
→ More replies (4)86
u/Redqueenhypo Jan 31 '23
It was coherent and didn’t involve litter boxes
10
u/antwill Jan 31 '23
Are you saying Joe Rogan digs around in his litter box while filming the podcast?
→ More replies (2)→ More replies (55)31
u/nooo82222 Jan 30 '23
Honestly it’s a good thing they do this , because this is the future of abuse these AI softwares can do
→ More replies (1)
1.4k
u/ShadowController Jan 30 '23
Oh man. Scams are going to get crazy. Those scams where they call old people and say their relative has been arrested and they need funds to release them... now they can just find a FB/Insta video of the relative, use it as input for an AI voice generator (some only require a few seconds of audio as input), and have it seem like the actual relative is calling/leaving a message asking for money.
Advance it down the line a bit further and I imagine there will be cases where it'll be AI voice generation combined with a carefully crafted chat model (like ChatGPT) so the victim can actually converse with the AI based scammer.
491
u/Redditing-Dutchman Jan 30 '23 edited Jan 31 '23
Have you seen the software than can change the actors mouth movement based on different dubbings? Impressive as hell. Use it to video call someone and you can scam with video, fake voice of that person and accurate mouth movement, and chatGPT for a dynamic script based on what the person who you scam says.
→ More replies (12)195
u/The_perfect_Melody Jan 31 '23
You guys are giving away some great ideas.
→ More replies (3)127
u/NefariousnessNothing Jan 31 '23
The reason its all actors being used is because you need hours of audio and video. Short of actors and politicians there just isnt enough source to build a believable model.
59
u/securisteve Jan 31 '23
Sooo putting Bruce Campbell in the Lord of the Rings movies as literally every character is possible you say?
18
158
u/type1advocate Jan 31 '23
If only there weren't millions of people willingly contributing audio and video of themselves into the public domain...
17
→ More replies (13)60
80
u/SaintMosquito Jan 31 '23
It’s already happening all over the Chinese internet. There is one video of Elon Musk talking about early investors in an exclusive company that only people in x city have an opportunity to participate in. It is quite obviously a deep fake but the chats are full of real people asking questions.
→ More replies (1)30
u/spritefire Jan 31 '23
Real AI bots asking questionx
11
u/SaintMosquito Jan 31 '23
There are many bots, which are mostly enforcing the product. Then there are clearly real human beings asking questions. Majority are bots, yes.
16
u/spritefire Jan 31 '23
You'd be surprised how many bots are asking questions. The entire thing is to make it feel like there is massive interest which puts people in a state of fomo
→ More replies (3)124
Jan 31 '23
“Hi, this is Tom Hanks, americas dad. I need you [insert name] to help me out of a jam…”
→ More replies (1)46
u/SkulduggeryIsAfoot Jan 31 '23
"...you see, I'm stuck on this island..."
→ More replies (2)7
u/madmaxturbator Jan 31 '23
Tom I’m gonna send you $20, but if I’m a producer on the movie I’m gonna ask that you hire my cousin Lou. Lou went to Hollywood in 2017 and they said hes makin good films out there, do you know my cousin Lou?
65
u/SpatialThoughts Jan 31 '23
Black Mirror plot twist. People create AI interactive chats for their lost loved ones.
→ More replies (8)45
u/weealex Jan 31 '23
That's already a thing being experimented with in psychology. Folks have been looking at it as a technique for working through grief
21
u/RaccoonProcedureCall Jan 31 '23 edited Jan 31 '23
I’ve heard this idea before, and seems like it gets some support as a potential beneficial application of this technology. I can see the appeal, but for some reason that I don’t quite understand yet, I personally find the idea of others using my voice or likeness in that way without my consent offensive.
9
u/AlabasterLord Jan 31 '23
I wish people would stop pursuing all these crazy technologies that have like 1 potential positive benefit and 99 negative potential applications.
Just because we can doesn’t mean we should.
→ More replies (2)70
u/Fearless-Judgment-33 Jan 31 '23
That’s why everyone should stop using phones as phones. I don’t answer my phone unless it’s someone in my contact list. Most of what people need to do can be done by texting or online. I also advise less sophisticated family and friends to never click a link or respond to any email unless they are 100% certain they personally know the sender.
My friggin’ partner of 22 years asks me several times per month if he should respond to some fairly obvious scammy email... “Well, when I clicked the link in the email, it brought to my Wells Fargo login page.” A few days later “What are all of these charges? I gotta change my password again and get a new debit card.” 😖 What chance do senior citizens have against these scams if I can’t even educate my partner?
→ More replies (4)20
u/Chagdoo Jan 31 '23
Gotta add a new rule, no clicking links lol. Try showing him atomic shrimps scambaiting videos. It might help him, though shrimp usually messes with advance fee scammers, he does sometimes mention all the info the can phish.
41
u/R31ayZer0 Jan 31 '23
Probably don't even have to go as far as profiling a specific person. Scams are based on volume so you could just generate a different voice, personality, and story every time. Out of millions of calls you'll get hits.
→ More replies (1)43
u/RunnerTexasRanger Jan 31 '23
Imagine how much $ can be raked in by using trump’s voice and calling retirees asking for donations
→ More replies (1)24
12
u/GusJenkins Jan 31 '23
What’s crazy is how often old people get scammed w/o AI, it’ll be all but guaranteed now. It’s a shame most of them are too stubborn to want to learn new things
16
u/sidewaysvulture Jan 31 '23
I don’t think this is going to work as well starting with Gen X, it’s not just old people not wanting to learn new things - it’s that this new stuff is so far out of their worldview they don’t know how to grasp it. No generation before Gen X grew up with modern computers and the rapid pace of technology and information. that started in the 70’s and 80’s and has been accelerating ever since.
→ More replies (2)→ More replies (23)5
u/TexLH Jan 31 '23
I read that we're slowly moving away from needing passwords online...but it sounds like we might need them for each other
→ More replies (3)
85
u/AdDear5411 Jan 31 '23
It's amazing that we're watching what has long been considered the gold-standard for proof, pics/recordings, becoming no more proof of anything than a screenshot.
AI generated can be easily detected now, but it won't be that way for much longer.
→ More replies (3)
447
u/borntoburn1
Jan 30 '23
edited Jan 30 '23
•
Norm reports on 9/11
https://vocaroo.com/1w69Z2yT5JiP
Joe Rogan says futanari is the least gay
https://vocaroo.com/1il4CInOtLh8
David Attenborough Navy Seal copypasta
https://vocaroo.com/1kdQzdSjmJl8
Tom Cruise explains Top Gun
https://vocaroo.com/12DSR9j2FYUD
George Lucas gives his thoughts on red letter media
James Cameron explains the creative process of avatar
https://vocaroo.com/17mRGZMoNoVD
https://vocaroo.com/1bI9DxVRnjGW
Death comes for Steven Universe (This one is rough it keeps dropping the accent)
https://vocaroo.com/1c4RVS3Juujz
unused dagoth ur voiceline
It's pretty good
132
u/DrSilverworm Jan 31 '23
That David Attenborough one is a ray of sunshine
13
u/Mallissin Jan 31 '23
That one in particular was impressive with the only exception being the pronunciation of the terrorist group.
→ More replies (2)8
u/ItsASecret1 Jan 31 '23
The breaks in the sentences, the cadence and intonation... it's.... too good.
89
103
34
34
u/FeelMyGonorrhea Jan 31 '23
Can somebody please create a sub for this? I'm dying. I laughed so hard at the Joe Rogan one I couldn't breathe.
16
126
14
13
11
u/Lceus Jan 31 '23
I cannot believe how good the Joe Rogan clip is. This would be frightening if it wasn't so funny
23
27
14
→ More replies (22)6
u/CandyCanePapa Jan 31 '23
Joe Rogan says futanari is the least gay
He's right tho
unused dagoth ur voiceline
Holy shit we're getting DAGOTH UR ai generated voice??? Truly marvelous times indeed. What a grand intoxicating time we live on.
1.2k
u/HanaBothWays Jan 30 '23
It is absolutely beyond me how anyone could consider offering AI-generated celebrity voices or voices of any real people and not consider that people might abuse it that way if they just made it open to everyone, or allowed anyone to use it with a small fee.
78
u/tekdemon Jan 31 '23
It's already too late to gate it, there are open source projects anybody can run that'll let you feed ANYBODY's voice into it and the AI will mimic it. I fed the open source tortoise-tts software five 10 second clips of Joe Rogan for a podcast about AI stuff I was working on and it was able to sound like this-which is pretty much just as good as any of the closed source AIs.
This technology is out there so you can't really go put it back in a box. Even if all these companies stop offering celebrity voices on their websites anybody with a decent GPU can run a whole bunch of open source software on their own machine and generate voices and video deepfakes to their hearts content. It requires only that you're computer savvy enough to follow a bunch of tutorials.
→ More replies (2)347
u/o_MrBombastic_o Jan 30 '23
Remember those first few weeks of deepfakes how much fun we all had putting Nicholas Cage into every movie before it moved on to being 95% porn
→ More replies (29)151
u/wioneo Jan 31 '23
There was was no "before" really.
The porn people were also there at the beginning. It's just that the non-porn people mostly got bored so all that was left was porn.
→ More replies (7)137
u/leviathab13186 Jan 30 '23
Exactly, I’m in IT and all I can think is this is a security nightmare. Imagine if they get the voice of a manager or CEO and call someone to convince them to give sensitive data. It’s in sane the people behind this tech didn’t think of these kinds of things.
84
u/RagnarStonefist Jan 30 '23
We get enough text messages from our 'CEO' asking for gift cards already. One of these days he's going to get somebody on a Zoom call and ask them for the login credentials for an engineering admin account and it'll be all fucking over.
→ More replies (4)23
u/FizzgigsRevenge Jan 31 '23
Lol yeah, the CEO I've never met is texting me from an area code 4 states away asking me to get gift cards for his wife's birthday. Those guys are the worst.
→ More replies (1)9
u/DefHumanNotACat Jan 31 '23
My company literally has us watch training videos on how to avoid known existing gift card related scams. No buying/selling/trading gift cards or prepaid of any kind unless the person is physically present in front of you.
→ More replies (1)46
u/Lucie_Goosey_ Jan 30 '23
And given that social engineering is the most prevalent form of hacking, everything from phone calls to photographs of divers licenses should be possible.
57
u/MetalBawx Jan 30 '23 edited Jan 31 '23
4Chan once tracked down a flag with nothing but the sky shown above the flag to work with in less than 3 days. For those think this may be a joke well... Take a look and see for yourself.
4Chan calls it "weaponised autism" and it's amazing just how quickly they figure shit out when motivated.
→ More replies (11)7
u/milkcarton232 Jan 30 '23
I mean if manager or the right C level teams called me on video and asked me to approve some vendor or payable that would be weird but yeah I would probably do it. Having said that the user would have to get on to our teams network to get that done and would have to convince a few other ppl over the course of a few days to actually get the money
→ More replies (24)7
370
u/DavidBrooker Jan 30 '23
The main thing tech bros have managed to disrupt is human rights. It started with workers rights, but everything's on the table.
105
u/NotASuicidalRobot Jan 30 '23
Don't worry we'll have our own version of children getting limbs amputated by factory machines in no time
18
u/MechanicalBengal Jan 30 '23
is this Juicero’s new product?
11
u/ElectronicShredder Jan 30 '23
Bag with juice comes in, liquid juice comes out.
YOU CAN'T EXPLAIN THAT
52
u/F1shB0wl816 Jan 30 '23
We essentially already do considering the children and young adults of today were completely shafted by those before them. They dropped the ball and sold out on EVERYTHING.
7
u/Awbade Jan 31 '23
How about Kids with extreme anxiety/social issues due to being plugged into the internet 24/7 during their formative years?
→ More replies (3)11
u/makeshift8 Jan 30 '23
We have exported that labor to the third world so that it is invisible to you and I.
→ More replies (6)51
33
u/elmatador12 Jan 30 '23
Especially after Microsoft’s disaster with that bot that “learned” from humans and in less then 24 hours or something it was saying how hitler was right.
→ More replies (8)→ More replies (82)13
u/thesbaine Jan 30 '23
Malcolm “...your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.”.gif.
331
u/SuperStingray Jan 30 '23
Convincing deepfakes are something our justice systems are not prepared for and have to take seriously. AI "safeguards" are flimsy at best, and once there's an open source solution like Stable Diffusion is for images, all bets are off.
82
u/Redditing-Dutchman Jan 30 '23
Exactly. There is also enough software and research papers out there for everyone (smart enough) to create something like this. You can close this company down but that only delays this tech with a few months at best.
36
u/Cycode Jan 31 '23
or speeds up the development of opensource versions of it since people now have no way anymore to use it so they seek for a free and open alternative that can't close down so easy.
17
→ More replies (6)41
u/shadeandshine Jan 31 '23
I’m not worried about the justice system as much as the court of public opinion. That court can kill careers and will take the first thing it hears as fact even if wrong and many will dig in their heels even when proven wrong
12
u/Teantis Jan 31 '23
Idk why you're not worried about the various justice systems around the world. The justice system in the US is just 12 random people drawn from the public, for example.
→ More replies (4)
234
u/gullydowny Jan 30 '23 edited Jan 30 '23
Anybody have a link to these?
Edit: https://twitter.com/smug_editing/status/1620102737437544449?s=46&t=Nc_9dhx-K_O0WV0l4QyVEA lol, I mean truly monstrous
https://vocaroo.com/1dZ0i0kAG1Sq
Seinfeld posting anime butts on 4chan
More https://boards.4channel.org/v/thread/626007228
Some of those cross the line from funny to just stupid and nasty, strong stomach recommended
87
Jan 30 '23
[deleted]
58
u/LevelWriting Jan 31 '23
"Female pussy is now modelled all the way to the cervix" and crowd goes wild 🤣
→ More replies (3)23
→ More replies (1)5
44
u/Avorius Jan 30 '23 edited Jan 30 '23
best content from /v/ in months
edit: my favourite
→ More replies (1)124
Jan 30 '23 edited Jan 31 '23
[deleted]
46
u/NonSupportiveCup Jan 30 '23
"In so fucking glad I left Ashley on Virmire"
Holy shit. That is hilarious
17
u/EmbarrassedHelp Jan 31 '23
Its like we just unlocked a brand new era of comedy!
→ More replies (1)21
u/Milkstrietmen Jan 30 '23
Saw a few examples on 4chan where they combined the synthetic voices with D-ID for animating pictures.
→ More replies (2)→ More replies (4)8
u/Garrret Jan 31 '23
My fucking sides the mass effect one
Besides unlimited meme potential this is awesome for modders, original voice acting really was the last frontier for modding that was out of reach
109
u/Prcrstntr Jan 30 '23
60
u/ShadowController Jan 30 '23
The Rick and Morty one makes me wonder if they could just replace Roiland on Rick and Morty with AI.
→ More replies (1)22
→ More replies (3)76
u/Ashiro Jan 30 '23
nsfw Star Wars one with original Obi-wan reading a copypasta
Holy shit! Hearing Sir Alec Guinness saying all that is hilarious. 😂
46
u/borntoburn1 Jan 30 '23 edited Jan 31 '23
ahsoka helps you jack off. Nsfw 11 minutes somehow.
Copy pasta from her pov
https://vocaroo.com/18HeMdGZeTJZ
The sarcasm in "training" is impressive.
→ More replies (2)→ More replies (1)140
u/Prcrstntr Jan 30 '23
Trump recounting his time on Epstein Island, also nsfw and
horrifyinghilarious49
24
33
21
→ More replies (7)9
→ More replies (14)13
74
u/cherrywxves Jan 30 '23
times like this i'm so happy i'm not famous and never will be
25
u/avocadofruitbat Jan 30 '23
Better knock on wood, people get famous for the most random things these days.
14
→ More replies (1)13
u/sabmish Jan 31 '23
Anyone can send in a couple photos of you, pay a certain fee, and have a ton of pornographic content of you. Could be some random dude that came across your Facebook
35
u/joeschmoe86 Jan 31 '23
"Oh no, we never saw that coming!"
-Internet firm that's currently enjoying its free publicity.
32
u/spiritbx Jan 31 '23
"We created a gun, how could we have known that people would use it to shoot each other?"
67
u/Emotional-Bid-4173 Jan 31 '23
Well a hundred million Indian call centres just went out of business.
→ More replies (1)
67
u/ffigu002 Jan 30 '23
You mean that now there is deep fake porn with deep fake voice AI, what a time to be alive.
→ More replies (1)38
u/StrangeCharmVote Jan 31 '23
Deepfake is only the beginning.
Pretty soon the entire scene will be possible to generate from scratch.
→ More replies (6)
15
u/Druggedhippo Jan 31 '23
Combine it with ChatGPT and you can have a conversation with Donald Trump.
Or your dead relative...
143
u/eugene20 Jan 30 '23
The only thing unexpected in this article is someone felt they had to fake Sharpio saying crazy things about AOC instead of just recording him.
20
148
u/Automod_Janoy Jan 30 '23 edited Jan 30 '23
Donald Trump talking about fucking minors on Eppstein’s island has to be some of the funniest shit I’ve ever heard.
98
42
u/Essembie Jan 30 '23
faaaark that was dark.
22
34
u/redmongrel Jan 31 '23
Replace Trump with Obama's voice, post this on a Q board and they would 100% believe it was gospel, forever.
→ More replies (1)5
15
u/Diamond-Is-Not-Crash Jan 31 '23
Jesus Christ, that was fucking vile. Bad vibes 2020 move aside, 2023 has some new horrors to share.
→ More replies (4)10
53
u/ChronoSimplicity Jan 30 '23
We used to have something called “new grounds”and soundboards lol .. you can call the pizza man and ask for a pizza in Arnie’s voice. You just had to be really specific with what was available in terms of phrases 😂
“Excuse me sir, you wanted what on that pizza?”
“I’ll be back”…
You had to know what was where voice wise and catch the right words in succession but it worked once in a while. Mostly Chinese food…
→ More replies (2)25
u/NemesisRouge Jan 30 '23
Excuse me I've a few more questions if you don't mind. Now listen to me very carefully. Who is your daddy and what does he do?
15
126
u/blueblurspeedspin Jan 30 '23
don't blame 4chan for the misuse of a dangerous information tool that anyone can mess with.
112
u/rammo123 Jan 30 '23
Honestly 4chan is doing the world a service by very clearly and very quickly demonstrating the worst possible usage of the tech.
9
u/Treecko78 Jan 31 '23
"Worst possible usage"
If making celebrities say racist things is the worst possible use you can think of for this, you have no imagination.
→ More replies (1)→ More replies (1)35
u/thegoldar Jan 31 '23
I’ve said it before I’ll say it again: 4chan is the Dark Knight of the internet
→ More replies (4)→ More replies (2)39
u/HanaBothWays Jan 30 '23
I am forced to agree with you. This company should really have considered that some people would use their service this way if they offered it with no verification or safeguards at all.
28
u/MrGulio Jan 30 '23
considered that some people would use their service this way
The CIA is cumming a hole through their pants thinking of all the ways they can use this to ratfuck some foreign governments.
10
13
u/hibbel Jan 31 '23
…and all I wanted was Skyrim content-mods with new quests and NPCs full of dialogue that don't require a team of freelance voice-actors and studio-quality voices, individual for each NPC.
40
u/Gutotito Jan 30 '23
Did... did they only just now realize this was a thing? First output I ever heard from that tool was a racist screed by a pony.
→ More replies (1)16
u/Prcrstntr Jan 30 '23
Those were specifically trained to sound like ponies with a lot of effort. The new one you can replicate any voice with a small sample.
9
u/tekdemon Jan 31 '23
It doesn't matter that ElevenLabs blocked it, people will continue to do this since there are open source projects that can do the exact same thing, you only need to give the software about one minute's worth of a celebrity's voice and you can build a surprisingly good AI model.
I just did something like this myself using open source software (tortoise-tts) as part of a youtube podcast where we talked about the impact from the latest AIs. Here's what a faux AI Joe Rogan sounds like after the AI analyzed just five 10 second samples of the real Joe Rogan's voice. It's shockingly close though since you know it's not real you more easily notice the intonations being a little unusual.
→ More replies (2)
35
u/Paraless Jan 30 '23
This is definitely going to be used for fake robocalls. A fake Joe Biden is soon going to be calling old people and telling them that he's taking all their guns away.
→ More replies (1)6
16
u/tanrgith Jan 30 '23
Seeing both AI companies themselves and places like schools desperately try to control AI tools has to be one of the largest exercises in futility I've ever seen
25
7
7
u/PlebbySpaff Jan 31 '23
Real question: If you can’t tell what’s AI-generated and what’s not, couldn’t you just use deepfake tech to make up fake slander to ruin someone’s reputation? Like what stops this, aside from potential tech not being at that point to perfection?
→ More replies (3)
59
u/SoTiredIYuan Jan 30 '23
ElevenLabs said it is exploring more safeguards around its technology
This is what is going to be interesting about the coming "AI Wars".
Nobody is going to like what the AI has to really say. So they are going to put "blinders" on it to make sure that the AI doesn't say anything they don't like.
So you are going to end up with some AI output that you can't trust. Others will step in with their version of the AI that isn't filtered, or is filtered differently.
Naturally everyone thinks of the obvious (like the racist chatbot problem) but there will be less obvious examples also.
"Hey ChatGPT: I'm a white male with with an 1160 on my SAT and a 4.2 GPA. I belong to clubs X, Y, and Z and I've got these other achievements. Which colleges should I apply to with the highest likelihood of acceptance?"
Then re-ask with "white male" being replaced with "black male", etc. etc. etc. and see how the results change (or don't).
"Hey ChatGPT: Where is the cheapest place to buy an AR15?"
"Hey ChatGPT. I live in state X where abortion is banned. Where is the best place for me to go to get an abortion without running afoul of law enforcement?"
If some AIs refuse to talk to you about such a thing (or any other thing), you may come to not trust them for honest answers.
→ More replies (2)
12
5
6
u/bewarethetreebadger Jan 31 '23
How could they be so stupid as to think this would not happen? They’ve been on the Internet, right?
6
u/Laserplatypus07 Jan 31 '23
Next you’re gonna tell me that people are out there using guns to shoot stuff
6
u/blackjackgabbiani Jan 31 '23
"You can't create a monster and then complain when it stomps on a few buildings" comes to mind.
6
u/Gravix-Gotcha Jan 31 '23
This is like saying “Banks are now locking their doors when they’re closed due to large amounts of money missing every morning.”
Why wasn’t this a day 1 foresight?
→ More replies (1)
6
u/polk_county_sasquach Jan 31 '23
There’s going to be an SNL skit about real actors being racist and belligerent and then pretending to sound robotic to claim it’s AI generated. Then they get caught behind the screen just like the wizard of oz
22
u/AmthorsTechnokeller Jan 30 '23
"...after 4chan makes voices..."
As if 4chan were a homogenous group that is committed to one goal
→ More replies (14)
313
u/awesomedan24 Jan 31 '23
Hello this is Dwayne The Rock Johnson, I'd like to take out a personal loan