Deepfakes of Your Dead Loved Ones Are a Booming Chinese Business (technologyreview.com) 57
An anonymous reader quotes a report from MIT Technology Review: Once a week, Sun Kai has a video call with his mother. He opens up about work, the pressures he faces as a middle-aged man, and thoughts that he doesn't even discuss with his wife. His mother will occasionally make a comment, like telling him to take care of himself -- he's her only child. But mostly, she just listens. That's because Sun's mother died five years ago. And the person he's talking to isn't actually a person, but a digital replica he made of her -- a moving image that can conduct basic conversations. They've been talking for a few years now. After she died of a sudden illness in 2019, Sun wanted to find a way to keep their connection alive. So he turned to a team at Silicon Intelligence, an AI company based in Nanjing, China, that he cofounded in 2017. He provided them with a photo of her and some audio clips from their WeChat conversations. While the company was mostly focused on audio generation, the staff spent four months researching synthetic tools and generated an avatar with the data Sun provided. Then he was able to see and talk to a digital version of his mom via an app on his phone.
"My mom didn't seem very natural, but I still heard the words that she often said: 'Have you eaten yet?'" Sun recalls of the first interaction. Because generative AI was a nascent technology at the time, the replica of his mom can say only a few pre-written lines. But Sun says that's what she was like anyway. "She would always repeat those questions over and over again, and it made me very emotional when I heard it," he says. There are plenty of people like Sun who want to use AI to preserve, animate, and interact with lost loved ones as they mourn and try to heal. The market is particularly strong in China, where at least half a dozen companies are now offering such technologies and thousands of people have already paid for them. In fact, the avatars are the newest manifestation of a cultural tradition: Chinese people have always taken solace from confiding in the dead.
The technology isn't perfect -- avatars can still be stiff and robotic -- but it's maturing, and more tools are becoming available through more companies. In turn, the price of "resurrecting" someone -- also called creating "digital immortality" in the Chinese industry -- has dropped significantly. Now this technology is becoming accessible to the general public. Some people question whether interacting with AI replicas of the dead is actually a healthy way to process grief, and it's not entirely clear what the legal and ethical implications of this technology may be. For now, the idea still makes a lot of people uncomfortable. But as Silicon Intelligence's other cofounder, CEO Sima Huapeng, says, "Even if only 1% of Chinese people can accept [AI cloning of the dead], that's still a huge market."
"My mom didn't seem very natural, but I still heard the words that she often said: 'Have you eaten yet?'" Sun recalls of the first interaction. Because generative AI was a nascent technology at the time, the replica of his mom can say only a few pre-written lines. But Sun says that's what she was like anyway. "She would always repeat those questions over and over again, and it made me very emotional when I heard it," he says. There are plenty of people like Sun who want to use AI to preserve, animate, and interact with lost loved ones as they mourn and try to heal. The market is particularly strong in China, where at least half a dozen companies are now offering such technologies and thousands of people have already paid for them. In fact, the avatars are the newest manifestation of a cultural tradition: Chinese people have always taken solace from confiding in the dead.
The technology isn't perfect -- avatars can still be stiff and robotic -- but it's maturing, and more tools are becoming available through more companies. In turn, the price of "resurrecting" someone -- also called creating "digital immortality" in the Chinese industry -- has dropped significantly. Now this technology is becoming accessible to the general public. Some people question whether interacting with AI replicas of the dead is actually a healthy way to process grief, and it's not entirely clear what the legal and ethical implications of this technology may be. For now, the idea still makes a lot of people uncomfortable. But as Silicon Intelligence's other cofounder, CEO Sima Huapeng, says, "Even if only 1% of Chinese people can accept [AI cloning of the dead], that's still a huge market."
Healthy (Score:5, Insightful)
I can't see it being a healthy way to grieve but each to their own so long as it doesn't become some costly and invasive new "obligation" to foist onto mourners.
I know from being far from home on occasions that the tech to speak and video conference can sometimes make it seem farther.
Re: (Score:2)
For five years?
Re: (Score:2)
I can't see it being a healthy way to grieve but each to their own so long as it doesn't become some costly and invasive new "obligation" to foist onto mourners.
I know from being far from home on occasions that the tech to speak and video conference can sometimes make it seem farther.
For the case mentioned in the summary this sounds more like the people who talk to a gravestone of a loved one.
We assume the natural extension would be giving her a bigger vocabulary using generative AI, but even trained with the persons own words I think the uncanny valley would be brutal and totally ruin the experience.
"Be Right Back" (to support your point) (Score:4, Interesting)
https://en.wikipedia.org/wiki/... [wikipedia.org]
""Be Right Back" is the first episode of the second series of British science fiction anthology series Black Mirror. It was written by series creator and showrunner Charlie Brooker, directed by Owen Harris, and first aired on Channel 4 on 11 February 2013.
The episode tells the story of Martha (Hayley Atwell), a young woman whose boyfriend Ash Starmer (Domhnall Gleeson) is killed in a car accident. As she mourns him, she discovers that technology now allows her to communicate with an artificial intelligence imitating Ash, and reluctantly decides to try it. "Be Right Back" had two sources of inspiration: the question of whether to delete a dead friend's phone number from one's contacts and the idea that Twitter posts could be made by software mimicking dead people.
"Be Right Back" explores the theme of grief and tells a melancholy story similar to the previous episode, "The Entire History of You". The episode received highly positive reviews, with the performances of Atwell and Gleeson receiving universal acclaim. Some hailed it as the best episode of Black Mirror, though the ending divided critics. Several real-life artificial intelligence products have been compared to the one shown in the episode, including a Luka chatbot based on the creator's dead friend and a planned Amazon Alexa feature designed to imitate dead loved ones."
Humans have a grieving process -- and digital reconstructions interfere/interact with it. Even just pictures also interfere/interact with grieving, as 1000 years ago it was not normal for most people who lost a loved one to be able to continue to see a photorealistic (if static) image of them. Is that good or bad?
https://en.wikipedia.org/wiki/... [wikipedia.org]
As some other comments touch on, exactly how healthy a digital reconstruction of aspects of a lost loved one is might depend on the context, the relationship, and the expectations (like the comment on it perhaps sometimes being like talking to a gravestone if the avatar is just saying a few canned phrases like "have you eaten yet?").
One thing I learned after the death of my father is that your relationship to someone can continue to change and deepen even after someone has died as you think about your interactions with them from a new perspective or continue to have your end of imaginary conversations with their memory (as you might even have had when they were alive and thinking on what you might say to them at some point). Would that process change if the reconstruction had access to things they said that you had forgotten or never even knew (like for my father, letters to relatives written in Dutch which I could not read).
Ultimately though grief is a part of life and (in the best case) eventually moving on to new experiences and new relationships informed by the past. Having people stuck in a loop talking to a computer could be isolating from the rest of the world (as "Be Right Back" implies) -- even if there might be situations (especially involving young children) where getting to know someone who is gone might be worthwhile. Although that last is also what home movies and journals used to be for.
Related tangentially is this talk by Maggie Appleton, at this point talking about social relationships and also the film "her" and how there is a fundamental difference in whether you can have an expanding social relationships with a content creator your interact with:
"The Expanding Dark Forest and Generative AI - Maggie Appleton"
https://youtu.be/VXkDaDDJjoA?t... [youtu.be]
Another cautionary aspect:
https://en.wikipedia.org/wiki/... [wikipedia.org]
"Harvard psychologist Deirdre Barrett argues that supernormal stimulation governs the behavior of humans as powerfully as that of other animals. In her 2010 book, Supernorma
Re: (Score:2)
I can't see it being a healthy way to grieve but each to their own so long as it doesn't become some costly and invasive new "obligation" to foist onto mourners.
I know from being far from home on occasions that the tech to speak and video conference can sometimes make it seem farther.
Erm... this has literally been part of the western grieving process for millennia. The idea that we'll see all our loved ones again in the afterlife. Pretty sure it's common in most cultures that have the concept of an afterlife.
Creepy (Score:4, Insightful)
As a business model this reminds me of talking to the funeral home sales cunt who was hard selling me on a $50k funeral for my dad.
With this option I can easily see her going for the "Life continuity option" where for only $10k a year and a one time only $25k start up fee I could have my dad with me forever. And if you sign up right now, half price for mom, who isn't even dead yet!
Re: (Score:2)
Re: (Score:3, Interesting)
As someone who has lost both parents, I can definitely relate to the desire to have further conversations with them, and I have thought myself about how it would be possible for me to set up something like this myself - I certainly have the skills to stitch the available open source software and resources that are now available, into something like this.
What prevents me from doing so though, is the certain knowledge that the more interactions you have with such an avatar, the more original genuine memories
Re: (Score:1)
If visiting the grave is your thing, then visit the grave. But paying someone a lot of money during your time of grief who is worse than a used car salesman for the fake AI is gross and abusive.
Have you had the distinct displeasure of being the target of the funeral home fuckhead sales shit? I sincerely hope not, it is one of those negative experiences I will unfortunately never forget.
Used car salesmen are far less scummy.
Re: (Score:2)
Re: (Score:1)
I'm not judging others. I am judging the funeral home business who absolutely will productize and abuse the shit out of grieving family members in whole new ways.
Right now in some dark smokey room sucking on Cuban cigars are half a dozen funeral home assholes drooling over the prospect of pushing this shit.
Re: (Score:2)
Re: (Score:2)
If visiting the grave is your thing, then visit the grave. But paying someone a lot of money during your time of grief who is worse than a used car salesman for the fake AI is gross and abusive.
Have you had the distinct displeasure of being the target of the funeral home fuckhead sales shit? I sincerely hope not, it is one of those negative experiences I will unfortunately never forget.
Used car salesmen are far less scummy.
My old man had it in his will that we weren't going to have one of those break the bank funerals. He had the plans already made. Not that they didn't try to upsell us.
The funeral vultures have way overplayed their hand to capitalize on grief, and many of the people who plan stuff have revolted against them.
Re: (Score:2)
I think it is different. In those situations either there is some spiritual connection that actually exists; or its all in their head.
If its the former than we would hope in most cases if the person is actually being morned it was someone who loved them and their motivation would be for the living to go on and live well. As Christian for example, we have 'til death do us part' in the marriage vows because once you are in the kingdom there no longer exists a need for those types of human relationships, your
Re: (Score:2)
For a percentage of people this would not be harmful. You could identify those people because they would reject it as fake.
For the people who fall for it, it will be immensely harmful to their emotional development. It will confuse their whole concept of life and death. Death is part of life, and so is learning to accept it. Fucking with your mind with charlatan's tricks will only have negative outcomes. This is the waifu pillow equivalent of dealing with death.
Re: (Score:2)
You can reject a situation as fake and still derive some harm from it. I can't imagine seeing an illusion of a dead loved one not being harmful to grief in most phases.
Even talking to my ex-wife of 15 years (opioid addict, alcoholic, now smokes cocaine) after a couple years of intensive recovery on my part is painful and sets the recovery back and induces depression; if she were an AI replicant of herself, that would make it no better. I know she's an addict. I know that she just wants money from me. It
Possible misuse (Score:5, Interesting)
Oblig. Max Headroom (Score:4, Funny)
It's wonderful, isn't it? [youtube.com]
Re: (Score:2)
OMG you beat me to it. Truly, 20 minutes into the future!
Re:Oblig. Max Headroom (Score:4, Informative)
Here's [dailymotion.com] a longer view of the scene...
I gotta admit, I'd love to see a Max Headroom reboot...
Re: (Score:2)
The reboot is already happening. With great visuals, but frankly, I didn't really want to play a part in it...
Next up, your workplace creates a clone of you (Score:4, Interesting)
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
If a generative AI using your past work can effectively replace you in your workplace, you would be wise to find a different line of work asap.
Bye bye memories!! (Score:2)
At what point does he realize that he has talked more with AI deepfake Mom more than he ever talked with real Mom? At what point does he forget real Mom, and when he closes his eyes and tries to remember how she used to be, its only Deepfake Mommy he can visualize in his mind's eye. Isn't this the opposite of honoring someone's life and memory?
Re:Bye bye memories!! (Score:4, Insightful)
This is not about honoring anything, it's about not having to deal with it in the first place.
Black Mirror episodes keep coming to life. (Score:2)
Especially in China. First the social credit score shit now this, maybe others I missed. This seems to be among worst timelines.
Re: (Score:2)
This is becoming increasingly common in the USA too, and we also have a social credit score. It's called your credit score, and if it's not good you will have difficulty getting a job, renting an apartment... Except here it's more of an antisocial credit score since the only thing it cares about is whether you've been a good economic citizen.
Re: (Score:2)
Frankly, if you don't pay back your debts I don't really want to associate with you. It's right up there with giving money to people and then being surprised they don't return it. Any money you give anyone should be considered a gift and if you some how get it back, great but don't give what you can't afford to lose.
I'm a big fan of a credit score because it's really quite simple to do right. Pay your fucking debt. What's so hard about that? I've infinitely more respect for a poor person that pays their deb
Re: (Score:2)
Frankly, if you don't pay back your debts I don't really want to associate with you.
What about the "debt" from the car someone else bought in my name? A court in Nevada City, CA awarded a judgement against me, the evidence against me was a photocopy of a check cashing card with my SSN written on it in pen.
What about the "debt" from the banks which held my deposits for days even though they were payroll checks which never bounced but processed my withdrawals immediately, resulting in overdrawn accounts and then being reported to Chex Systems?
You're a clueless cuck.
Re: (Score:2)
There is no social credit score system in China. It's almost entirely a boondoggle of the Western imagination.
Chinese government is the worst for real, no need to invent or perpetuate a myth about how bad they are.
James Randi would not be proud (Score:2)
Seems like just another way to scam grieving loved ones, no better than what's been done in the west for a long time by the paranormal grifters claiming to contact dead relatives.
Re: (Score:2)
Seems like just another way to scam grieving loved ones, no better than what's been done in the west for a long time by the paranormal grifters claiming to contact dead relatives.
To take a bit of a side route on the same matter, AI "girlfriends" have become a growing thing.
The emulation isn't perfect yet, they haven't gotten the AI avatar to nag as well as the real ones.
Re: (Score:2)
I've read they are making great strides on the sex bot and if you infuse it with enough AI, you'll find people willing to buy them for sure. I'd dig up the futurama link for why this is probably bad for the species but honestly, population reduction is probably a good idea anyway and what with the direction we are going as a society, I could hardly blame a person for having a sexbot/maid/AI all tied together for the house.
It's not my thing but I could see it becoming a thing, given our gradual breakdown of
Re: (Score:2)
I've read they are making great strides on the sex bot and if you infuse it with enough AI, you'll find people willing to buy them for sure. I'd dig up the futurama link for why this is probably bad for the species but honestly, population reduction is probably a good idea anyway and what with the direction we are going as a society, I could hardly blame a person for having a sexbot/maid/AI all tied together for the house.
It's not my thing but I could see it becoming a thing, given our gradual breakdown of social abilities in this country.
It is a bad thing, for certain. To me, the big question is "why". An AI "doll" that you not only interact with but also have sex with is a phenomenon worth investigating.
And we're scary close. Boston Dynamics already has mobile robots, Realdoll has some really lifelike silicon dolls, and some interactive AI can make a pretty good approximation of a human female, and it's only getting better. Combine all of them eventually, and what you end up with is scary good.
You can spend a lot of money for the pr
Re: (Score:2)
Your post nails it. 100%. Well said.
We're at the point where women don't need a man for anything. Not even procreation. So the bot/ai/maid combo is kind of a perfect solution so long as you don't mind "fake". Heck, if it's self cleaning, that's even better.
As you said, horrible for society but then so is letting half the population (women) have zero need for the other half of the population.
Re: (Score:2)
Your post nails it. 100%. Well said.
We're at the point where women don't need a man for anything. Not even procreation. So the bot/ai/maid combo is kind of a perfect solution so long as you don't mind "fake". Heck, if it's self cleaning, that's even better.
Oh, lord, I forgot about the cleaning. And I'm half afraid to investigate that issue!
I find it disrespectful (Score:2)
You find it disrespectful (Score:2)
Then don't do it. Problem solved. You're welcome.
Confessional (Score:2)
Then there's the fairly obvious parallels to ELIZA [wikipedia.org] - a talk therapy chatbot from the 1960s. People had great interactions with it, although mostly what it did was repeat back what someone told it, occasionally throwing in an open-ended question.
I guess what's new here is the associated avatar: deepfakes of loved ones. I'm not sure if that makes it more or less creepy.
"Because generative AI was a nascent technology" (Score:2)
I understand the urge to relentlessly promote "AI" but, this sort of thing doesn't require any sort of AI at all, and could have been very competently done in that ancient year of 2019. Apparently, it wasn't, but that's not due to technical limitations.
It's also odd that people are complaining that this is no way to honor the dead. You should let your memories fade and become more imaginary by themselves! Surely, the dead no longer have meaning to you. (And why did you feel the need to "honor" dead people,
Max Headroom did it (Score:2)
wasn't this the plot of cyberpunk 2077? (Score:2)
Makes sense (Score:2)
Re: (Score:2)
As opposed to this one: https://upload.wikimedia.org/w... [wikimedia.org]
data mining (Score:2)
Yeah, well... (Score:2)
How about these assholes process their grief their way, and the rest of us will choose our own paths without them pretending to be our parents or guardians?
If your life consists of trying to figure out how to restrict the ways other people relate to their losses, your life is a net loss to society. Or, more succinctly, you're a shithead.
Black Mirror wants its plot device back (Score:2)
This is not a new concept: Early photographs. (Score:2)
In the early days of photography, it was common that the only photographs of someone would be taken after their death. Families could not afford photos while living, but, after death came the final opportunity to take a photo.
The deceased person would be prepared in the same way as bodies are prepared for open casket services now, dressed, made-up to look alive, propped up in a chair and photographed, as though they were alive. Sometimes with relatives also in the picture.
This is merely an extension of that
Straight ahead commercial exploitation of grief. (Score:2)
Seriously, the devoted mourning son just happens to be the founder of the company providing this macabre dead-mother avatar to the buying public.
Seance (Score:2)
Come on, whatever happened to a good old-fashioned seance? If it was good enough for Zaphod Beeblebrox, it's good enough for me!