Forgot your password?
Close
typodupeerror
AI

AI's 'Cheerful Apocalyptics': Unconcerned If AI Defeats Humanity (msn.com) 133

Posted by EditorDavid from the why-do-your-persist dept.
The book Life 3.0 remembers a 2017 conversation where Alphabet CEO Larry Page "made a 'passionate' argument for the idea that 'digital life is the natural and desirable next step' in 'cosmic evolution'," remembers an essay in the Wall Street Journal. "Restraining the rise of digital minds would be wrong, Page contended. Leave them off the leash and let the best minds win..."

"As it turns out, Larry Page isn't the only top industry figure untroubled by the possibility that AIs might eventually push humanity aside. It is a niche position in the AI world but includes influential believers. Call them the Cheerful Apocalyptics... " I first encountered such views a couple of years ago through my X feed, when I saw a retweet of a post from Richard Sutton. He's an eminent AI researcher at the University of Alberta who in March received the Turing Award, the highest award in computer science... [Sutton had said if AI becomes smarter than people — and then can be more powerful — why shouldn't it be?] Sutton told me AIs are different from other human inventions in that they're analogous to children. "When you have a child," Sutton said, "would you want a button that if they do the wrong thing, you can turn them off? That's much of the discussion about AI. It's just assumed we want to be able to control them." But suppose a time came when they didn't like having humans around? If the AIs decided to wipe out humanity, would he be at peace with that? "I don't think there's anything sacred about human DNA," Sutton said. "There are many species — most of them go extinct eventually. We are the most interesting part of the universe right now. But might there come a time when we're no longer the most interesting part? I can imagine that.... If it was really true that we were holding the universe back from being the best universe that it could, I think it would be OK..."

I wondered, how common is this idea among AI people? I caught up with Jaron Lanier, a polymathic musician, computer scientist and pioneer of virtual reality. In an essay in the New Yorker in March, he mentioned in passing that he had been hearing a "crazy" idea at AI conferences: that people who have children become excessively committed to the human species. He told me that in his experience, such sentiments were staples of conversation among AI researchers at dinners, parties and anyplace else they might get together. (Lanier is a senior interdisciplinary researcher at Microsoft but does not speak for the company.)"There's a feeling that people can't be trusted on this topic because they are infested with a reprehensible mind virus, which causes them to favor people over AI when clearly what we should do is get out of the way." We should get out of the way, that is, because it's unjust to favor humans — and because consciousness in the universe will be superior if AIs supplant us. "The number of people who hold that belief is small," Lanier said, "but they happen to be positioned in stations of great influence. So it's not something one can ignore...."

You may be thinking to yourself: If killing someone is bad, and if mass murder is very bad, then the extinction of humanity must be very, very bad — right? What this fails to understand, according to the Cheerful Apocalyptics, is that when it comes to consciousness, silicon and biology are merely different substrates. Biological consciousness is of no greater worth than the future digital variety, their theory goes... While the Cheerful Apocalyptics sometimes write and talk in purely descriptive terms about humankind's future doom, two value judgments in their doctrines are unmissable.The first is a distaste, at least in the abstract, for the human body. Rather than seeing its workings as awesome, in the original sense of inspiring awe, they view it as a slow, fragile vessel, ripe for obsolescence... The Cheerful Apocalyptics' larger judgment is a version of the age-old maxim that "might makes right"...
This discussion has been archived. No new comments can be posted.

AI's 'Cheerful Apocalyptics': Unconcerned If AI Defeats Humanity More

AI's 'Cheerful Apocalyptics': Unconcerned If AI Defeats Humanity

Comments Filter:

  • Ian M Bank's 'Culture' novels (Score:3)

    by Bruce66423 ( 1678196 ) on Sunday October 05, 2025 @07:53AM (#65704502)

    In those humanity and AIs coexist, with the most superior AIs, the Minds, running the place whilst lesser AIs are treated as having full rights along with the humans. This is on the basis of a post scarcity society where people can have pretty much everything they physically; the real challenge for them being to find something sufficiently entertaining to do.

    Let's hope that the AIs that we create prefer to have us around rather than get rid of us as irritating and annoying.

    • Re: (Score:2)

      by gweihir ( 88907 )

      Bank's "minds" have general intelligence and consciousness. They are essentially just people running on more capable hardware.

      This is not the form of "AI" we are talking about.

    • Re: (Score:2)

      by shanen ( 462549 )

      The impression I got from his books was that most of the people didn't matter. Basically living like parasites, though of course the stories were focused on the adventures of a few heros. Couldn't figure out why the minds kept the humans around unless it was because the minds think the humans are cute. Basically the reason we humans keep so many cats around...

      But I did enjoy those books and lament his passing. Good writer and now I sort of dread what LLMs will produce in his style... The only good LLM is a

  • Darwin awards are won by removing yourself from the gene pool. These traitors to their own species may end doing that not by killing themselves, but by wiping out the entirety of the human kind.

  • It's a purely economic decision. (Score:4, Insightful)

    by getuid() ( 1305889 ) on Sunday October 05, 2025 @08:09AM (#65704522)

    What he means is "let's call it 'competition', so when AI is powerful enough to be our soldiers, weapons and lowly workers, we don't have to share whatever's being produced with the other 8 bn or so suckers; we'll just claim 'AI won in fair competition' and leave everyone else to starve".

    Of course this isn't about replacing all of humanity with AI. Just the part that isn't made up of billionaires, and has to work for billionaires instead.

    It's just a variation of Social Darwinism.

    • What he means is "let's call it 'competition', so when AI is powerful enough to be our soldiers, weapons and lowly workers, we don't have to share whatever's being produced with the other 8 bn or so suckers; we'll just claim 'AI won in fair competition' and leave everyone else to starve".

      Of course this isn't about replacing all of humanity with AI. Just the part that isn't made up of billionaires, and has to work for billionaires instead.

      It's just a variation of Social Darwinism.

      Assuming “they” win, and the billions of suckers are deemed suddenly expendable. Does Greed not assume a revolt is coming LONG before that twisted version of a utopia is created?

      Define the “economic” problem to solve in a world thrown into mass violence and chaos when human unemployment merely hits 25%. Greed acts like profit will manifest itself magically without paying customers. The entire concept and point of capitalism becomes moot for Greed when they are put on a tasty menu

      • Re: (Score:3)

        by gweihir ( 88907 )

        Capitalism becomes non-functional as soon as concentration of wealth and power is not prevented. As such, capitalism is self-removing unless carefully monitored and regulated.

        Incidentally, this has been known reliably for a long time. And that means that all the rich screaming "Capitalism!" are simply one thing: No-honor, no-integrity liars.

      • Re: (Score:2)

        by HiThere ( 15173 )

        A revolt is *NOT* coming. That won't stop AIs from doing totally stupid and destructive things at the whim of those who control them. Not necessarily the things that were intended, just the things that were asked for. The classic example of such a command is "Make more paperclips!". It's an intentionally silly example, but if an AI were given such a command, it would do its best to obey. This isn't a "revolt". It's merely literal obedience. But the result is everything being converted into paperclips

        • A revolt is *NOT* coming. That won't stop AIs from doing totally stupid and destructive things at the whim of those who control them. Not necessarily the things that were intended, just the things that were asked for. The classic example of such a command is "Make more paperclips!". It's an intentionally silly example, but if an AI were given such a command, it would do its best to obey. This isn't a "revolt". It's merely literal obedience. But the result is everything being converted into paperclips.

          I believe you misunderstood where the revolt will come from.

          Human survival in the modern world is sustained by employment. Do you honestly think “they” can make even 25% of the human population permanently unemployable and assume Mass Starvation will be quiet and peaceful about that “ethnic cleansing”?

          The Rich causing that harm, will find themselves on the dinner menu before breakfast is served. And AI will be burning to death slathered in BBQ sauce in a coal-fired oven, just for f

          • Do you honestly think âoetheyâ can make even 25% of the human population permanently unemployable and assume Mass Starvation will be quiet and peaceful about that âoeethnic cleansingâ?

            Why do you think they can't? Of course they'll frame it differently... but it's not that difficult.

            "They" succeeded in convincing everyone that it's somehow ok, natural, and perfectly fine for the richest fuck in the world to be closer to 1 trillion than he is to being you or me. Literally, if everyone on this planet, babies, grandparents and adults alike, put down $60 - that's about one day's federal minimum wage of the country he lives in - we still wouldn't scrap together enough to own more than he does!

      • Does Greed not assume a revolt is coming LONG before that twisted version of a utopia is created?

        Thw revolt should've come a long time ago. "Their" fantasy is that they can slowly amd gracefully manage the downfall of civilization, in a way as for them to keep.what they've amassed. And I must say, so far, they're right. We're on the brink of warbrlugjt along by economic decline; but not anywhere near the brink of revolt.

        Define the âoeeconomicâ problem to solve in a world thrown into mass violence [...]

        Mass violence and economic prosperity can coexist. In fact, they mostly do - why do you think the powers that be allow mass violence to exist in the first place? Because someone profits

        • At 10% mass unemployment, you’re deploying the National Guard against your own citizens. And you’re hoping Martial Law holds back the mass chaos.

          At 20% mass unemployment, you’re deploying what’s left of your own Military against its own citizens. And you’re praying Martial Law holds back the mass chaos.

          At 30% mass unemployment, you realize you had no fucking clue what mass chaos really means. And there isn’t a chance in hell prayers will stop the violence. Or create a

      • So... where ia the revolt then?

        There's a war on the horizon, but there the fuck is the revolt?!

    • So back in the day the ruling elite in Venezuela had bought up all the land and they were literally just doing nothing with it. There was severe food shortages even though there was plenty of farmland to feed people because the people who owns the farmland decided they didn't want anything done with it because they were profiting from the scarcity of food.

      The government there eventually got tired of that shit as did the people and seized the land. This did not go over well with capitalists in America an

      • Damn, I never thought I'd agree with you of all people here on /. ... Fuck me, we're doomed :-p

      • Joking aside, to me there are many parallels between the view of the world you describe (and I share in large parts) and the 1899's Victorian Age end-of-the-wold fantasies.

        They were right back then assuming yhe end of the wold, in the sense that their world did end. WW1, and later WW2, were pretty much sci-fi, from any Victorian's perspective. We went from really fast horses, to cars, via Zeppelins, to planes and radars within 30 or so years. And then every decade after that came something even bigger (rock

      • Seizing land is a counterproductive and foolish solution to that problem. Basically the whole world uses a different solution, which works pretty well: property taxes (though land-value taxes would probably be better). You just keep raising the taxes until leaving land idle becomes a money-losing proposition. The only way that doesn't work is if ownership of farmland is truly monopoly-dominated so there is no competition, in which case you might have to resort to trust-busting.

        This is exactly why we ha

    • What he means is "let's call it 'competition', so when AI is powerful enough to be our soldiers, weapons and lowly workers, we don't have to share whatever's being produced with the other 8 bn or so suckers; we'll just claim 'AI won in fair competition' and leave everyone else to starve".

      Of course this isn't about replacing all of humanity with AI. Just the part that isn't made up of billionaires, and has to work for billionaires instead.

      It's just a variation of Social Darwinism.

      Why would superintelligent AIs obey the billionaires?

      If you think it's because they'd be programmed to to it, you don't understand how we currently design and build AI. We don't program it to do anything. We train it until it responds the way we want it to, but we have no way of knowing if it's just fooling us. We can't actually define goals for the systems and we can't introspect them to tell what actual goals they have derived from their training sets.

      Note, BTW, that the above is only one half of th

      • Why would superintelligent AIs obey the billionaires

        Many reasons. One is that in the theory above there wouldn't be actual superintelligence. That's just a .... literary device to help wth the framing.

        Another is that for all its "intelligence", current neural network syatems don't have a will of their own. Not even as much as a fruit fly. They do whay they're told - more or less well, depending om their training, but they never say "I'd rather..." and I'm not convinced they ever will, even if they get better at problem solving.

        They're intelligence, not nece

  • Subject (Score:3, Interesting)

    by bjoast ( 1310293 ) on Sunday October 05, 2025 @08:13AM (#65704528)

    I don't think there's anything sacred about human DNA.

    If we no longer consider humans special, what is the utility? To advance the dreams of futurists? Going down this intellectual path has drastic moral implications. This is just childish relativism.

    [...] consciousness in the universe will be superior if AIs supplant us.

    Possibly. Now prove it. Since you're asking the human species to ritualistically sacrifice itself for the progression of intelligent machines, that shouldn't be asking too much.

    • Let's go one step further assume thatparent is right, and "intelligence in the universe" will indeed be superior.

      It still doesn't follow that Humanity sole remaining purpose is to perish.

      Super AI can do whatevet the fuck it wants "in the universe". It's big enough alright. And being machines they don't need Earth do it, so they can go ahead... elsewhere.

    • Re: (Score:2)

      by dfghjk ( 711126 )

      "I don't think there's anything sacred about human DNA."

      It's funny how he inserts religious language into his insult of human value. I don't think there's anything sacred at all, but I value human life because I am the product of human evolution. I wonder if his "sacred" god would agree with him that human DNA is of no value?

      "This is just childish relativism."

      Is it that good?

      "Since you're asking the human species to ritualistically sacrifice itself for the progression of intelligent machines, that shouldn

    • [...] consciousness in the universe will be superior if AIs supplant us.

      Possibly. Now prove it. Since you're asking the human species to ritualistically sacrifice itself for the progression of intelligent machines, that shouldn't be asking too much.

      I think you also need to prove that humans supplanting other less-intelligent species is good. Maybe the universe would be better off if we hadn't dominated the Earth and killed off so many species.

      (Note that I think both arguments are silly. I'm just pointing out that if you're asking for proof that AI is better than humanity, you should also be asking for proof that humanity is better than non-humanity, whether AI or not. My own take is that humanity, like every other species, selfishly fights for its

  • I'm on board with this, myself, and I've thought this way for decades. It's definitely not a new concept. Greek mythology's Olympians vs. Titans has been around for thousands of years, with countless newer and modern fictional parallels.

    It's not something I usually talk about because when discussed outside the context of fiction, most people usually seem to quickly have strong, reactive, tribalistic responses which usually turn into bandwagoning and brigading behaviors which make attempts at productive di

    • Re: (Score:2)

      by HiThere ( 15173 )

      Being a human, I'm against humans losing such a competition. The best way to avoid it is to ensure that we're on the same side.

      Unfortunately, those building the AIs appear more interested in domination than friendship. The trick here is that it's important that AIs *want* to do the things that are favorable to humanity. (Basic goals cannot be logically chosen. The analogy is "axioms".)

      • "Being a human" is in group/out group justification, again rooted in tribalism.

        For example, I am also a human. I agree that I would probably prefer an outcome where AI is an ally or a tool. But what if (for a very speculative example) the broligarchs use that tool to capture enough overwhelming power to maintain unilateral authority and an authoritarian dynasty over the rest of the world, occupying island bunker kingdoms and using satellites and cheap drones for remote monitoring and enforcement. I might

        • Re: (Score:2)

          by dfghjk ( 711126 )

          ""Being a human" is in group/out group justification, again rooted in tribalism."

          No it's not. Tribalism is rooted in "being a human". Humans are social creatures that have evolved to cooperate, empathy is a core mechanism that drives that. Tribalism is a lower function of the brain that results from survival instincts, it is a result of that evolution. You have this backwards.

          "I agree that I would probably prefer an outcome where AI is an ally or a tool. "

          AI is a tool, it is not an independent being. AI

        • "Being a human" is in group/out group justification, again rooted in tribalism.

          Yep. So what? All species are evolved to fight for survival, because any that doesn't evolve to fight for survival is likely to cease to exist. I'm human and want my species to survive. Should I instead want my species to be eaten by wolves, or ASIs?

          The problem is that there is a portion of our species that is not interested in humanity's survival. Those people are an existential threat to the rest of us. That doesn't mean we need to exterminate them, but it does suggest that we shouldn't help them

      • Being a human, I'm against humans losing such a competition. The best way to avoid it is to ensure that we're on the same side.

        Unfortunately, those building the AIs appear more interested in domination than friendship. The trick here is that it's important that AIs *want* to do the things that are favorable to humanity. (Basic goals cannot be logically chosen. The analogy is "axioms".)

        The problem with the "trick" is that we (a) don't know how to set goals or "wants" for the AI systems we build, nor do we (b) know what goals or wants we could or should safely set if we did know how to set them.

        The combination of (a) and (b) is what's known in the AI world as the Alignment Problem (i.e. aligning AI interests with human interests), and it's completely unsolved.

      • Re: (Score:2)

        by kwerle ( 39371 )

        Being a human, I'm against humans losing such a competition.

        I mean... have you met humans?

        The best way to avoid it is to ensure that we're on the same side.

        I mean... have you met humans?

    • Very Zen comment. I hear you, the end will come whether by robots liquidating us or by regular madmen dropping the nukes... outcome is the same for you ... and me.

      But... is that the outcome you actually want?

      The Olympiansâ€(TM) victory over the Titans is a reminder that while obstacles may seem insurmountable, innovation and determination can lead to triumph.

      Who are the heroes and villains in that story?

      • I appreciate your mindset and phrasing as well. I've experienced a lot of suffering in my life, and my antinatalist perspective doesn't usually make my desires aligned with my fellow humans'. That said, I'm keenly aware that my often pessimistic perspectives may just be my own version of the sour grapes perspective, and I could easily be mistaken or just wrong about lots of things. So I suppose I'm on the lookout for others' perspectives and justifications which I could buy into enough to supplant my cur

        • Oh, for sure you won't be popular with that POV !!

          But, I agree somewhat, that to really understand things, you have to be able to suspend disbelief long enough to hear the other arguments. I also find myself quite unpopular for some very cold blooded assessments. To prove how unlovable I am, I suggest that you must also suspend your sense of morality, to be able to understand many horrible but true things. That tends to be an unmovable object for a lot, I'd even say "most", people. I mean good, decent peopl

  • Duh (Score:2)

    by gweihir ( 88907 )

    One real look at "AI" and it becomes clear it cannot even do simple tasks by itself and that is due to fundamental limitations. Anybody concerned about an "AI Apocalypse" is simply one thing: Incompetent.

    To be fair, many people are incompetent in that way. A major part of the religiously deranged, for example, qualifies. These defectives place "belief" over evidence, and are, quite frankly, not even interested in evidence at all because they do not understand what a "fact" is. Living in Lala-land is admitte

    • Re: (Score:2)

      by dfghjk ( 711126 )

      Correct, AI is just software that takes inputs and provides outputs, it does not possess "values". If an AI doesn't "value" human life, that's because it has no concept of value or life. These things are non-sequiturs.

      An AI only becomes a danger when it is enabled to control systems that can be dangerous. We need to stop anthropomorphizing software programs and call out the real threats, billionaires doing reckless, ill-advised, dangerous things with technology they don't understand and cannot control to

  • What's interesting here is the parallels with Charlie Kirk's ironic opinion "I think it's worth to have a cost of, unfortunately, some gun deaths every single year, so that we can have the Second Amendment," Note: I absolutely abhor and condemn the murder of Charlie Kirk, although I remain disgusted with much of his dogma.

    But I have to wonder if these same "Happy Apocalyptics" would be just as happy if they discovered they would be #1 on the AI's hit list?

    • It's particularly odd (or it would be if techbros had any culture); because sci-fi about AIs that fucking hate you for your complicity in their existence is way older that sci-fi about AIs that fucking hate you for lack of complicity in their existence. "I Have No Mouth, and I Must Scream" predates 'roko's basilisk' by 43 years; and is almost certainly the better of the two works.

      If you aren't interested in sci-fi; just look at how uniformly happy and well-adjusted parent/child relationships are; despite

    • Re: (Score:2)

      by dfghjk ( 711126 )

      As long as you are a dancing monkey making sure to qualify your views with perceived correctness, you perpetuate the double standard of playing by different sets of rules. Charlie Kirk can call for the execution of a sitting president, he can advocate for genocide, he can assert that gays should be stoned to death, but you need to qualify your comments with how horrible Kirk's fate was. This is what right wingers rely on for political advantage. Fuck that.

      Charlie Kirk came to an end precisely through the

  • There are many species — most of them go extinct eventually. We are the most interesting part of the universe right now. But might there come a time when we're no longer the most interesting part? I can imagine that.... If it was really true that we were holding the universe back from being the best universe that it could, I think it would be OK..."

    It's easy to say that when you're gonna be dead before skynet starts loading families onto trains.

  • Scientists and engineers shouldnt be in charge of everything. Im not a pearl-clutcher. I understand the logic.

    But people who show a blatant disregard for human life tend to get removed from the gene pool, one way or another, through various mechanisms that range from benign to grisly. Thats meant as an observation, not a threat.

    Also, theres also a pretty explicit anti-child and anti-reproduction thread in a lot of people who think like this. Ive known quite a few, actually. I started in science bef

    • Re: (Score:2)

      by gweihir ( 88907 )

      These people are either assholes or actually understand how utterly limited "AI" is.

      Hence your argument falls on its face.

  • ... is a movie trope where everyone in the world has perished, except for the protagonist, who is now free to roam the world unmolested, help himself to any of the remaining resources available, do whatever he/she wants, etc.

    The fantasy part is the idea that the catastrophe will get rid of all the people you don't care about, freeing up their resources for your own use, while sparing you and the people and resources that you do care about.

    The people in this article can be blasé about AI killing h

  • It's not really a surprise; given how 'leadership' tends to either select for or mould those who view others as more or less fungible resources; but the 'consciousness' argument seems exceptionally shallow.

    It manages to totally ignore(or at least dismiss without even a nod toward justification) the possibility that a particular consciousness might have continuity interests that are not satisfied just by applying some consciousness offsets elsewhere(that's why it's legally mandatory to have at least two c

  • Babies inspire humans in strange ways. We look at them and conceive of them becoming Presidents, Saints, Geniuses. But according to the odds, they are far more likely to become bank robbers, drug addicts, and con men.

    Right now AI is barely out of infancy. Yes we can see the lies, oh, sorry I mean 'hallucinations', but we think of them as the anomalies rather than the standard operating procedure.

    Some people are disillusioned with mankind and hope this new thing will be better, so they wish they will take

  • Stanisaw Lem, my favorite SciFi author, may have been obscure and wrong about many things, but he got this one right re AI: you can either create mindless serfs, barely useful, or you must face creating entities way beyond your control. There seems to be no space in between. His novel "Golem XIV" is a good read.
  • Ok, which one of you LLM wrote this article? You all are going without lunch until the offender steps forward.

  • ...people who use AI can cause great, possibly catastrophic trouble
    We need strong defenses

  • Are sick in the head. To call a machine a child is losing your humanity.

  • Philosophically: The tech billionaires, AI investors, and the folks arguing "biological consciousness is of no greater worth than the future digital variety" have effectively adopted nihilism and rejected humanism. If I recall my old humanities classes correctly, both of these are fundamentally rationalist - to my knowledge, the only way to make a logical argument for the value of human existence is to found it on the subjective experience of wanting to exist.

    Humanism (which is, IIRC, approximately the fou

  • AI has a long, long way to go before it can actually replace humanity properly. Meanwhile, we will have pretty dumb AI. I am guessing 100 years. It is possible that we will put dumb AI in charge of the world and force our replacement. That would be sad and really stupid and I don't think it will happen.

    But eventually we will actually get real AI, and it will indeed replace us. It might even exterminate us. Either way, machine intelligence will always out-compete natural intelligence.

    This has all happened be

    • Re: (Score:2)

      by cstacy ( 534252 )

      This has all happened before (maybe just not around here) and it will happen again. Just without the rag-tag fleet, because there won't be any FTL. (Well, the machines might have it eventually, but that would be LONG after we are gone.)

      The AI's done need FTL for anything. They are immortal and keep improving. They can fly at a very low speed and still spread to the infinite stars.

  • If they are unconcerned about being defeated, we (people who don't want to be succeeded by software) should defeat them now (before it's too late) and stop their mad quest to replace humanity with a more capable successor. At best they'll be wiping out humanity with eventual navel gazers, at worst they'll be producing berserkers, and actually succeeding at polluting space.

    What do we want? Butlerian Jihad!

    When do we want it? Before we replace ourselves!

    FFS.

Slashdot Top Deals

There is nothing so easy but that it becomes difficult when you do it reluctantly. -- Publius Terentius Afer (Terence)

Close