Anthropic Safety Researcher Quits, Warning 'World is in Peril' (semafor.com) 77
An anonymous reader shares a report: An Anthropic safety researcher quit, saying the "world is in peril" in part over AI advances. Mrinank Sharma said the safety team "constantly [faces] pressures to set aside what matters most," citing concerns about bioterrorism and other risks.
Anthropic was founded with the explicit goal of creating safe AI; its CEO Dario Amodei said at Davos that AI progress is going too fast and called for regulation to force industry leaders to slow down. Other AI safety researchers have left leading firms, citing concerns about catastrophic risks.
Anthropic was founded with the explicit goal of creating safe AI; its CEO Dario Amodei said at Davos that AI progress is going too fast and called for regulation to force industry leaders to slow down. Other AI safety researchers have left leading firms, citing concerns about catastrophic risks.
Re: (Score:2)
Said before a AI-designed superbug took out you and your entire family. Do you think or just spew gibberish?
Re:wheeee censorship (Score:4, Interesting)
The problem isn't superbugs. It doesn't need the ability to innovate. It just has to enable a complete idiot to do something they couldn't figure out on their own. There are plenty of scientifically trivial things that are very dangerous in the wrong hands. I also don't think censorship is the answer, yet that is also not quite what this is.
Last year it was openai (Score:4, Interesting)
The company we could not do without.
This year it is arthritic, the outfit that admitted that "AI" is a dead end, but then slapped a neverending loop of reparsing long json bits onto that shit and called it a business model.
It is almost like all knowledge of architecture design is gone and the whole of "AI" is vibe-coded by stoned schoolchildren.
Wait, exactly as it was in the year 2000.
Re: (Score:3)
The company we could not do without.
This year it is arthritic, the outfit that admitted that "AI" is a dead end, but then slapped a neverending loop of reparsing long json bits onto that shit and called it a business model.
It is almost like all knowledge of architecture design is gone and the whole of "AI" is vibe-coded by stoned schoolchildren.
Wait, exactly as it was in the year 2000.
Correction. AI is vibe coded by *RICH* stoned schoolchildren that want to utilize it to get richer. If it ruins all of society in the process? No skin off their nose. They've convinced the world that it's too big and important to fail, so they'll get bailed out when they crush society and/or the economy under their arrogant failure.
Re: (Score:2)
Of course, how would you get to know the "investors" otherwise?
But don't worry, there is an important role for the plebes, too.
It is to pay for the program that will save the rich after the burst.
We have lost our ability to debate and decide (Score:5, Insightful)
I don't have a good idea as to what has caused it, but look at any TV debate up to sometime around the 1980s and you are likely to find a logical discussion, with little tolerance for lies, exaggeration, or logical error. That seems to have gone now.
Scientists might show that we are in trouble, and the result is that the most powerful deride the scientists, and most voters are incapable of the thought required to arrive at an informed opinion.
Politics and social debate need to fundamentally change.
Re:We have lost our ability to debate and decide (Score:4, Insightful)
Re: We have lost our ability to debate and decide (Score:1)
Re: (Score:2, Insightful)
If it isn't tech, it is nation-states with their psy-ops teams wanting a media platform to push their agendas or outright do falsehoods.
The problem is that due to their self-inflicted wounds, journalism is gone, and there is zero trust in the press. This allows almost anyone to come in, assert almost anything, just because there is no need for them to have to prove otherwise, and there is no penalty for lying, such as blaming the trash piles in NYC on "socialism". (No... the fact that the city had snow an
Re: (Score:3)
Re: (Score:2)
Not many sources of real journalism. A lot are paywalled, and people don't have the cash often for them. Which leaves the news "sources" that are paid for, but with some reason behind them.
Absolutely, real journalism costs money. I dislike paywalls as much as anyone, but in reality free news on the internet is like free anything else on the internet, it is not there primarily for your benefit. In the end I do pay for some news sources, because largely you do get what you pay for. The fact that the free stuff is all many (most?) people can afford is unfortunate, and also explains a lot.
Re: (Score:2)
I suspect many of us are adverse to paying a news site directly. I'm old enough to remember news just came with my television subscription. I'm sure many people feel that by "paying for Internet" we shouldn't have to pay again for specific sites.
Of course, the money we paid for a cable sub was split x ways between all those channel owners, but you don't see that as a consumer. The ISP just gives you network access and that's it. Most people don't really understand how the Internet works, so to them it reall
Re: (Score:2)
We know what happened, in the 80s everyone got their news from the same source.
Perhaps the only real difference back then was everyone tended to believe that news source, regardless if it was true or not.
Good Morning, Vietnam is both a comedy and a documentary on exactly how news is modified/censored to warp narratives. Tends to show just how old certain tactics in media truly are. Today, damn near anyone with a microphone can bullshit-for-profit. Lot more noise because of that.
That said, government and media still have a monopoly on Weapons of Mass Distraction. The Epstein file
Re:Science: the god that failed (Score:4, Informative)
the predictions of science didn't live up to the hype
1/10 really weak trolling. fuck off.
Re: (Score:2)
Yeah? But that account name!
Re: (Score:2)
Hey...that's the combination to my luggage!!!
Re:Science: the god that failed (Score:4, Insightful)
It's pretty much a trope that coffee has been bad for us one week and then good for us again the next week, ad infinitum.
Scientists are not the ones telling you this. The people telling you this are the same ones that inject themselves into every conversation and try to convince you to buy things you don't need.
Re: Science: the god that failed (Score:3)
I partly agree. It's also a bunch of "journalists" who will write anything for views, including anything they don't understand themselves. Things like looking at a summary of one scientific article, misunderstanding it, and ignoring all other science on the same topic. This kind of nonsense already existed in the old media landscape, but it was a minority of content, at least where I lived. Nowadays the suggestion algorithms of a few social media companies decide how much of that content people will see.
Re:Science: the god that failed (Score:5, Insightful)
It would be very amusing to see the world through your eyes.
How exactly do you think science works? Do you think that someone asks a question, and then scientists all get together in a single meeting to answer that question, and then they post the answer and claim it is the full and unquestionable truth?
No, that would be ridiculous. Instead, science is performed by millions of individual scientists, who each seek to understand some particular aspect of reality just a little bit better. They perform discovery, form a hypothesis, test the hypothesis, and then publish their results for the world to review.
There is no illuminati-style organization that coordinates all of the scientists, and their findings, together. Thus, one thousand different studies about various effects of coffee upon human health might be performed, and they might all study slightly different aspects of the topic, or test things in different ways. This in no way implies that the scientific method is flawed, or that "scientists" are just a bunch of goofy mind-changers who can never quite figure out which way is up when it comes to figuring out what science actually means.
This is to be expected for something as complex as science, unless you have an extraordinarily simple mind.
Re: (Score:2)
Thanks
Very well explained, 100% with you
So the problem has been identified, it's incredible that millions of scientists haven't yet realized that we need the illuminati on top to actually get things done, they do nice work and all, scientific method, sure fantastic, but it's clearly not giving the expected results in the end, this is because of the obvious lack of leadership and vision that the illuminati would provide
(in case it's not totally obvious, only the 3 first and the last lines a
Re: (Score:1)
LOL
None of that changes the parents point of view. The issues is that journalists have been wildly irresponsible for a long time, and the scientist doing the interviews let them get away with it because the probably thought it meant more grant money in the next round.
For decades we went round and round on things like eggs and coffee and every last one of those articles that ran in the Sunday paper would have had a line like "scientists now believe two or three cups of coffee a day will keep your heat healt
Re: (Score:2)
It would be very amusing to see the world through your eyes.
The funniest thing is that I don't think I have ever read a "coffee is bad for you" article. I've read some "coffee, with or without caffeine, is associated with lower liver disease" articles and some more recent "coffee and tea with caffeine is associated with a lower chance of dementia" ones. I guess large amounts of caffeine can cause health problems, as my gastric ulcer a few decades ago can attest to. But... yeah. That doesn't strike me as inconsistent.
I guess there's also the "coffee drinks with sugar
Re:Science: the god that failed (Score:4, Interesting)
The predictions of science led to every single thing about whatever device you're typing this drivel on. They led to the microwave you nuked your dinner in and likely to most of the food you put in there. They also led to whatever lifesaving treatments have kept you from dying from any of the numerous ailments that left most families with at least one dead child until very recently. They're why you don't have smallpox and never will.
Absent the predictions of science, you live in a hut farming the same land your great grandfather did for the same wealthy landowner. Just exactly how did the predictions of science "not live up to the hype"?
Re: (Score:2)
Re: (Score:2)
Absent the predictions of science, you live in a hut farming the same land your great grandfather did for the same wealthy landowner. Just exactly how did the predictions of science "not live up to the hype"?
Huts are more of a tool, so you are right about that. Basic tools do not need science; however, farming does. You would not be farming anything without science. You would be a hunter/gatherer.
Re: (Score:2)
Science is not to blame here, it's the media taking one small bit of info from the science, and deciding it's not dramatic enough to engage the consumer.
There's a limit to the number of times people will listen to failed prophets.
That can't be true. Most people still believe in a "God" of some form after all. Humans love to follow prophets, that way they don't have to think for themselves.
Re:We have lost our ability to debate and decide (Score:4, Insightful)
Yeah, because Ronnie Raygun didn't make up a bunch of BS about welfare queens that years later turned out to be BS. Nevermind the back door comms to Iran to get the hostages held until he secured the Presendency ... etc.
The 80s are not the bastion of good behavior and one can't have an honest conversation when folks lie and hide things.
It's all BS and lies by folks in power.
Re: (Score:2)
Re: (Score:2)
One thing the science does tell us is that we all have a very hard time separating the world that existed when we were children from our perception of that world through the eyes of a child.
Ask nearly any population in the United States when this country was best and you'll get a majority who'll swear to you it was when they were teenagers. The age of the group doesn't matter. You get the same result from 20 year olds as 40 year olds as 60 year olds as 80 year olds. And what you're seeing is people looki
Re: We have lost our ability to debate and decide (Score:1)
I don't have a good idea as to what has caused it, but look at any TV debate up to sometime around the 1980s and you are likely to find a logical discussion, with little tolerance for lies
No it wasn't. That was the era of "they're putting fishhooks in your kids' candy bars" and the moral panic over dungeons and dragons being a gateway to the occult. Nobody ever saw this shit happen but it was taken seriously on TV as if it's this daily thing. It might have helped if there were other outlets doing actual investigation.
Re: We have lost our ability to debate and decide (Score:2)
Yup. The difference was everyone had the same shitty social media feed. Word of mouth, church gossip, etc. Razor blades in your apples, vans with tinted windows, dungeons and dragons something something your art teacher is a satan worshipper, real estate only goes up, etc.
People didn't get dumber, they diversified. Now you can be part imminent disclosure fermented caveman diet, and your neighbor can be AI singularity bitcoin investor, all these different flavors of stupid bullshit.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Today we have a sizeable chunk of the population who firmly believe that it is their religious and civic duty to ignore facts and make shit up. How are you supposed to have a "debate" with that?
I share your frustration.
I do still see some examples of debate used as a means to search for truth. The young intellectual Alex O'Conner, and The Oxford Union debates are often worth watching.
In the modern social media age we were initially drawn to "debates" where one debater "smashes" the other, as it seems exciting in the way a tennis match might be exciting. But perhaps we will eventually tire of this, and find genuine debates more stimulating overall... Fingers crossed.
Re: (Score:1)
The problem is the femininisation of the school systems all over the Western world. Read https://www.compactmag.com/art... [compactmag.com]
Now "empathy" is more important than the philosophy of science. Go figure!
Ah yes (Score:2)
Quitters, the true heroes
Re: (Score:3, Insightful)
Actually it is a smart move, and the guy is spot on on security, there is none of it in the "AI".
The "architecture" of the claude and the whole "AI" crap is beyond ridiculous.
You have a stateless "AI" produce unsafe code that nobody checks, then you feed the descriptors of that code fed to the "AI" at every request so that it can re-learn what it did and call some other process to run that very code. Or some similar code. Or some code that has that descriptor. Or something.
And 99.9% of the people who do tha
Re: (Score:2)
While that's obviously accurate, the way the argument frames his points makes it sound like Anthropic are preventing them putting in protections against people using AI to spontaneously generate viral weapons that can be used to target specific populations.
Every single part of that explanation, including the unstated premises, is narrative made with the assumption that those things are functionally possible and just a matter of time... Including but not limited to implicit computational chemistry, genetical
Re: (Score:2)
Makes me think this, rather than being negative... is being spun in a way that is intended to boost the share price.
They think they're entitled to not understand. They're going to start getting impatient if we don't translate it for them. etc.
Yeah, I'm sure they seriously discuss the most absurd hallucinations that the "AI" spits at them when they ask it how to achieve world domination, and what you're suggesting isn't even uncanny or outlandish in that forum.
Re: (Score:2)
My feeling is... the majority of folks who drive this kind of thing aren't fully aware that their ideas about what the future should look like come from fiction where technology was just a backdrop to a narrative... not on a superficial level, mind you. I'm not saying they're so stupid they can't differentiate fiction from news. Mostly.
Re: (Score:2)
If you have zero ability to stop the train, the best you can do is not be complicit in the crash. Get off and let people know.
Re: (Score:2)
The one thing they never teach you about the Trolley problem is what if the driver jumps off and makes it somebody else's problem first.
Re: Ah yes (Score:2)
I don't think people like this person or Timnit Gebru are necessarily right. But any person hired as an "ethics expert" can't be taken seriously until they've quit at least one job in protest.
Crying wolf...yet again. (Score:2)
Over a billion dollars was spent on a persistent global scare mongering / PR blitz of utter bullshit that ultimately amounted to absolutely nothing. Subsequently enough people have used and or been subjected to AI. According to public polling vast majority now find it mildly useful while being largely annoyed by the nonsense and zero effort slop.
Bio-terrorism risks have been a concern long before the rise of generative AI fueled by reduced cost and improved capability of enabling technology. Despite nons
Fear advertising? (Score:1)
Surely (Score:5, Interesting)
"Drink up, the world's about to end." (Score:2)
"Drink up, the world's about to end." -- Hitchhiker's Guide to the Galaxy
Re: (Score:3)
Well, at least let us get through this Mardi Gras and party a little, you know?
"Something Big is Happening" (Score:2)
No shit... Your command of obvious is impressive!
On the upside, the tech bros are buying their own nuclear power plants, so when the AI thing blows over, we might actually have clean power or even fusion (of course, it is still 5-10 years away from commercialization (tm)). YMMV. Check with your doctor if AI is right for you. Do not taunt Happy Fun Ball.
Re: (Score:2)
Maybe we can see some advances in memory, storage (short term SSDs, and long term archival storage) created by this AI stuff. Otherwise, it may be 20-50 years before we see something for backups that is better than LTO tapes.
I don't know what to think (Score:1)
Yeah, I agree the World is in peril... (Score:2)
But it's because of this AI gold rush.
This story is 300 years old (Score:2)
Politicians and CEOs prioritize profits over people's health and safety: News at 11.
People might have a right to life and family but, too often, that means nothing.
He has a point (Score:2)
The problem is that while most engineers and scientists could do tremendous damage, they are also smart enough to not want to. The problem with LLM type AI is not that it can enable any new attacks or anything creative or the like. The problem is that it may put things that are well-known to experts in the hands of idiots and extremists who would not be able to do anything that damaging otherwise. And there are a lot extremists around and a lot of idiots that cannot even do basic fact checking and believe t
ai safety? (Score:3)
We already have an artificial entity with no guard rails capable of creating bioweapons that has a goal function that makes them take actions that are not in the best interest of humanity. It's called a corporation. So, if AI safety is really a problem that ship sailed like 500 years ago.
Journalism today... (Score:3)
Re: (Score:2)
Maybe I'm just old, but the Semafor article seems to be based on a Forbes article which seems to be based on a Twitter post. Is that really what counts as journalism today?
You just performed more research than most "journalists". Perhaps you should try your hand it? Attention from others is the only thing you are lacking.
We have lost even semi-independent (Score:2)
This is typical (Score:2)
Companies hire people to do saftey testing, if anything affects revenue they won't do it. Forget about morality or common good when it comes to making money.