Google Paper: AI Potentially Breaking Reality Is a Feature Not a Bug (404media.co) 81
An anonymous reader shares a report: Generative AI could "distort collective understanding of socio-political reality or scientific consensus," and in many cases is already doing that, according to a new research paper from Google, one of the biggest companies in the world building, deploying, and promoting generative AI. The paper, "Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data," [PDF] was co-authored by researchers at Google's artificial intelligence research laboratory DeepMind, its security think tank Jigsaw, and its charitable arm Google.org, and aims to classify the different ways generative AI tools are being misused by analyzing about 200 incidents of misuse as reported in the media and research papers between January 2023 and March 2024.
Unlike self-serving warnings from Open AI CEO Sam Altman or Elon Musk about the "existential risk" artificial general intelligence poses to humanity, Google's research focuses on real harm that generative AI is currently causing and could get worse in the future. Namely, that generative AI makes it very easy for anyone to flood the internet with generated text, audio, images, and videos. Much like another Google research paper about the dangers of generative AI I covered recently, Google's methodology here likely undercounts instances of AI-generated harm. But the most interesting observation in the paper is that the vast majority of these harms and how they "undermine public trust," as the researchers say, are often "neither overtly malicious nor explicitly violate these tools' content policies or terms of service." In other words, that type of content is a feature, not a bug.
Unlike self-serving warnings from Open AI CEO Sam Altman or Elon Musk about the "existential risk" artificial general intelligence poses to humanity, Google's research focuses on real harm that generative AI is currently causing and could get worse in the future. Namely, that generative AI makes it very easy for anyone to flood the internet with generated text, audio, images, and videos. Much like another Google research paper about the dangers of generative AI I covered recently, Google's methodology here likely undercounts instances of AI-generated harm. But the most interesting observation in the paper is that the vast majority of these harms and how they "undermine public trust," as the researchers say, are often "neither overtly malicious nor explicitly violate these tools' content policies or terms of service." In other words, that type of content is a feature, not a bug.
Re: (Score:3)
Why would anyone have doubted that Google wants to be The Ministry of Truth?
Its literally in their mission statement
I don't want to say they aren't evil, because in many ways they clearly are, however the one thing I really love about Google is how upfront they often are about the bad things they are doing. They have straightforwardly said that they want to arrange all the worlds knowledge. It's the same with their advertising. They are much more in your face and clearly tracking a
Re: (Score:2)
Re: (Score:2)
Joe Biden is old and has had a lifelong stutter. He acknowledged in a fiery rally following his debate performance: “I might not walk as easily or talk as smoothly as I used to.” This has impeded his ability to communicate with mass audiences with consistent success. AI augmentations and video renderings could serve to smooth out these bumps while allowing the Biden campaign to effectively disseminate true information about the state of our democracy and the Biden administration’s accomplishments.
Google says AI makes search suck even more (Score:1)
For those who didn't wade through the buzzword ridden summary, this is just Google saying that the net will get filled with tons of AI generated crap which will destroy traditional Google style search results which come from scraping web pages.
Re: (Score:1)
Hard to imagine it making google search even worse. It's terrible now compared to what it used to be.
Re: (Score:2)
For those who didn't wade through the buzzword ridden summary, this is just Google saying that the net will get filled with tons of AI generated crap which will destroy traditional Google style search results which come from scraping web pages.
So, just like it is today, but worse? That sounds about right.
Re: (Score:1)
Yup, pretty much. Probably with AI generated ads, too.
It's not a bug, it's a feature! (Score:3)
Now, where have I heard that crap before?
Re: (Score:2)
In other worlds it might be labelled as an undesirable side-effect.
why should AI be any different? (Score:5, Interesting)
Re: (Score:1)
Or, gaslighting an entire country into voting for a potato for president? Well done, well done.
Tech industry colluded with Government to install the potato, that's called fascism
Fuck you, and everyone who enabled this.
Re: (Score:2)
Though, of course, I would rather have a potato as president that someone who is a malignant cancer on society.
Re: (Score:2)
And your sig says, "Civilized Society" jumped the shark ca. 1973"
So, how in the fuck am I supposed to take your rantings seriously?
Re: (Score:2)
Or, gaslighting an entire country into voting for a potato for president? Well done, well done.
Tech industry colluded with Government to install the potato, that's called fascism
Yeah, that's bad alright, but it's not fascism. [wikipedia.org] From the Wikipedia article:
Fascism (/fæzm/ FASH-iz-m) is a far-right, authoritarian, ultranationalist political ideology and movement characterized by a dictatorial leader, centralized autocracy, militarism, forcible suppression of opposition, belief in a natural social hierarchy, subordination of individual interests for the perceived good of the nation or race, and strong regimentation of society and the economy.
Entities could collude to install a "potato" who is a fascist dictator, but the actual collusion is not facism. Such collusion could be employed by parties with other goals.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Sorry, but that's NOT treason. Read the constitution again:
Article III, Section 3, Clause 1: Treason against the United States, shall consist only in levying War against them, or in adhering to their Enemies, giving them Aid and Comfort. No Person shall be convicted of Treason unless on the testimony of two Witnesses to the same overt Act, or on Confession in open Court.
I'm pretty sure that in context "Enemies" means a foreign nation that we are at war with. (I don't think even a "police action" would cou
Re: (Score:2)
I'm pretty sure that in [the context of the Constitution Article III, Section 3, Clause 1] "Enemies" means a foreign nation that we are at war with.
Not necessarily foreign. The Civil War Confederates committed treason. Some were even indicted for it, but AFAIK none stood trial.
(I don't think even a "police action" would count. I think it requires an official declaration by the Senate.)
Or at least by Congress, but yes, I think this is correct.
Re: (Score:2)
Trump isn't really a potato, more like a poisonous yam.
Re: (Score:2)
Better king log than king stork. --Aesop
Re: (Score:2)
So a habitual criminal is better than a potato? Incidentally, you need to look up what "Fascism" means. You are throwing around big words you do not understand.
Re: (Score:2)
We already have pundits from all walks of life doing this, why should AI be any different.
Intent.
Re: (Score:2)
I think the idea is that AI is "pure" in some sense (which is basically impossible if it gets trained on data written by humans) and hence has access to "truth" that humans have not. essentially a deity-myth, of which the human race has an incredible amount. No connection to reality, obviously.
Obviously, antrophomorzation and a complete lack of clue how this tech works by most people that comment on it also plays a major role. But somebody rational can see that AI has no understanding of anything after a 10
Re: (Score:2)
The problem is that AI may come up with a narrative that breaks all of the narratives that we are being fed. Lying for personal gain is already covered in the rules of the game, but lying sincerely for no concrete purpose changes the nature of the game entirely. This is gonna be "fun". Wheeeee
Now they're worried? (Score:2)
personality disorders (Score:2)
Re: (Score:2)
I doubt that. Most people will just listen even less to what does not match their misconceptions. Remember the average person is _dumb_. Dumb enough that it would be a personality disorder if it were not so prevalent.
I Wouldn't Worry (Score:2)
AI is for the most part unregulated, which means the industry will self regulate itself.
This is exactly how the free market is meant to work so what's to worry about?
Re: (Score:2)
When has any industry ever successfully "regulated itself"?
Re: I Wouldn't Worry (Score:2)
MPAA ratings system.
Re: I Wouldn't Worry (Score:2)
Re: I Wouldn't Worry (Score:2)
What regulations? From the government? Or are you talking about the Hays Code? Cause the Hays Code is what the ratings system was introduced to replace and the Hays Code was an example of the industry self regulating.
Misinformation existed before AI (Score:2)
Massive misinformation campaigns (aka propaganda and marketing) existed way before AI. Is the current criticism or concern that AI misinformation is opaque or seemingly random? Shouldn't we be far more concerned about misinformation and subliminal control by (1) Russian/Chinese agents influencing US elections and (2) large social media corporations?
Re: (Score:2)
I think the real problem is that the AI easily generates superficially convincing bullshit, so that those too incompetent to do their own bullshitting are empowered. And apparently there are LOTS of them.
(Or perhaps it's that the artificial bullshit is cheap enough that we are being crapflooded.)
Misleading title (Score:3)
Google's claim that a "harm" that "undermines public trust" isn't overtly malicious doesn't sound like an endorsement to me. Most bugs are not overtly malicious. It sounds like they recognize the problem.
Reticulum (Score:2)
Solved (Score:2)
This was, for the most part, a solved problem before the advent of Web 2.0. Before the internet we had gatekeeping organizations (newspapers and network TV news) that employed journalists, who were professionals skilled in perceiving their own bias, presenting "just the facts", doing fact checking, etc. There was even a regulation called the "fairness doctrine" that required network television to give equal airtime to both sides.
When the web started, the only people who could actually publish information
Re: (Score:2)
Re: (Score:2)
Dude, I was there calling for the change. We literally thought the web would allow all of humanity to access the wealth of knowledge they couldn't access. In a way, it has. Wikipedia is pretty amazing, and has held up pretty well to the misinformation pressure (though it's also susceptible).
Our society uses the same mechanism of improvement that evolution does: we try out a bunch of new ideas, and keep the ones that work. That means you need two mechanisms: one that generates new ideas to try, and a comp
Re: (Score:2)
Well, you are not wrong about the causes. Unfortunately, the "fix" you propose will not work and did not work before. Journalists are notoriously badly paid and most are easy to manipulate or to control outright.
No (Score:2)
There is no such thing as scientific consensus. Science is an academic discipline that can be performed equally well by one person or a team of people.
Scientific evidence is not affected in any way by the number of people who agree, nor is it concerned with social or political orthodoxy. The scientific method does not (and should not) adapt to accommodate opinion.
Meanwhile, people who claim to revere science are often found to be quite impatient with anyone who speaks in declarative sentences, and are even
Re: (Score:2)
P.S. I majored in English, and I've found over the years that I understand science far better than some who claim to be "scientists," even though my greatest achievement was passing sophomore physics.
I think this statement really needs to be up front. People like you are really needed to that we can come to you instead of getting our knowledge about things like black holes from idiots like Neil deGrasse Tyson who think that just because they know the physics and science of the subject they might have some knowledge to impart.
There is no such thing as scientific consensus. Science is an academic discipline that can be performed equally well by one person or a team of people.
Scientific evidence is not affected in any way by the number of people who agree, nor is it concerned with social or political orthodoxy.
Absolutely. I've always thought the way that single geniuses with perpetual motion machines were laughed at just because all the serious scientists in the world think they are wrong
Re: (Score:2)
The scientific consensus at the time of Galileo was that the Earth was at the center of the universe. Science wasn't just a little bit wrong, and it wasn't for a short time. It literally took science about 2000 years to figure out the Earth wasn't the center of the universe, and about 200 years for the idea to finally be accepted by the public at large.
The overwhelming majority of scientific ideas about the universe have later been proven wrong, or required subsequent revision because reality was a bit
Re: (Score:2)
contemporary "consensus" understanding is so far off base in so many areas of "science", its frightening. The scientific method has largely been abandoned in favor of voodoo math, thought experiments, simulations, circular reasoning, inadequate controls, and peer wanking. Personally, I plan to homeschool my children so they won't be taught total nonsense as if fact, and will instead learn to evaluate alternative theories and form their own.
I cringe to think of the ignorance you are passing on to your children. Let them learn science from scientists, and then they can teach it to you.
Re:No (Score:4, Interesting)
The scientific consensus at the time of Galileo was that the Earth was at the center of the universe. Science wasn't just a little bit wrong, and it wasn't for a short time. It literally took science about 2000 years to figure out the Earth wasn't the center of the universe, and about 200 years for the idea to finally be accepted by the public at large.
The work of Copernicus that proposed heliocentrism pre-dates that of Galileo by almost a century. I'll grant you that heliocentrism was not widely accepted even at the time of Galileo, but its main detractor was the Catholic Church. Scientists were the first to correct their own mistake (because that's how science works) and others followed eventually.
The overwhelming majority of scientific ideas about the universe have later been proven wrong, or required subsequent revision because reality was a bit more subtle than humans could understand. The likelihood that contemporary scientific understanding is actually true is vanishingly small.
Scientist are quicker than you think to adapt their theories to new knowledge. They keep using what is proven to work until they find situations where it doesn't. And it is very rare that scientific ideas need to be discarded completely. Usually they need to be adjusted to accommodate observations made possible by new technologies and experimental procedures. For a description of the details, see the excellent essay by Isaac Asimov entitled The Relativity of Wrong. [hermiene.net]
Re: (Score:2)
They keep using what is proven to work until they find situations where it doesn't.
As a great example of this, every day you use the theory pre-Copernican theory of the world, measuring the speed of your car by assuming that the ground is not moving and that the car is moving relative to it. For 99% of the things people do with speed, it's actually simpler and more effective than the Copernican theory that the planet is moving around the sun. "All models are wrong, but some are useful".
Re: (Score:2)
The overwhelming majority of scientific ideas about the universe have later been proven wrong, or required subsequent revision because reality was a bit more subtle than humans could understand. The likelihood that contemporary scientific understanding is actually true is vanishingly small.
There are two different worthwhile definitions of wrong here. Firstly, "useless and needing abandoned". For that, there is zero chance that current quantum theory and the basic physics that scientists use today will be abandoned. The calculations just work too well and most of the technology developed since the middle of the 20th century relies on them too much for them to be wrong in any real meaningful way.
The second definition is "not fully correct and doesn't describe the way the world actually works".
Re: (Score:2)
Re: (Score:2)
Well, that story is nice, but not actually true. The church knew. Galileo said some other things the church really did not like.
Re: (Score:2)
I think this statement really needs to be up front. People like you are really needed to that we can come to you instead of getting our knowledge about things like black holes from idiots like Neil deGrasse Tyson who think that just because they know the physics and science of the subject they might have some knowledge to impart.
Yeah because people like me don't know anything about science.
Props though for trotting out Neil deGrasse Tyson as a serious example of a scientist. That brought a tear to me eye.
I've always thought the way that single geniuses with perpetual motion machines were laughed at just because all the serious scientists in the world think they are wrong was very discriminatory and wrong.
Galileo. Oh, I'm sorry, that's history not science.
Re: (Score:2)
Yeah because people like me don't know anything about science.
Props though for trotting out Neil deGrasse Tyson as a serious example of a scientist. That brought a tear to me eye.
Glad you spotted that. Originally I thought about using Steven Hawkins, because he did a bunch of good actual work on black holes, but then I realized that a) he's dead and so not so useful now and b) that didn't point out the true gap between your comments and what a basic physics graduate could understand.
I've always thought the way that single geniuses with perpetual motion machines were laughed at just because all the serious scientists in the world think they are wrong was very discriminatory and wrong.
Galileo. Oh, I'm sorry, that's history not science.
I suspect you are thinking of an Orrery. In any case, it seems most of the evidence seems to be against any real involvement of Galileo in perpetual motion.
Re: (Score:2)
There is no such thing as scientific consensus.
There really is though.
No one can afford the time to verify everything from scratch themselves, so they go with the consensus. You know on the broad brush things, like the general sphericity of the earth, the laws of thermodynamics and so on.
Science rarely works with the lone wolf making a grand pronouncement which overturns the status quo with a clear new truth.
P.S. I majored in English,
This does not surprise me.
Re: (Score:2)
No one can afford the time to verify everything from scratch themselves
That does not necessarily imply a consensus. To do so is to deny critical thinking skills and scholarship.
Science rarely works with the lone wolf making a grand pronouncement which overturns the status quo with a clear new truth.
Science always advances with a lone wolf making a grand pronouncement. See Clyde Tombaugh. Louis Pasteur. Einstein. Bell. Feynmann.
This does not surprise me.
Science always seems to be so afraid of creativity. Wonder why that is?
Re: (Score:2)
No one can afford the time to verify everything from scratch themselves
That does not necessarily imply a consensus. To do so is to deny critical thinking skills and scholarship.
Human beings all have one thing in common: someday, they're going to die. And they have to be selective about how they spend the short time they're here on this earth.
Accepting a consensus is not a denial of critical thinking or scholarship, but rather an acceptance of that thinking and scholarship done by others whose reputations you trust.
Science rarely works with the lone wolf making a grand pronouncement which overturns the status quo with a clear new truth.
Science always advances with a lone wolf making a grand pronouncement. See Clyde Tombaugh. Louis Pasteur. Einstein. Bell. Feynmann.
A lone wolf did not discover the Higgs boson.
A lone wolf did not sequence the human genome.
A lone wolf did not land a spacecraft on any moon or planet, or launch a space
Re: (Score:2)
Science thrives on creativity. If it didn't, there wouldn't be any new science.
Fair enough. Then why do you go out of your way to be a dick to people who don't major in science?
Re: (Score:2)
Sigh. Einstein.
He was no lone wolf.overturning an established order.
Newtonian mechanics was creaking. The precession of mercury was right there and couldn't be explained, along with action at a distance of gravity which even Newton himself decried.
Relativity was already buried in the equations of Maxwell: they did not about the possibility of a standing electromagnetic wave. It's always the same speed.
The attempted hack was the luminiferous aether, which was already disprove my Michelson and Morley.
There wa
Re: (Score:2)
Science always advances with a lone wolf making a grand pronouncement. See Clyde Tombaugh. Louis Pasteur. Einstein. Bell. Feynman.
It really does not. Those are the exceptions, and even they base their stuff on the (at that time) current work by many. All science is a collaborative effort, some things more than others.
Re: (Score:2)
Indeed. Occasionally, said "lone wolf" finds something, but unless others verify it, that insight dies with the lone wolf.
So while it can be a challenge in itself to find the scientific consensus when some topic boils over or is emotionally charged in the public view, it typically is possible. But it can require talking to people off-the-record for some things.
For example, about 30 years ago at a conference, I did that with a scientist working on quantum computing. Off-the-record, he said that he does not t
Re: (Score:2)
There is no such thing as scientific consensus. Science is an academic discipline that can be performed equally well by one person or a team of people.
There is such a thing as scientific consensus, just as there is such a thing as consensus in other human endeavours. However, consensus is not proof. It's an indication of how widely accepted an idea is. Proof in science is determined by experiment, and it takes only one contrary experiment to defeat a theory.
Scientific evidence is not affected in any way by the number of people who agree, nor is it concerned with social or political orthodoxy. The scientific method does not (and should not) adapt to accommodate opinion.
Agreed. But then you continue:
Meanwhile, people who claim to revere science are often found to be quite impatient with anyone who speaks in declarative sentences, and are even more impatient if they cite scientific evidence and draw appropriate conclusions.
Whoever these "people" are, they don't sound like scientists. [must...resist...true...scotsman...fallacy]
P.S. I majored in English, and I've found over the years that I understand science far better than some who claim to be "scientists," even though my greatest achievement was passing sophomore physics.
Congratulations on getting through sophomore physics. Perhaps you
Yes there is. (Score:2)
Scientific Consensus [wikipedia.org] is real and absolutely necessary. Here's why:
From a practical perspective, the vast majority of people cannot independently verify scientific studies. Most people don't have the equipment, let alone the knowledge, to do the tests themselves. Do you have a Large Hadron Collider in your back yard? Do you even know how to operate one? Neither do most people. Even if they do have the necessary education level to skim a published paper, that's not enough since they cannot do what the p
Re: (Score:2)
How does the scientific method establish objective truth?
Science does not establish truth. It establishes facts. They are two completely different things.
You need a whole bunch of people to do the same test, and get the same result, in order to say "we have reasonably eliminated insanity and human error from these results, so this is probably true.
You could also achieve a consensus without the experiments. That's one among many reasons some people distrust that word. "This group of people all agree." Therefore, what?
That's not science.
we remain faced with the problem of figuring out all the details in a world full of individual error, unknown variables, strong mental biases, and outright insanity.
Any or all of which can manifest quite readily in a crowd. The trust we place in science should be inspired by rigorous academic discipline, not slogans from toothpaste commercials.
Re: (Score:2)
You aren't making an argument, you are splitting semantic hairs. Since you majored in English, it should have been outright obvious to you what these words mean in the context that I used them. "How does the scientific method establish objective truth?" I was not talking about some weird nonsense like scientists creating reality, or reality changing depending on some democratic process. I was very clearly and
Re: (Score:2)
You aren't making an argument, you are splitting semantic hairs.
Truth and fact are not synonyms.
I was very clearly and obviously talking about the act of figuring out what the truth is!
That's outside science's purview. Truth is meaning. Facts are data. Science is limited to the latter.
The scientific method seeks to learn what is true about the real world
If we're going to have a conversation about science, then be precise. The scientific method seeks to establish facts. Science can tell you what it is. Science cannot and should not tell you what it means.
We were talking about achieving consensus on the results of the experiments, and what they mean, as required by the scientific process.
I agree. But I would also agree one needn't look far or long to find corruption giving rise to consensus. This is why consensus on its own carries so little weight.
You seem to be inventing your own definitions of words, and speaking from a position of ignorance about the scientific method and its philosophical foundations.
If that's w
Re: (Score:2)
Nope, you are still playing word games and just digging in your heels.
You are trying to draw very precise, context-dependent, technical distinctions that actually have no bearing on the simple point that I made, in order to make yourself "feel right" when your fundamental premise is dead wrong.
The need for scientific consensus is a practical reality. Without it we have nothing but wild religion, superstition and outright nonsense all over the place, precisely because (as I pointed out earlier) ordinary peo
Re: (Score:2)
The need for scientific consensus is a practical reality.
That implies science without consensus is incomplete. That is provably false.
Clyde Tombaugh discovered the planet Pluto. His discovery did not need to be ratified by a consensus to be proper science.
On the contrary, numerous other astronomers tried and failed (over many decades) to match his accomplishments. In fact, the ghouls who now claim to be the arbiters of the "consensus" on what is and is not a planet propose that we withdraw approval of Tombaugh's work 82 years after the fact. The Inquisition at i
Re: (Score:2)
You have a really messed up and disconnected view of things there.
Re: (Score:2)
Yes, pretty much. Obviously, there are always people that do not like what the scientific consensus says, so they try to discredit the approach. Except for very rare cases, these people are simply anti-Science.
Re: (Score:2)
You are wrong. Yes, _some_ sciences are based on facts verificable by anybody smart enough (mathematics, but those facts do not apply to physical reality). But most scientific evidence needs interpretation and that is where the scientific consensus comes in. Obviously, such a consensus is subject to pressure and manipulation. But it is still massively better than a consensus of non-experts.
And... (Score:2)
Yup, departure from reality is definitely a feature
They can't fix it (Score:2)
Re: (Score:2)
Indeed. Thy cannot fix it, so they try to make it a feature. There currently is a legal challenge to AI companies in the EU because they cannot fix or remove information about people that happens to be wrong. The problem is that they must fix or remove that information, no exceptions.
Re: (Score:2)
It's quite similar to inflation the "economics" term... in economics there is an inverse relationship between the quantity of dollars in circulation and the value of a dollar. The more dollars in circulation, the less each dollar is worth.
I think that's a corollory to the Dead Internet Theory. As the number of words published on the Internet rises, the value of each word goes down. In a sense, the meaning of the words is less. And here we are. As we approach an infinite number of words published, the
A black hole for meaning ... (Score:2)
... and purpose. That's what AI driven IT will be and is already becoming. I personally find it fascinating to see AI voiding my high paid occupation as a software expert and developer as we speak. All things digital, all things software and lots of things related to that are becoming a commodity. Hand coding software is turning into a niche hobby and the shifts in the IT job market that go along with that are nothing short of epic.
Two days ago I interviewed for an expert job that will officially require on
Poe's Law? (Score:2)
"...the vast majority of these harms and how they 'undermine public trust,' as the researchers say, are often 'neither overtly malicious nor explicitly violate these tools' content policies or terms of service.' In other words, that type of content is a feature, not a bug."
I'm struggling to understand whether that last sentence ("that type of content is a feature, not a bug") is sarcasm, or a very bad non sequitur. Have I been trapped by Poe's Law?