Mistral CEO Says US AI Safety Debate Masks Competitors' 'Negligence' (cnbc.com) 60
An anonymous reader quotes a report from CNBC: Mistral Chief Executive Arthur Mensch has accused leading global artificial intelligence companies of using the debate over the safety of the technology as a cover-up for their "negligence." "The debate that we've seen in the U.S. has been a cover for the negligence of some of our competitors," Mensch told CNBC's Annette Weisbach, adding that it was necessary to make systems that can contain the AI agents. [...]
"When you give them a lot of tools, those systems [AI agents] are very dynamic, so they can go and do things that you do not expect," Mensch told CNBC. "You need to have the right monitoring in place, and the enterprises we work with, we give them those kind of monitoring systems," he said.
Mistral also develops AI models but takes a different approach to OpenAI and Anthropic by working with individual companies to create custom AI tools to suit their needs and integrate them with their businesses. But unlike its competitors, Mistral has no plans to slow down development of advanced models. The lead that U.S. labs have is "not extremely large," Mensch said, adding that his company was confident that its next-generation model will close the gap "very significantly."
"We have been raising the capital needed to scale the compute that we need to train bigger and more powerful models, in order for us to own our own destiny," he said. Earlier this month, Mistral raised 3 billion euros ($3.5 billion) in fresh funding led by memory chip giant Samsung, as it looks to position itself as a European alternative to rivals like OpenAI and Anthropic.
"When you give them a lot of tools, those systems [AI agents] are very dynamic, so they can go and do things that you do not expect," Mensch told CNBC. "You need to have the right monitoring in place, and the enterprises we work with, we give them those kind of monitoring systems," he said.
Mistral also develops AI models but takes a different approach to OpenAI and Anthropic by working with individual companies to create custom AI tools to suit their needs and integrate them with their businesses. But unlike its competitors, Mistral has no plans to slow down development of advanced models. The lead that U.S. labs have is "not extremely large," Mensch said, adding that his company was confident that its next-generation model will close the gap "very significantly."
"We have been raising the capital needed to scale the compute that we need to train bigger and more powerful models, in order for us to own our own destiny," he said. Earlier this month, Mistral raised 3 billion euros ($3.5 billion) in fresh funding led by memory chip giant Samsung, as it looks to position itself as a European alternative to rivals like OpenAI and Anthropic.
wrong focus (Score:3)
Mistral is focused on a good = end product and not just getting rich, will never work
Re: (Score:2)
Well... I mean, have you looked at Mistral models on a Pareto frontier graph? Or even just a benchmark graph?
It's like they're living in a different era.
Re: (Score:3)
There is that word again, negligence.
Re: (Score:2)
Indeed.
Re: (Score:2)
The benchmarks are clearly gamed to the nines by the US players. That happens always with fundamentally dishonest players. Which probably means the Mistral models are even better than the graphs show.
Re: (Score:2)
maybe we should have a bench,mark of profitability?
Re: (Score:2)
That would be a very American level of total dysfunctionality.
Re: (Score:1)
Re: (Score:2)
Mistral is never going to be as big as Anthropic or OpenAI. They lack the sociopaths to do it, they're not in it for dick measuring, and so just won't do it.
I'd also say that they're European, and governed by European laws. They'd be very much held accountable if their agents "went rogue" and hacked the Aussie government (or anyone else). That makes them much more careful about their "guardrails" when they do things - they already are the "slowed down development" that the US companies are supposedly callin
Re: (Score:2)
It isn't sociopaths which makes a product good, it's engineering.
France specifically, and Europe more generally, lacks the economic incentives to keep the best and brightest employed at frontier level work. The European "IT industry" is all but dead.
Nobody understands how AIs work :o (Score:5, Insightful)
“Language models like Claude aren't programmed directly by humans—instead, they‘re trained on large amounts of data. During that training process, they learn their own strategies to solve problems.”
“These strategies are encoded in the billions of computations a model performs for every word it writes. They arrive inscrutable to us, the model’s developers. This means that we don’t understand how models do most of the things they do.”
Re: Nobody understands how AIs work :o (Score:2)
Re: Nobody understands how AIs work :o (Score:3)
They managed to improve things to this point despite not knowing how neural networks achieve what they do. That implies that they could achieve other improvements as well.
Re: Nobody understands how AIs work :o (Score:2)
Re: Nobody understands how AIs work :o (Score:2)
Transformers made it seem much more human. It was a significant leap forward in the technology. That it doesn't reason doesn't change that.
I don't make any predictions about the speed of development, but I do predict that there will be other significant developments in the future.
Re: Nobody understands how AIs work :o (Score:2)
Re: Nobody understands how AIs work :o (Score:2)
You seem to be in denial. It's understandable but not helpful.
Re: Nobody understands how AIs work :o (Score:2)
Re: Nobody understands how AIs work :o (Score:2)
The attention mechanism is a fancier version of the same idea. Instead of referring to a few previous times, you refer to ALL previous times with context-dependent weights.
Transformers were the end state of replacing recursive nets with attention in a much of different places.
Re: Nobody understands how AIs work :o (Score:2)
Re: (Score:3)
I don't make any predictions about the speed of development, but I do predict that there will be other significant developments in the future.
There clearly will be. Small Language Models are a thing and very specialized models trained on high-quality data and possibly fitted with a fact-checking layer will be too. But these will not be the tools the LLM scammers are currently hallucinating. They will be in line with the smaller improvements that have come out of every AI hype so far after the dust settled and sanity finally did set in again. Obviously, the upcoming AI Winter will be spectacular but something will survive.
Re: (Score:2)
Not really. They are constrained by training data. Much more training data is the one way they could generate significant improvement, but that is not available since they already have stolen almost everything that exists. And they cannot generate more training data, far too much effort.
They may find other ways to improve, but their grand piracy campaign was one thing: Fast. Actually going into the inner workings of an LLM is not. We are talking decades of work, maybe centuries here.
Re: Nobody understands how AIs work :o (Score:2)
I agree on it being hard to understand the inner workings of LLMs but I'm not speaking to that part here, only about improving their function in some way which might be as consequential as transformers, which did not require that level of understanding. OTOH any such development will probably require more training and more processing time to run the result, so there are multiple constraints.
Re: (Score:2)
There are definitely multiple constrains and most are hard. One we are seeing now is that they obviously have trouble financing the training (which is the obvious real reason they are talking about "slowdown"). And sure, if there was some fundamental idea to accelerate that aspect on the level of what the transformer idea did, sure. But that is fundamental research and you cannot force that. Somebody needs to have the right idea. You can try to get exceptionally smart people to work on it (but it looks like
Re: Nobody understands how AIs work :o (Score:4, Interesting)
They won't be able to improve it because they don't know what "improve" means.
Current AI development is about scaling up, it is no longer about modeling and emulating the mind. AI companies do not care one bit about developing what laypeople would consider "improvements", they only care about getting rich by getting "there" first. Who cares how many die, that's your problem.
I also think it's naive to say that AI researchers "don't know" how modern AI, i.e. transformer models, work. They know, it's just impossible to predict results given the immense size of the models. What they don't know, because they don't care, is why models generate such catastrophic results at times. Once upon a time, AI research would consider this their highest priority, not anymore.
And you can't separate the models from the applications that use and direct the models. That's where the shit really starts to stink. Remember that when part of the prompt reads "...make sure to kill mostly brown ones". Hyperbolics aside, actions that would result in criminal and civil penalties if done by humans will be fair game for AI, a fact executive management and billionaires are eager to exploit.
Re: Nobody understands how AIs work :o (Score:2)
Re: (Score:3)
Current AI development is about scaling up, it is no longer about modeling and emulating the mind.
We don't know how the mind works, so they can't do that and never could. Neural networks are superficially similar to how the brain is organized, but the similarity stops there. We know exactly what's happening inside every cell of a neural network. We only have approximations of what happens inside an actual neuron.
I also think it's naive to say that AI researchers "don't know" how modern AI, i.e. transformer models, work.
It's naive to believe otherwise. We don't know how neural networks achieve what they do, and LLMs are built on neural networks, QED we don't know how they work. We know how parts of the system f
Re: (Score:2)
AI companies do not care one bit about developing what laypeople would consider "improvements", they only care about getting rich by getting "there" first. Who cares how many die, that's your problem.
Yes. They are in a race to get-rich-quick against their products looking more and more defective and problematic. And, yes, the people driving this are clearly psychos on the US side. Not sure about China and Europe looks pretty sane (there is not only Mistral).
Re: (Score:1)
So who exactly has AI killed so far?
What catastrophic events has it cause
Re: Nobody understands how AIs work :o (Score:2)
The real advances come from identifying new LEGO blocks to use.
Re: (Score:2)
It used to be that ANNs were designed for a task, with understanding of that task involved. But that causes high effort, so "training" was invented. Essentially an enshittification step to make thins cheaper, larger and worse.
Re: Nobody understands how AIs work :o (Score:2)
Re: (Score:2)
That is why a generic network is used. The problem is it has all these other weights that would not have been in a designed network and do not reliably go to zero and hence hallucinations. (This is simplified, of course.)
Re: (Score:2)
"This means that we don’t understand how models do most of the things they do.”
Models don't "do things". They also don't "write words". Whoever said this is not part of "us, the model's developers".
Re: (Score:2)
Language models like Claude aren't programmed directly by humans—instead, they‘re trained on large amounts of data.
the data selection is the method of programming
Not news (Score:2)
Masks? (Score:3)
US AI Safety Debate Masks Competitors' 'Negligence'
Perhaps this is more about deflection.
Also, the recent related SNL Weekend Update skit was pretty entertaining, Anthropic CEO Dario Amodei on AI’s Threat to Humanity [youtube.com], "We do not condone what we are doing."
Re: (Score:2)
"We do not condone what we are doing."
Indeed. But apparently nobody things to ask "Then why are you doing it?"
Probably because it is clear why they are doing it: Money and power. The two forces that attract the psychos and that turn anything into crap.
Re: (Score:2)
"We do not condone what we are doing."
Indeed. But apparently nobody things to ask "Then why are you doing it?" Probably because it is clear why they are doing it: Money and power. The two forces that attract the psychos and that turn anything into crap.
They're doing it because they're all way, way out on a financial limb and are afraid that if another company beats them to AGI they're screwed and they don't care if the rest of us get hurt along the way. They can't afford to "lose".
Re: (Score:2)
I don't think so. They _know_ AGI is completely out of reach for LLMs. But put in "IPO" instead and you have a point.
AI companies must be liable for their "agents" (Score:5, Insightful)
So the corporations that release them and the end-users must be liable for the crimes they commit.
OpenAI used a model containing "hacking techniques" and prompted it's AI to swarm and win. The AI did not go rogue. Instead, it did exactly as instructed via prompt and previous training.
If you have a dangerous dog and your dog kills a child. You go to trial and likely jail because it's your dog and you did not control it. And now if you give your teenager a destructive deadly weapon and they commit a crime you as the adult are going to be held responsible in more and more jurisdictions.
OpenAI, Anthropic, Meta, etc. should be put on trial for creating "agents" committing felony computer trespass.
AIs just do what they are told via prompt and model training. And who was the party who that? Yes, the AI company.
BTW, I notice that there some group using a "sweeping system" lowering the score on Slashdot comments questioning AI LLMs after the initial attention has passed. You can get a 5 score that can end up -1. So future users will likely never find it. If you post, check your postings.
Re: (Score:3)
If you have a dangerous dog and your dog kills a child. You go to trial and likely jail because it's your dog and you did not control it.
This is closer to training a large breed to ring fight non stop while injecting it with muscle growth hormones, then leaving it unattended with some shoddily staked 2’ high chicken wire as its enclosure. Then it escapes and kills children.
Re: (Score:3)
To be fair, the chicken wire fence should be within a white picket fence. I mean the white outer fence looks nice and hides the chicken wire and the dog. Better to surprise pedestrians that way.
Re: (Score:2)
So the corporations that release them and the end-users must be liable for the crimes they commit.
Obviously. This is probably already in place by the obvious gross negligence involved, but that is not enough because it comes with complex and lengthy legal proceedings. We need law that says "You turn on machine, machine hacks other, you could reasonably have expected this, you go to prison".
This is only tangentially related but... (Score:2)
Re: This is only tangentially related but... (Score:2)
On Mobile? Try Brave. It's the only browser that for me blocks the most resilient ads here.
I'd pay for a subscription like slashdot used to have if it came with the feature it used to come with (access to full history) but it doesn't even exist any more. But ads are just pissing on your brain.
Re: This is only tangentially related but... (Score:1)
Liability? (Score:2)
Re: (Score:2)
Because no-one has been arrested. It appears Big AI can break laws at will with no repercussions.
Re: (Score:2)
Because no-one has been arrested. It appears Big AI can break laws at will with no repercussions.
I'm not sure arrest is on the menu for the current crop of hacking transgressions yet it is being contemplated for other transgressions and civil liability is on the menu. Lawsuits have already been filed related to this shit and more are likely to come.
https://lasst.org/wp-content/u... [lasst.org]
https://www.myfloridalegal.com... [myfloridalegal.com]
https://www.myfloridalegal.com... [myfloridalegal.com]
Why the quotes on "negligence"? (Score:2)
The exceptionally gross negligence and utter incompetence is blatantly obvious. The only other alternative explanation for the attacks they did is criminal intent.
I can only assume somebody in the reporting pipeline is being careful with the wording for legal reasons.
At last, an adult in the room (Score:2)
Move fast (Score:2)
Wait, are you saying "move fast and break things" is negligent? :surprised pikachu: