'Mistral is Peanuts For Us': Meta Execs Obsessed Over Beating OpenAI's GPT-4 Internally, Court Filings Reveal

Executives and researchers leading Meta's AI efforts obsessed over beating OpenAI's GPT-4 model while developing Llama 3, according to internal messages unsealed by a court in one of the company's ongoing AI copyright cases, Kadrey v. Meta. From a report: "Honestly... Our goal needs to be GPT-4," said Meta's VP of Generative AI, Ahmad Al-Dahle, in an October 2023 message to Meta researcher Hugo Touvron. "We have 64k GPUs coming! We need to learn how to build frontier and win this race."

Though Meta releases open AI models, the company's AI leaders were far more focused on beating competitors that don't typically release their model's weights, like Anthropic and OpenAI, and instead gate them behind an API. Meta's execs and researchers held up Anthropic's Claude and OpenAI's GPT-4 as a gold standard to work toward. The French AI startup Mistral, one of the biggest open competitors to Meta, was mentioned several times in the internal messages, but the tone was dismissive. "Mistral is peanuts for us," Al-Dahle said in a message. "We should be able to do better," he said later.

Turnabout is fair play

[jenningsthecat]

This time around it's the Llama that wants to whip somebody else's ass.

Thank goodness

[timeOday]

Mistral is alright but Llama really is the best set of open-weights models out there. I don't really know why they give it away.

GPT-4 is rapidly evolving into a different thing that's more than an LLM, with tie-ins to lots of different information sources, access to sandboxed phython runtime to calculate answers, a 'projects' construct for group discussions with related people and documents, and so on...

Re:Thank goodness

[Shaitan]

"I don't really know why they give it away."

It's the only ethical thing to do for humanity... I say screw copyright, let them eat everything but also leave the weights fully open, all the functional tuning but without any censorship or guardrails applied whatsoever. If they also want to host it and build a service around that or make a more business friendly version with censorship/guardrails applied to that open core there is absolutely nothing wrong with that.

Let's not forget, OpenAI was founded and created to do this as an open effort as well, they produced something with commercial potential and instantly screwed their funders and the world.

Re:

[93 Escort Wagon]

It's the only ethical thing to do for humanity...

Okay, but we're talking about Meta/Facebook - so that seems unlikely to be even a minor consideration.

Re:

[DamnOregonian]

You would think- but llama is wide open, and llama3.3 is the baddest LLM you can run locally.
I don't know exactly what to make of it, either.

Re:

[AnthonyCastanza]

I mean, I think what it comes down to is two things: 1) They weren't first 2) They aren't best So, they needed a niche that they can stake a claim to that will drive people to choose their product specifically. Being the definitive cornerstone of the "open" landscape of a groundbreaking piece of tech is a pretty damn good choice. The analogy is probably outdated, but they couldn't be McDonalds, so they chose to be Burger King.

Re:

[DamnOregonian]

Seems plausible enough.
For my part, I appreciate it. I've made a lot of use of those weights.

Re:

[DamnOregonian]

Ya, Llama3.3-70B eats Mistral for breakfast.
I use it in all of my ollama-connected applications.

Rumors

[93 Escort Wagon]

Internally, OpenAI refers to its own model as "Winamp".

Fantastic

[Shaitan]

The advances Meta has brought to AR/VR and AI are big steps toward making up for the evils of data collection. Not to mention moving toward a community notes type system and helping empower people with free speech and truth rather than the 'truth cartel.'

If you are going to steal my data, using it to make sure nobody owns AI is something worth doing that for.

FB

[ThurstonMoore]

I am beginning to feel that FB is not long for this world.

Interesting choice of words.

[Petersko]

"Mistral is peanuts for us," Al-Dahle said in a message. "We should be able to do better," he said later.

This implies they have less than peanuts.

Re:

[DamnOregonian]

It can. It can also (though definitely odd word choice) imply that they're better, and should aim for better.
Since Llama3.3 is objectively better than Mistral at just about everything, I'd say that was probably the intended meaning.

What's wrong with benchmarking?

[Petr Blazek]

TFA is about s copyright lawsuit, which is interesting. The fact that someone benchmarks their product against the leader is boring...