After Meta Cheating Allegations, 'Unmodified' Llama 4 Maverick Model Tested - Ranks #32 (neowin.net) 17
Remember how last weekend Meta claimed its "Maverick" AI model (in the newly-released Llama-4 series) beat GPT-4o and Gemini Flash 2 "on all benchmarks... This thing is a beast."
And then how within a day several AI researchers pointed out that even Meta's own announcement admitted the Maverick tested on LM Arena was an "experimental chat version," as TechCrunch pointed out. ("As we've written about before, for various reasons, LM Arena has never been the most reliable measure of an AI model's performance. But AI companies generally haven't customized or otherwise fine-tuned their models to score better on LM Arena — or haven't admitted to doing so, at least.")
Friday TechCrunch on what happened when LMArena tested the unmodified release version of Maverick (Llama-4-Maverick-17B-128E-Instruct).
It ranked 32nd.
"For the record, older models like Claude 3.5 Sonnet, released last June, and Gemini-1.5-Pro-002, released last September, rank higher," notes the tech site Neowin.
And then how within a day several AI researchers pointed out that even Meta's own announcement admitted the Maverick tested on LM Arena was an "experimental chat version," as TechCrunch pointed out. ("As we've written about before, for various reasons, LM Arena has never been the most reliable measure of an AI model's performance. But AI companies generally haven't customized or otherwise fine-tuned their models to score better on LM Arena — or haven't admitted to doing so, at least.")
Friday TechCrunch on what happened when LMArena tested the unmodified release version of Maverick (Llama-4-Maverick-17B-128E-Instruct).
It ranked 32nd.
"For the record, older models like Claude 3.5 Sonnet, released last June, and Gemini-1.5-Pro-002, released last September, rank higher," notes the tech site Neowin.
Re: (Score:3)
Re: (Score:3)
Oh come on (Score:4, Funny)
There's no way Zuckerberg's company would lie to us!
I don't think it would be a good idea (Score:2)
You can rip off as many little old ladies as you want and if you can come up with an AI scam that only rips off a little ladies of their life savings go right ahead I guess, Lord knows for the next 4 years is going to be absolutely no law enforcement around those scams, but God help you if you rip off one of the rich people you
Re: I don't think it would be a good idea (Score:2)
Cold war (Score:1)
That's all it is. Counted in the megawatts of energy spent to train completely useless models.
And yet... (Score:1)
Oblig. Milo Murphy Reference (Score:2)
Llama, you're the bomba! [youtube.com]
Over-fitting... (Score:2)
*clap* *clap* *clapclapclap*
Over-fitting...
*clap* *clap* *clapclapclap*
Re: (Score:2)
-1 Trolling
Tall and Leggy (Score:2)
Did Meta remember to add legs to their models this time? Some products are better with legs.