Claude, Microsoft Copilot Fail Again to Predict the Winners of the Kentucky Derby (yahoo.com) 57
In 2016 an online "swarm intelligence" platform generated a correct prediction for the Kentucky Derby — naming all four top finishers in order. (But its 2017 predictions weren't even close.) Slashdot checked in again on how modern AI systems performed in 2023, 2024, and 2025 — but their predictions were still pretty bad. Would AI-generated Derby predictions be any better in 2026?
This year's winner was 24-to-1 longshot "Golden Tempo" — though a lot of oddsmakers had favored a horse named Further Ado (which ultimately only finished 11th). So when USA Today prompted Microsoft Copilot for its own picks for the Kentucky Derby, Copilot also went with Further Ado. (Even worse, it predicted Golden Tempo would come in... 13th.)
Here's how Copilot's picks actually performed...
This year's winner was 24-to-1 longshot "Golden Tempo" — though a lot of oddsmakers had favored a horse named Further Ado (which ultimately only finished 11th). So when USA Today prompted Microsoft Copilot for its own picks for the Kentucky Derby, Copilot also went with Further Ado. (Even worse, it predicted Golden Tempo would come in... 13th.)
Here's how Copilot's picks actually performed...
- Further Ado (finished 11th)
- Chief Wallabee (finished 4th)
- The Puma (SCRATCHED)
- Renegade (finished 2nd)
- Commandment (finished 7th)
- So Happy (finished 9th)
- Emerging Market (finished 10th)
- Danon Bourbon (finished 5th)
- Potente (finished 12th)
- Incredibolt (finished 6th)
- Robusta (finished 14th)
- Ocelli (finished 3rd)
- Golden Tempo (finished 1st)
- Pavlovian (finished 18th)
- Great White (SCRATCHED)
- Wonder Dean (finished 8th)
- Litmus Test (finished 17th)
- Albus (finished 15th)
- Six Speed (finished 13th)
- Intrepido (finished 16th)
Copilot was told to use the latest odds, conditions, and analysis of favorites, best bets, expert picks, previous results and race history with the post positions, according to USA Today.
And meanwhile, Yahoo Sports asked Claude "to simulate the race using the opening odds, draw and potential track conditions. We also asked it to factor in some human predictions."
Like Microsoft Copilot, Claude also picked Further Ado to finish first (though it came in 11th) — and predicted that Golden Tempo (the eventual first-place finisher) would finish 12th.
- Further Ado (finished 11th)
- The Puma (SCRATCHED)
- Commandment (finished 7th)
- Chief Wallabee (finished 4th)
- Renegade (finished 2nd)
- Emerging Market (finished 10th)
- So Happy (finished 9th)
- Incredibolt (finished 6th)
- Danon Bourbon (finished 5th)
- Potente (finished 12th)
- Pavlovian (finished 18th)
- Golden Tempo (finished 1st)
- Litmus Test (finished 17th)
- Albus (finished 15th)
- Wonder Dean (finished 8th)
- Six Speed (finished 13th)
- Intrepido (finished 16th)
Just lucky? (Score:3)
I'm curious how the 2016 platform managed to get it so right but everything since then has failed. Would be amusing if models are actually getting worse as they are hyped more in business.
Re: (Score:2, Funny)
greatest human made disaster (Score:2)
I wonder if people realize that AI is going to ruin gambling, even if it never manages to predict accurately. But AI is going to ruin everything. The economy, music, television, relationships, etc.
Re: (Score:2)
I think that's a weird take away, because the top 6 billionaires are not casino owners, they're neck deep in AI hype.
Re:Just lucky? [But why not Incredicolt?] (Score:2)
Like the joke, but the mutated Subject about the horse named Incredibolt was the joke I was looking for.
But seriously, folks, I wonder how much difference the name makes. I doubt the horse cares, but the humans are affected. Reminds me of a book about the people who live by training race horses, but it was in Japanese so I can't even share the title on Unicode-less Slashdot.
On the other hand, taking the question seriously, I count the results as evidence that "luck" is more important than "data" and my conc
Re: (Score:3)
The gamblers are absolutely known to to be losing on average, because the way parimutuel betting works is that they put all the bets in a pile, take out the house percentage -- which pays for just about everything in racing except sponsored races -- and distribute the rest to the winners. And that take is not small; it's classically 17% for win/place/show betting and 25%-30% for exotics (top finishers in order or winners of multiple consecutive races).
Re: (Score:2)
I'm familiar with the basic idea, but it did get me to wondering about pathological cases. What if the bets on losing horses were so few and so small that the winnings went negative after the house percentage was taken? You bet on the winner but the return is less than your wager? That would suggest starting by putting aside the winners' wagers to make sure they don't lose money before taking the house cut off of the losing bets... Or some kind of escape clause where the race gets cancelled if there aren't
Re: (Score:2)
Ah! That's called a "negative pool", and when it happens, all winning bettors get a minimal positive payout, like $2.05 for a $2 bet. It almost never happens for "win" betting but you sometimes see it in the "show" pool (1st, 2nd, or 3rd place) for a big favorite. So that's a rare positive expectation. And people know this, and sometimes place huge bets to show on a big favorite, which gets you a few percent at a time until one loss wipes you out. It's called "bridgejumping", as in "if that horse doesn'
Re: (Score:2)
Sounds like you play the horses? Or is your interest purely a mathematical one? In that case, maybe you work for the house?
Re: (Score:2)
I do play some, used to more in the past.
Re: (Score:3)
You're looking at this the wrong way, the 2017 - and subsequent - platform(s) got it right, but the Las Vegas bookmakers either bribed the predicted top 4 jockeys to slow up a bit or they manipulated the top 4 predictions in the first place. Let's hear it for conspiracy theories!
Re: (Score:3, Insightful)
I'm curious how the 2016 platform managed to get it so right but everything since then has failed. Would be amusing if models are actually getting worse as they are hyped more in business.
Which logical fallacy is it that seeing a random-guesser be correct once makes one think it was anything more than a lucky guess and be more likely to trust that guesser to be correct in the future?
Re: Just lucky? (Score:2)
I think you can extrapolate from https://xkcd.com/882/ [xkcd.com] how this likely happened.
Re: (Score:3)
Re: (Score:3)
Probably. If you make enough predictions, somebody will get lucky. People that like to claim they can predict the stock-market work like that (for example): They make tons of predictions and if just one is right they claim they can reliably predict the future. Because most people cannot think systematically, they only see the correct prediction and ignore the mountain of failed ones.
Re: (Score:2)
Many different AI's picking many different sets of winners. One of them happening to get it right is kind of expected if there are enough AI guesses. The one that happens to get it right claims it knew what it was doing and was not simply lucky.
Oh really?! (Score:2)
It's almost as if you cannot predict the future with absolute certainty and sometimes you cannot predict it at all with any certainty.
Win rate and probability is something well researched and all the bookies already know you can use tea leaves, astrology, advanced statistical models, AI, ninjitsu the house always wins in the long run.
At best the AI models can come up with the same probability distribution as the bookies. Perhaps the
Re: (Score:2)
Re: (Score:2)
The really interesting thing is whether any of the "AIs" used added important context, like this being a guess and not reliable and that nobody should depend on it. My guess would be that most add this only when specifically prompted to do so (whether by the user or by "guardrails").
24-1 Longshot wins? (Score:2)
If the AI got it right, I would say the fix is in.
How does AI do predicting lottery numbers? (Score:5, Insightful)
Re: (Score:2)
People who bet on sports, roulette, whatever, for purposes other than pure entertainment have to believe that predicting the outcome is a matter of skill. If it is a matter of skill then a really good model should at least have the potential to do it well.
Re: (Score:2)
Re: (Score:2)
That's the entertainment. Not everyone has the same motivations. Lots of people believe they've "figured out a system" and can actually win. Occasionally they're right, but most of the time they're not.
Re: (Score:2)
People building systems that at least can predict a bit feed them with a lot more specific data than general purpose AIs have. Claude is as good as average joe in making a guess and may be lucky or not.
Re: (Score:2)
The predictions look quite a bit better than chance, and the summary says they're pretty much what the experts predicted.
Re: (Score:2)
Sure, they have to believe it. That doesn't make it true.
On things like blackjack you can sway the odds a couple of percent by doing it well, and the statistical odds assist you, On sports and horse racing knowledge matters more... but consequently there is machinery in place to keep those odds always slightly against you. Hence 24-1 longshot, or paying less than even for the favorite.
A great model still can't beat the house statistically, unless it uncovers something fundamentally amiss. The better it mode
Re: (Score:2)
The guy asked why they would want him to teach the players to win, potentially taking money from the casino. The response was that the edge he seemed to have exploited would be difficult to maintain. But if the players *think* they can win, they'll play more
Re: (Score:2)
Bookies aren't dumb. They may or may not have their own model for something, but they definitely take into account actual placed bets (i.e. everyone else's model) when calculating odds so they don't lose money. What you're really doing is playing your model against everyone else's, minus the house's cut.
Most people's model is "I like this horse" so they're not hard to beat. Online poker is a nice simple example of this. I had a buddy who used to play. He'd play a dozen games at a time with software keeping
Re: (Score:2)
Yes we do. Where we equals investors who spent billions on dubious AI companies who deliberately mislead said investors about the true capabilities to get their money.
Yes we do. Where yes implies there is an economic crash coming and we implies that it will affect you too, even if you didn't technically believe it.
There is a reason liars should ordinarily be punished. When they are not, bad things happen.
This kind of story is just a symptom about the sheer magnitude of the lies.
Re: (Score:2)
AI read that book from Back to the Future that has all of the outcomes in it.
Re: (Score:1)
I can build an "AI" that can predict lottery numbers. It is very easy: Just have it predict all of them, then, after the fact bury the failed predictions and hype the one that was correct into high heaven and claim magic (or godlike-powers). See, done.
Obviously this is completely meaningless, but this tactic works on many, many people. And it is essentially what the LLM peddlers have done the last few years: Hype the (often accidental) successes and bury the mountain of failures.
Re: (Score:2)
That some idiot moderated this down does not change the facts.
Re: (Score:2)
I just use regular autocomplete to get my winning lottery numbers. I am now a multi millionaire!
Asking the House. (Score:2)
Asking AI about the outcome is like asking The House for advice.
(The House) ”The hell do you think my job is? I’m not here to advise a winner. I’m here to addict a loser. I already know who’s ultimately going to win. By design.”
I'm Sorry (Score:2)
Who the fuck is stupid enough to think that an inference database can predict the future? It's hard enough to prevent it from manufacturing its own version of known facts.
Furthermore, how is AI's inability to do anything news worthy?
Re: (Score:2)
From available evidence, a lot of people are this stupid and this clueless about how reality works. For an estimate, look to the frequency of independent thinkers (10-15%, apparently), those that can be convinced by rational argument (a weaker form, at 20-30%, but for important questions it is more like 20%), the religious (around 80%), and you begin to see a picture.
Furthermore, how is AI's inability to do anything news worthy?
See above. Many people deeply believe (!) that AI can do anything.
Why would an LLM ... (Score:1)
It's not even the LLM predicting... (Score:3)
From the article...
Copilot was told to use the latest odds, conditions, and analysis of favorites, best bets, expert picks, previous results and race history with the post positions
Odds? Conditions? Analysis? Best bets? Expert picks? This isn't the LLM doing any work. It's an LLM being used as a calculator to sum the combined human-generated odds.
If you want to use an LLM to pick horse races, this is what you do. You let it watch every horse race that each current Kentucky Derby horse has ever raced in. You let it watch every sort of training session each horse is put in. You let it see the data from every veterinary treatment and inspection. You use every
Re: (Score:1)
Please stop trying to treat LLMs as something they are not.
I'm not - that was the point of my rhetorical question :)
Re: (Score:2)
Because LLMs can to anything! I was told so right here in the marketing materials!
I think what is happening is that stupid people have swallowed the lies and are now trying to put them into application to get rich. And it does not work, which is a big surprise to them. Reminds me of that YT video where some flat-earthers set out to prove the earth was flat with some lasers (?) and found the earth was round. Apparently they were very surprised too.
Not fair (Score:4, Insightful)
The LLMs weren't trained on a story about who won. How TF is it supposed to predict it in advance? That's not how LLMs work. Just train it on one more thing, a story about who won in 2026, and I'm sure you'll see its accuracy go up.
Meanwhile.... (Score:2)
Over on the 'Racing News' website you can read about Linux distributions and the Tiobe rankings.
Link doesn't work and geometric distribution (Score:2)
1) The link "Swarm Intelligence" (which is on the link to "generated a correct prediction" in this original post) doesn't open. Googled "Swarm intelligence kentucky derby" to find an explanation of how this model predicted the winner and didn't find any matching results. So, does anyone have a link (that works)?
2) If I were trying to predict the winner of the Kentucky Derby using data, I would think that predicting the winner would be nearly impossible. However, it might be possible to get a lower bound of
Re: (Score:2)
I think the only "reasonable" goal is (as you say) not to pick the winner, but to correctly compute the odds of each horse winning, and then only bet if you can get more than that. An extra obstacle here is the dynamic odds in paramutuel betting, where you don't know your final payout for sure. (There are markets where you can get fixed odds, but that's not the typical case for US racing.)
Re: (Score:2)
Huh, I've never even heard of parimutuel betting. I had to read the WIkipedia page on it. In that case, am not sure my suggestion of using the geometric distribution would work.
Also, yes, finding the gradient between the stated odds of a horse winning and the observed odds is a common method (to find irregularities in how odds are calculated). I had a colleague who, like me, has a Phd in math. Unlike me, he is (or was) athletic. He played tennis as often as he could. When sports betting became legalized, he
Multiverse theory (Score:2)
Easy to ridicule the AI (Score:2)
Easy to ridicule the AI - did humans fare any better?
Re: (Score:2)
Real gamblers avoid races like the derby (Score:2)
Bookies the world over breathe a sigh of relief. (Score:2)
Actually, probably not. They've been running fairly deep statistics on races for decades (primarily for detecting unusual patterns of betting, indicating illegal collusion between gamblers, and voiding the resultant bets. While keeping the stakes.