Tech Pundit Cringely Co-Founds Startup '2Brains Inc' to Solve LLM Hallucinations (cringely.com) 56
Long-time tech pundit Robert Cringely started his career at the Stanford Artificial Intelligence Lab back in 1978. Last month 73-year-old Cringely explained why his site went on a two-year hiatus — and it's not just because of a heart attack and a stroke last July:
Just like everyone else, I've been busy all this time on Artificial Intelligence, founding with two partners a company called 2Brains... The work we were doing together is unfinished, but it's not stopped. The patents are filed, the architecture is documented, and the small team continuing the work includes me.
Cringely's first piece made the cast that "the trillion-dollar bet the AI industry is making right now may be wrong, and that there's an architectural alternative we've patented and built." In Machines of Loving Grace, Amodei made the case that scaling compute would eventually solve essentially every hard problem in artificial intelligence. Buried in that optimism — or maybe not buried, maybe right out in the open — was a quiet absolution. Hallucinations, the embarrassing tendency of these systems to state falsehoods with total confidence, would take care of themselves. Make the models big enough, train them long enough, and the problem dissolves. You don't have to solve it. You just have to wait, and spend. And so the entire AI industry breathed a sigh of relief.
I have spent forty years watching this industry, and I know a permission slip when I see one.
Because that is what the essay became, whatever Amodei intended. It gave every other person writing nine- and ten-figure checks a reason not to worry about the one thing that should worry them most. The hallucination problem is the difference between a clever toy and a system a hospital or a bank or a court can actually rely on. It is the whole ballgame for enterprise AI. And the prevailing wisdom, blessed from the top, is that you needn't address it directly. Scale will provide...
A small company I helped start, 2Brains Inc., set out in 2022 to solve hallucinations — before ChatGPT, before the scaling consensus hardened into received truth, back when the polite assumption was that the problem was simply insurmountable. We did not solve it by waiting for bigger models. We solved it architecturally, by separating the part of the system that generates language from the part that retrieves and verifies facts, and reconciling the two before anything reaches the user. It runs on ordinary processors. It is cheap. And on the industry's own benchmark for this kind of faithfulness, it more than doubles the published baseline, with no fabricated facts in the verified case at all.
The article asks whether scaling will, at tremendous cost, eventually reduce hallucinations — or even worse, if the largest companies in the world "are spending a fortune chasing a cure that is not coming."
And last week Cringely pitched more advantages for their solution, noting that most prompts aren't even chatbot-level creative prompts — but just requests to retrieve simple data: The reason 2Brains doesn't lie and the reason it's cheap are the same reason. It looks the fact up instead of guessing it — so it cannot fabricate, and the lookup runs on a processor that sips power instead of a chip that gulps it. Trust and thrift are not a trade-off you balance against each other. They fall out of a single design decision. You do not pay extra for the honest version. The honest version is the cheap version. That sentence is the whole company.
Cringely's first piece made the cast that "the trillion-dollar bet the AI industry is making right now may be wrong, and that there's an architectural alternative we've patented and built." In Machines of Loving Grace, Amodei made the case that scaling compute would eventually solve essentially every hard problem in artificial intelligence. Buried in that optimism — or maybe not buried, maybe right out in the open — was a quiet absolution. Hallucinations, the embarrassing tendency of these systems to state falsehoods with total confidence, would take care of themselves. Make the models big enough, train them long enough, and the problem dissolves. You don't have to solve it. You just have to wait, and spend. And so the entire AI industry breathed a sigh of relief.
I have spent forty years watching this industry, and I know a permission slip when I see one.
Because that is what the essay became, whatever Amodei intended. It gave every other person writing nine- and ten-figure checks a reason not to worry about the one thing that should worry them most. The hallucination problem is the difference between a clever toy and a system a hospital or a bank or a court can actually rely on. It is the whole ballgame for enterprise AI. And the prevailing wisdom, blessed from the top, is that you needn't address it directly. Scale will provide...
A small company I helped start, 2Brains Inc., set out in 2022 to solve hallucinations — before ChatGPT, before the scaling consensus hardened into received truth, back when the polite assumption was that the problem was simply insurmountable. We did not solve it by waiting for bigger models. We solved it architecturally, by separating the part of the system that generates language from the part that retrieves and verifies facts, and reconciling the two before anything reaches the user. It runs on ordinary processors. It is cheap. And on the industry's own benchmark for this kind of faithfulness, it more than doubles the published baseline, with no fabricated facts in the verified case at all.
The article asks whether scaling will, at tremendous cost, eventually reduce hallucinations — or even worse, if the largest companies in the world "are spending a fortune chasing a cure that is not coming."
And last week Cringely pitched more advantages for their solution, noting that most prompts aren't even chatbot-level creative prompts — but just requests to retrieve simple data: The reason 2Brains doesn't lie and the reason it's cheap are the same reason. It looks the fact up instead of guessing it — so it cannot fabricate, and the lookup runs on a processor that sips power instead of a chip that gulps it. Trust and thrift are not a trade-off you balance against each other. They fall out of a single design decision. You do not pay extra for the honest version. The honest version is the cheap version. That sentence is the whole company.
So... (Score:2)
... they've put a language interface on top of a standard search engine or database? I'm pretty sure thats not a new idea.
Re: So... (Score:2)
Isn't Robert X. Cringely a pseudonym? (Score:3)
I don't have the exact link, but I remember reading that more than one person wrote the column under that name
Re:Isn't Robert X. Cringely a pseudonym? (Score:5, Interesting)
That's what I remember too. There was a real Cringely at first, but somehow he ended up signing away his name in the context of the column and then it was done by the magazine staff. (memo: read the fine print)
I ran headlong into what we now call hallucinations in 1996 working on my Ph.D. on process control using neural networks. I decided it wasn't going work for real-word real-time control (and the committee agreed). I've been very amused by this whole AI rush.
As the saying goes, "It's human to err, but it takes a computer to really screw things up."
Re: Isn't Robert X. Cringely a pseudonym? (Score:3)
Except neural networks currently control the lane keep assist in modern cars, never mind self drive ones. So clearly they now do work for process control.
Re: Isn't Robert X. Cringely a pseudonym? (Score:5, Informative)
https://www.wesh.com/article/w... [wesh.com]
That was easily refuted.
Re: (Score:3)
"Sorry, this content is not available in your region."
Very useful.
Re: (Score:2)
WESH2 News:
Brandon Hogan
MOUNTAIN VIEW, Calif. â"
A recall submitted by Waymo this week to the National Highway Traffic Safety Administration states the automated driving system in nearly 4,000 of the company's driverless vehicles may have caused some of the cars to "enter and drive at speed in freeway construction zones."
The recall centers on the 5th Generation Automated Driving System in 3,871 of Waymo's vehicles, listing the following examples from April and May in which the automated cars allegedly
Re: Isn't Robert X. Cringely a pseudonym? (Score:3)
Re: (Score:2)
"I ran headlong into what we now call hallucinations in 1996..."
Seems highly unlikely. "what we now call hallucinations" is a 21st century phenomenon, a decade later. "what we now call hallucinations" is an LLM failure mode, LLMs first appeared two decades later.
I wrote software with bugs back in the 80s, perhaps I have your hallucination claim beat by a decade, given that the term can mean anything.
Re: (Score:3)
The control system responded to a combination of inputs it had never seen before that resulted in a low pH condition by increasing acid flow to lower it further.
That is close enough for me. All those weights between layers result in a non-linear failure mode which is not acceptable in the real world, rather like Elon's self-driving cars ramming firetrucks.
As Cringley points out, all they've done is throw horsepower at it. It's like the old argument in physics, if we knew every particle's position and veloci
Re: (Score:3)
what?
The term 'hallucinations' in this context is cope to deal with a fundamental feature of language based associative decisions systems that we've known about since the 1970s, in math if not in practice.
I was running MegaHAL on IRC in 1998 and it sure as hell 'hallucinated'.
Re: (Score:2)
AI hallucinations are not a new observation, even if people with no understanding of computing history think so.
Re: (Score:2)
I ran headlong into what we now call hallucinations in 1996 working on my Ph.D. on process control using neural networks. I decided it wasn't going work for real-word real-time control (and the committee agreed).
Probably not much of a surprise, but always worthwhile to verify.
I've been very amused by this whole AI rush.
As the saying goes, "It's human to err, but it takes a computer to really screw things up."
Indeed. I mean I did a few ChatGPT queries back when it came out and found its use a waste of time. I see the same level of incapability today in the AI summaries for Internet search and I am mostly ignoring these now. Occasionally I check and just find "still incapable". And while the paid models will surely do better, they will still not do well.
Re: (Score:3)
The original guy got to keep using it. There was someone else hired for a brief time.
I remember the author's name but he really doesn't want to use it, so that's OK to respect. He's given me a lot to think about over the years. I remember when he wrote on his PBS site about unicast becoming cheaper than radio broadcast for TV, predicting that it would overtake by 2012 (IIRC). Youtube became huge around then. We were smart folks around the water cooler in the late 90's who could follow the math but had nagg
Re:Isn't Robert X. Cringely a pseudonym? (Score:5, Interesting)
I don't have the exact link, but I remember reading that more than one person wrote the column under that name
A Brief History of Robert X. Cringely
(Tech Pundit? LOL. Nope.)
In 1987, Mark Stephens was hired by Infoworld magazine where he began writing under the name Robert X. Cringely. When he left Infoworld in 1995, Stephens continued using the Cringely name and Infoworld sued him. They eventually reached an agreement where he was allowed to continue using the Cringely name as long as he wasn't working for a competitor of Infoworld.
For many years Mark Stephens has claimed that he is "the original Robert X. Cringely". But he isn't. Before he was hired by Infoworld there were at least two other people who wrote columns using the Cringely pseudonym.
At various points in his career, he has also claimed that he was employee number 12 at Apple, he helped them move out of Steve Jobs' garage, and he designed the original Mac trash can icon. There is no credible evidence that any of this is actually true.
In 2015 Cringely announced "The Mineserver Project" on Kickstarter. These miniature Minecraft servers would be small, inexpensive ARM-based boards, running Linux, slightly more powerful than a Raspberry Pi and selling for $99. The project raised $35,000 and the finished boards were supposed to ship in December 2015. But they didn't.
All through 2016 Cringely repeatedly promised that the Mineserver boards would be finished and shipped "soon". But there was always "one more little problem" that was holding things up. In November 2016 Cringely wrote on his Kickstarter page: "We'll finally start shipping the week after Thanksgiving. Thanks for your patience and support."
Nothing was ever shipped, and there were no more updates posted to the Kickstarter project. Ever.
In July 2017 Cringely posted on his blog that he was suddenly blind from cataracts, but he would have his sight restored in a couple of weeks, so maybe everyone could stop asking about the Mineserver boards until then. Three months later, Cringely claimed that his house burned down and all the Mineserver boards were destroyed. Like everything else with Cringely, none of these stories can be confirmed with any credible evidence.
In May 2018 Cringely wrote that he was preparing a new model of the Mineserver boards because the available parts have all changed and he promised that every supporter will "get their Mineserver before the end of the year." But 2018 ended with nothing.
In June 2019, Cringely posted his thoughts on "The Future of Television", with no mention of the Mineservers at all, and he didn't post anything to his blog for the rest of 2019, although he was still posting about airplane trivia on Quora.com.
In January 2020, a new tall tale hit Cringely's blog when he announced his new business venture called Eldorado Space. This would be a company using F-104 jets to launch satellites. Cringely says revenue from this business will fund his retirement (he was 67 at the time) and give him enough money to finally deliver those Minecraft servers he's been promising for the last 5 years.
He also claimed that the business is guaranteed to succeed because his new company has bought all the F-104s in existence, so he won't have any competition. (Wikipedia says that there are only four airworthy F-104s in existence). To prove that this is all real and legitimate, Cringely found a picture of an F-104 and photoshopped the word Eldorado onto it.
There has been no further mention of Eldorado Space since the original blog entry, and Cringely still hasn't paid back the $35,000 he stole from people for the Mineserver project.
"Never Meet Your Heroes"? (Score:2)
Wow. Thanks for posting this, A.C.. In trying to verify any of what you posted (which was all news to me), I found this:
"The cost of lies: A Mineserver story" by Jeremy Reimer
https://jeremyreimer.com/rocke... [jeremyreimer.com]
"Creating and shipping a brand new product is insanely difficult. It takes a ton of money, sweat, and time. Even people with tons of experience can underestimate timelines and encounter unexpected difficulties. So telling the story of a failed Kickstarter is not especially interest
On AI design and also irony (Score:2)
I just wanted to add that whatever the truth there, this idea that LLMs are not (by themselves) the way forward is increasingly appearing in various places. One recent example on Slashdot:
https://slashdot.org/story/25/... [slashdot.org]
"Project Prometheus is building AI systems that learn from physical experiments rather than just analyzing digital text."
Humans learn to speak usefully with just a few years of immersion in a social world and without reading the entire internet. My college advisor back in the 1980s (George
Re: (Score:2)
It has seemed to me, for a very long time, that modern AI systems would need to be integrated with standard RDBMS systems for reliable persistant storage of raw information, some sort of no-sql database (memcache or some variant) for persistant storage of associations, some sort of document database for blocks of textual information, a SPARQL system for searching semantically-marked information within the document database, and a more old-fashioned back-propogation NN to provide a store of understanding tha
Re: (Score:2)
I think it's just whoever bothers to show up. Like Journey.
Ya, but ... not sure two is better than one. (Score:5, Interesting)
Co-Founds Startup '2Brains Inc' to Solve LLM Hallucinations
Makes me think of that saying, "A man with one watch knows the time, a man with two is never sure."
Re: Ya, but ... not sure two is better than one. (Score:3)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Did you notice they've put both hands in: A left-hand 'pile' under a right-hand 'pile'?
Re: (Score:2)
Re: (Score:2)
Um, the point of that saying is that two is better than one. It's saying ignorance leads to bad decisions because you don't know what you don't know. AKA, known unknowns is better than unknown unknowns.
Yeah..... (Score:2)
Re:Yeah..... (Score:4, Interesting)
You don't know what the alleged patents are, or whether they are granted rather than just filed. If "everyone already does this", where this is what is claimed in the patent, then there will be documentation. If there is documentation, the patent will not be granted. It's not magic.
Re: Yeah..... (Score:2)
Re: (Score:2)
If "everyone already does this", where this is what is claimed in the patent, then there will be documentation. If there is documentation, the patent will not be granted.
Clearly you don't have much experience with the USPTO.
Re: (Score:3)
Re: Yeah..... (Score:2)
not AI then (Score:2)
"The reason 2Brains doesn't lie and the reason it's cheap are the same reason. It looks the fact up instead of guessing it ..."
Then it is constrained by what it can look up. A search engine with a natural language interface, not AI.
The reason humans do not lie (except when they do) is because they have values, not because the world is a multiple choice test with a cheat sheet. Humans can show their work, this doesn't even do work, it isn't a solution to anything and a proper solution obviates the need. W
Re:not AI then (Score:4)
This is the problem. The practical uses cases as I understand them pretty much fall into the same buckets we use NLP for now. If you don't want to use LLM or GenAI technology we already have a lot of really great ML/NLP tools that do a really really good job.
In fact a lot of these tools would do a better (or at least more reliable) job of about 70% of what I see companies deploying in the customer service chat bot space, they'd be much cheaper and faster too. I have tried to explain to several clients, "You know you could do all this with Google DialogFlow" but no they'd rather wank around building MCP/SEE/Agenic replacements for the REST services they already have, futz around with prompt design, and then figure out how to test for abuse cases all so they can pay for tokens..
By they time you chain down Gemini/CoPilot/GTP down to respond in corporate approved ways half of customers could not tell the difference anyway and most would probably enjoy an experience that is consistent focused and quick.
And so it seems to go with 2Brains here, seems like an expensive and complicated way to do things we have been able to do well with NLP for 15 years now. Using LLM at scale means an expensive and complicated pile of machinery, but what is attractive about using them places where they are not really needed is "Its what all the cool kids are doing" not the expensive and complex part... Good luck 2brains...
Re: (Score:2)
Yes. But most people do not understand NLP. They do understand "magic intelligent box", or think they do.
Re: (Score:2)
Enterprises need to wise up, these tools aren't being developed to do a good job but to make billionaires richer.
Obviously. Look at Musk's porn generator, for example, that does not mind undressing anybody underage, even when that is clearly illegal. Or at Googles AI summaries that may hallucinate stuff to the detriment of enterprises or people. This is a short-term money-grab and they do not mind if it all comes crashing down in a few years or tomorrow.
interest to hear how they define the tools. (Score:2)
I think I still agree about what this article says about "user interfaces", even if the word user has lost the connotation of 'human' it used to have.
But if you are an actual flow user who actually needs to get something done, WA could give you an alternative, manual interface for selecting your tool. You might perform the discovery task by browsing, say, a good old-fashioned menu. For example, the Nutrition Facts tool might come with its own URL, which you could bookmark and navigate to directly. There might even be a special form for entering your recipe. Yes, I know none of this is very high-tech. (Obviously the coolest thing would be a true command lineâ"but the command line is truly not for all.)
https://www.unqualified-reserv... [unqualifie...ations.org]
btw, I am astonished that there has been almost no progress in designing interfaces to be used by programmers.
Might work on the easy problems (Score:2)
This might work when there is a simple, easy search that can verify a fact. But that's often not the case. In my experience most cases of hallucination are cases where the LLM needs a fact mid-response, and the fact check requires both a non-trivial query and complex evaluation of the response data, sometimes involving judgement calls. When that happens, the LLM just gets lazy and goes with its guess rather than doing the check.
I'm speaking in the context of advanced models, mind, not the kind of thing
Re: Might work on the easy problems (Score:2)
I was feeding some graphs of my PV production to claude.ai . There was clipping - a flat line near the top - and i was trying to get it to tell me how much more production i would get if the inverter was bigger. It kept insisting there was no flat line or clipping going on, making up the peak number. Confidently. Even after multiple prompts, including some as "are you blind". There are things these tools are just terrible at. Reading graphs is one of them, apparently. I also tried to get it to create a diag
Re: (Score:2)
It's almost as if we need some non-generative AI to actually get the job done.
There is no AI tech that can do what GenAI is missing. All we have is actual human intelligence, but that requires expert-level capabilities.
Re: Might work on the easy problems (Score:2)
The LLMs i have been using clearly demonstrate some expert level capabilities. The problem is that they also fail some basic elementary school tasks once in a while. Reviewing the entirety of their voluminous output to screen those out is not humanly feasible.
Re: (Score:2)
No, they do not demonstrate any "expert level capabilities". Experts always understand the "why", not just the "how". LLMs are incapable of that.
I do agree that review effort for LLM output is probably what kills their use in most application scenarios.
Re: (Score:2)
Agreed. I get like 30% hallucinations in the AI responses to stuff I ask search engines now. But I generally do not ask simple stuff. The other problem is that for simple stuff, you do not need AI and its use will only have significant negative impact on your mental processes. And for advanced stuff (where you actually need to know and understand the simple stuff to verify the answer), hallucinations become pretty likely and so does missing context and caveats in the answer.
I do not see how this tech can be
Looking the facts up when the facts might be wrong (Score:2)
or intentionally manipulated by someone.
#911_was_an_inside_job
Are hallucinations even a problem? (Score:2)
I mean dumb humans are confidently wrong (hallucinating) all the time and especially humans in managerial positions he's positing this should replace.
Although LLMs hallucinations are driven by biases alien to humans.
This seems like a quick cash grab aimed at AI antis of Ed Zitron ilk, who famously insist that "AI is real only if it is never wrong and never hallucinates", but they never provide any insight why the AI needs to be always 100% correct.
Meanwhile, the real problem is that LLMs simply descend into
Re: (Score:2)
Hallucination are a severe problem whenever the damage is large. For example, driving a car, controlling an industrial process or writing software that will be exposed to the Internet. The problem is, humans generally do not hallucinate (unless they are managers or politicians or cult leaders and these all can and often do extreme damage), they are just somewhat off bust still mostly where they should be. Even a drunk driver is basically driving fine, just with reduced reflexes and capabilities. That is not
Re: (Score:2)
> driving a car, industrial control, vibe coding are bottlenecked by hallucinations
No they're not, this isn't 2022 anymore. Let me reiterate, they're fairly trivial to spot and largely accounted for in serious autonomous systems. The issue is monkey paw property of the LLM - they're maliciously compliant by seeking shortcuts too greedily while skipping over subtler ontologies that aren't verbalized in corpus enough so as to differentiate into a concept ("carwash is typically somewhere you might want to d
Re: (Score:2)
What posting are you replying to? Because it is not mine...
Bottom line, 2Brains is unfinished. (Score:2)
Exponential effort (Score:2)
That is why scaling will not solve things. Effort grows exponentially bot in training data needed and effort of training the model. Hence, forget it. I am constantly surprises that nobody but the actual experts understand what exponential effort means. Incidentally, this is the same problem that will make sure QCs never reach useful sizes.
Unlikely (Score:2)
LLMs don't hallucinate sometimes (Score:2)
LLMs hallucinate 100% of the time. More often than not the hallucination is in the shape of the correct answer. Sometimes it isn't. But you can't stop LLMs from hallucinating ... because everything is a hallucination.
All you can do is keep trying to find more edge cases and writing rules against them.
Which is basically computer programming again but with 1000x the resource cost and none of the determinism.
So far, so what (Score:2)
Sounds interesting, but nothing concrete yet.
Wake me up when there's an actual interface that I can ask 3 questions of (like a genie with 3 wishes) and maybe a local instance I can run in virtualization.