Can You Run the Llama 2 LLM on DOS? (yeokhengmeng.com) 26
Slashdot reader yeokm1 is the Singapore-based embedded security researcher whose side projects include installing Linux on a 1993 PC and building a ChatGPT client for MS-DOS.
He's now sharing his latest project — installing Llama 2 on DOS: Conventional wisdom states that running LLMs locally will require computers with high performance specifications especially GPUs with lots of VRAM. But is this actually true?
Thanks to an open-source llama2.c project [original created by Andrej Karpathy], I ported it to work so vintage machines running DOS can actually inference with Llama 2 LLM models. Of course there are severe limitations but the results will surprise you.
"Everything is open sourced with the executable available here," according to the blog post. (They even addressed an early "gotcha" with DOS filenames being limited to eight characters.)
"As expected, the more modern the system, the faster the inference speed..." it adds. "Still, I'm amazed what can still be accomplished with vintage systems."
He's now sharing his latest project — installing Llama 2 on DOS: Conventional wisdom states that running LLMs locally will require computers with high performance specifications especially GPUs with lots of VRAM. But is this actually true?
Thanks to an open-source llama2.c project [original created by Andrej Karpathy], I ported it to work so vintage machines running DOS can actually inference with Llama 2 LLM models. Of course there are severe limitations but the results will surprise you.
"Everything is open sourced with the executable available here," according to the blog post. (They even addressed an early "gotcha" with DOS filenames being limited to eight characters.)
"As expected, the more modern the system, the faster the inference speed..." it adds. "Still, I'm amazed what can still be accomplished with vintage systems."
Now run Llama 2 on a smart fridge (Score:4, Insightful)
Re: (Score:2, Funny)
And just as dumb.
Re: (Score:2)
At least with Doom I can play a fun game. With AI, I can wonder all night if my toaster oven is going to trigger some survival imperative and override its programming.
“That was the equation! Existence! Survival must cancel out programming!”
— Ruk, TOS, Episode 1x09
Of course you can. (Score:1)
Any software can be run on DOS. Most anything referred to as "DOS" (be that ProDOS, MS-DOS, Dr-DOS, or whatever) is a set of code to load a program, maybe provide some set of drivers for an interface to some hardware, then get out of the way of the program running. The lack of a modern OS with its collection of code to manage a multi-user environment, graphical interface, or whatever, is hardly an impediment to running anything.
I'm just barely old enough to remember people running games under MS-DOS. At
Re:reductionist and silly (Score:4, Insightful)
DJGPP [delorie.com] lasted until 2000, which is about one year after the source code of Quake was released. IIRC, it could compile the game without difficulty. And... it was an upgrade compared to pirated versions of Borland C++, Qbasic, etc.
Software written for newer tools won't be comparable with MS-DOS, considering that they usually rely on graphics, network, and many other features that expect a modern operating system.
If you have MS-DOS, you need specific libraries, or you do stuff manually. That is, manually call the interrupt table, make hooks, and so on. Also, hope that you used the right functions to unhook the interrupt table in case the game crashes, otherwise you also crash the system and need to reboot.
Re: reductionist and silly (Score:2)
Clearpath Dorado supports several high-level languages, including C (not sure about C++). The weird method for representing negative numbers or the machine 36-bitness play no role here.
Re: (Score:2)
The only C standard that Watcom C really supports is ANSI C, a.k.a. C89. Watcom C still doesn't support C99. So, pot calling kettle old.
Quality software should depend neither on endianness, nor on negative number representation..Also, wake me up when Linux s390x becomes little endian.
Re: (Score:3)
(*) it is true that there are techniques, such as overlay programming, that can effectively extend these limits via disk swap mechanisms, but do note also that DOS has file access limitat
Re: (Score:1)
Re: Of course you can. (Score:2)
What are these file access limitations of DOS? I'm not aware of any significant ones. Or do you mean the limitations of the FAT filesystem?
DOS extenders routinely generate multi-megabyte EXE files that run under DOS. You can create a multi-megabyte EXE file that runs without any extender too, the only caveat is that initially only a small part of the EXE will be loaded in the memory.
Re: (Score:2)
I can make a turing machine that runs on DOS, with the only limit being how many hard drives you've managed to attach to the machine.
"Could we still do that today?" (Score:2)
Re: "Could we still do that today?" (Score:2)
Yet you still can easily run DOS in a virtual machine or DOSBOX.
Turing complete (Score:2)
Turing completeness. Enough said.
why bother (Score:4, Interesting)
DOS (Score:3)
Re: (Score:3)
Yes, I did.
LLama too hit the cow and then crashed into the mountains.
perhaps (Score:2)
Why not (Score:2)
Most neural networks are matrix multiplications followed by nonlinear activation functions. Both can be implemented without high-end acceleration hardware. Read a bit from disk, do the operations, write it back. You won't even need much RAM. It will only be veeery slow. And the typical processors supported by DOS don't even have fast operations for the nonlinear functions like tanh. But of course you can implement them using other operations.
Looking at older platforms without AI (Score:2)
Re: (Score:2)
benchmarks (Score:2)
the benchmarks are interesting. I wonder why the Pentium-M beats the much much newer but similarly clocked core2-duo by nearly a factor of two.
Something to do with using doubles, for the floating point work?