Meta Launches Powerful Open-Source AI Model Llama 3.1 20
Meta has released Llama 3.1, its largest open-source AI model to date, in a move that challenges the closed approaches of competitors like OpenAI and Google. The new model, boasting 405 billion parameters, is claimed by Meta to outperform GPT-4o and Claude 3.5 Sonnet on several benchmarks, with CEO Mark Zuckerberg predicting that Meta AI will become the most widely used assistant by year-end.
Llama 3.1, which Meta says was trained using over 16,000 Nvidia H100 GPUs, is being made available to developers through partnerships with major tech companies including Microsoft, Amazon, and Google, potentially reducing deployment costs compared to proprietary alternatives. The release includes smaller versions with 70 billion and 8 billion parameters, and Meta is introducing new safety tools to help developers moderate the model's output. While Meta isn't disclosing what all data it used to train its models, the company confirmed it used synthetic data to enhance the model's capabilities. The company is also expanding its Meta AI assistant, powered by Llama 3.1, to support additional languages and integrate with its various platforms, including WhatsApp, Instagram, and Facebook, as well as its Quest virtual reality headset.
Llama 3.1, which Meta says was trained using over 16,000 Nvidia H100 GPUs, is being made available to developers through partnerships with major tech companies including Microsoft, Amazon, and Google, potentially reducing deployment costs compared to proprietary alternatives. The release includes smaller versions with 70 billion and 8 billion parameters, and Meta is introducing new safety tools to help developers moderate the model's output. While Meta isn't disclosing what all data it used to train its models, the company confirmed it used synthetic data to enhance the model's capabilities. The company is also expanding its Meta AI assistant, powered by Llama 3.1, to support additional languages and integrate with its various platforms, including WhatsApp, Instagram, and Facebook, as well as its Quest virtual reality headset.
Open Source Model? (Score:1)
Re: (Score:1)
So can you run it locally?
Certainly. All you need is 16,000 Nvidia H100 GPUs. Should be a snap for any hobbyist.
Re:Open Source Model? (Score:4, Funny)
16000 GPUs for training, ~30 GPUs will probably be enough for inference.
Well, shit. At 30k a pop, that's no problem at all. Let me shake my couch cushions.
Re: (Score:3)
Yes, the 70B you can run quantitized on pretty standard hardware even though you may not get a lot of tokens per second. Full 405B model is going to be more complicated to run locally but it's basically just a money issue
Re: Open Source Model? (Score:2)
Re: (Score:2)
Sounds like asking if you need 1 or 2 ferrari's to join some club, either way my wife probably not going to agree to me selling the house for it
Re: (Score:2)
Sure thing! Using Ollama, it's as easy as falling off a log.
https://ollama.com/library/lla... [ollama.com]
Re: (Score:3)
You can run it just fine if you have enough RAM. In this case that would be a 256 GiB system, of witch 231 GiB will be used to run the model. You don't need a GPU, but you need fast memory. ollama generates about one token (1 word +/-) for each time is scans the entire model's memory. It's more bound to memory performance than to CPU / GPU performance. GPUs have much faster memory, but less memory typically, but server processors with fast DDR5 and 8 or 12 channels should work just fine also. For smaller mo
Re: (Score:2)
What size are the parameters? (Score:2)
Does anyone know what size the parameters are? int8? float8? int16? float16?
Oops, can't generate that image... (Score:2)
...seems a little glitchy at the moment.
lalalalama (Score:2)
Golly! What SPECS! (Score:2)
...does it do anything?
It's not opensource (Score:5, Informative)
It has a restrictive license. Not opensource. Stop calling it opensource.
Re: (Score:1)
I tend to agree this is a binary distribution and the source for creating is not open at all.
Next Release: Llama 95 (Score:2)
Re: (Score:2)
If we follow the joke, then Llama 95 will be released in 2096.
Winamp (Score:2)