Apple Working To Cram Massive Gemini Model Into iPhone To Power New Siri (arstechnica.com) 58
Apple is reportedly working to shrink Google's Gemini models enough to power parts of a long-delayed AI-enhanced Siri on iPhones. But despite Apple's best efforts to run the AI locally, "the iPhone's Gemini makeover will lean heavily on Google and Nvidia in the cloud," reports Ars Technica. That could complicate Apple's privacy-first AI messaging, especially if more complex Siri requests are routed through Google infrastructure and Nvidia's encrypted cloud-computing platform. Ars Technica reports: After inking the Google deal, Apple apparently got to work distilling Google's giant cloud-based Gemini models. Distillation is a process in which a small, less resource-intensive model learns to mimic a large, expensive one. With enough time, this can reliably transfer useful capabilities while pruning less important weights from the model. That may enable Siri to handle some tasks with private local compute, but a cloud component looks inevitable.
Processing users' AI data in the cloud could be a problem for Apple. At WWDC, the company will probably promote its years of experience designing chips and how well that positions it for AI. However, The Information claims that Apple has struggled to even get Google's massive undistilled Gemini models running on its custom Private Cloud Compute infrastructure, which is built on on M-series Mac chips.
When the smarter Siri rolls out, it will probably route more complex tasks to Google's cloud infrastructure instead of Apple's, but it won't be running on Google TPUs. Apple has reportedly signed a deal with Nvidia to use its Confidential Computing platform for this purpose. Confidential Computing keeps data encrypted on Nvidia GPUs while it's being processed in the cloud, which could help Apple claim it's still sensitive to user privacy concerns. It might even retain its own Private Cloud Compute branding for the system.
The iPhone probably won't tell you which version of Gemini is handling individual Siri requests. Device makers designing hybrid systems that rely on local and cloud-based AI like to talk about making the experience feel "seamless." There might be clues, though.
Processing users' AI data in the cloud could be a problem for Apple. At WWDC, the company will probably promote its years of experience designing chips and how well that positions it for AI. However, The Information claims that Apple has struggled to even get Google's massive undistilled Gemini models running on its custom Private Cloud Compute infrastructure, which is built on on M-series Mac chips.
When the smarter Siri rolls out, it will probably route more complex tasks to Google's cloud infrastructure instead of Apple's, but it won't be running on Google TPUs. Apple has reportedly signed a deal with Nvidia to use its Confidential Computing platform for this purpose. Confidential Computing keeps data encrypted on Nvidia GPUs while it's being processed in the cloud, which could help Apple claim it's still sensitive to user privacy concerns. It might even retain its own Private Cloud Compute branding for the system.
The iPhone probably won't tell you which version of Gemini is handling individual Siri requests. Device makers designing hybrid systems that rely on local and cloud-based AI like to talk about making the experience feel "seamless." There might be clues, though.
In 2028? (Score:3)
That's when RAM shortages are supposed to subside.
If you're currently selling netbooks with only 8 Gig, how much RAM will a Gemini iPhone realistically require?
Re: (Score:3)
Re: In 2028? (Score:2)
Isnâ(TM)t deepseek already twice as efficient?
Re: (Score:2)
Depends. Current leading LLM technologies are super bloated and inefficient. When 2028 arrives the resource requirements for a natural language user interface bot may be a lot less. The software side requirements can certainly be finalized a few months before launch. On the other hand, hardware designs have a long lead time and must be locked in much sooner. On the other hand, data can be shipped to a beefy server and processed in the cloud, so the hardware doesn't really need a lot of resources.
Don't hold your breath. If the model becomes more efficient, they will likely use the free capability to increase the model's capabilities rather than make a compact version of the existing capabilities
Re: (Score:3)
how much RAM will a Gemini iPhone realistically require
Depends on what you want to do with it. If you want to generate realistic using images doing whatever you want locally, while answering every question in the universe I suggest you get a phone with at least 96GB of RAM. If on the other hand you are running small local models that do specific tasks and offload the rest to an internet search, you can run that AI model on a iPhone 3GS if so desired.
Not every AI is the same.
Re: (Score:2)
Again, depends on scope. The ability to generate anything requires processing a large model by its very nature.
But image generation on a phone does not play well with battery. Remember that Apple is the company that doesn't allow network connections for background apps to save battery.
I'm not sure if you're actually trying to make the point I think you are, but if you are then you're completely delusional. There's a difference between vampiric power drain and actual foreground tasks causing power drain. There's plenty of applications on the your iPhone that consume huge amounts of power when used.
Apple Watch today (Score:2)
Re: (Score:3)
I think chrome's gemini nano needs about 2 GB. I didn't test it yet. I would also expect Apple to distill to a similar size like Gemma E2B or E4B, that's also what Google uses for on-device AI (which uses exactly those models but as far as I know with added MTP layers).
Re: (Score:2)
I've seen estimates around 12 GB of RAM required for some Gemini models.
Re: (Score:2)
The large Gemma models already need about 24 GB (or offloading to CPU). Gemini is way larger.
Battery empty ... (Score:1)
... phone dead.
Siri, I need help!
Re: Battery empty ... (Score:2)
Re: (Score:3)
Yeah, it will be funny when you realize you can't swap your e-sim if your phone is dead either.
Why would you do that? No seriously why. I want to follow the train of thought here for a moment. Your phone just died, what befit do you get from swapping the SIM?
Why carry a whole second phone when a simple charging bank would do.
If you have another phone, why not just get a second SIM for it.
If you are borrowing a phone, I'm sure the person will let you briefly use their data.
If you need to make a phone call, then just use the other person's phone.
Heck if you have a second phone, just connect the two tog
Re: (Score:1)
Read carefully what he wrote: swapping an e-sim.
An e-SIM
A phone can hold nearly endless e-Sim. Perhaps you delete one, and install a new one, but that hardly counts as swapping.
While I am historically a heavy Apple user, I am not a "fan boi". So I made a joke about the well known fact that Apple phones historically have short battery live (the main reason I wont buy any in the foreseeable future - my chat apps work just the same on Android, so the "imaginary" better OS, is completely irrelevant for me.)
And:
Re: (Score:2)
There's no such 'well known fact'. Apple Phones have historically had some of the best battery life in their generations, with occasional outliers. (If we compare the latest Samsung vs. the latest iPhone, for example, the iPhone has significantly better battery life.)
On the other hand, there are occasional Android phones with absolutely absurd batteries that last a couple days on a charge with the tradeoff of looking like a pound of butter. The fact that these have better battery life is not surprising and
Re: (Score:1)
All my iPhones never lasted - with extremely sporadic usage - a day, or more than a day.
My old Android, with similar usage patterns, lasts nearly a week. My new Android sucks.
It even gets hot ... just when the hotspot is on. Not really sure where the power is going. Sometimes it lasts 1 or 2 days. Most of the time it is roughly 10h.
Keep in mind: all my phones are in flight mode over night till the first time I need them ... and the iPhone is dead when I go to bed. (iPhone SE ... from 2021 or so ...) It is d
Re: (Score:2)
I have no idea what you're doing with your phones, then.
The only Android phone that I used for any length of time was a Nexus 4 and it would die in 1 hour when I went to the pool because it would burn all of its energy desperately trying to get a signal from inside a locker.
My current iPhone 16 Pro lasts the whole day unless I'm using it actively and playing a game on it. It has notoriously poor battery life under load because it's not well cooled, but when idle? Days of battery life. I've been on trips whe
Re: (Score:1)
When they search for a signal, they are sometimes really bad.
The point is: I do not do anything with my phones ... and yet they are empty, that was the point.
My infinix, low budget phone: is okay though :D
"Nvidia's encrypted" what? (Score:1)
Re: (Score:2)
Since when does a hardware manufacturer own the servers?
Since about a couple years ago, from what I gather. Soon to be the standard, if they have their way.
Don't want an AI iPhone...... (Score:3)
Re: (Score:2)
So... buy another phone? I don't understand what you're worried about here, are you suggesting that every company should suit precisely your needs?
Re: (Score:2)
There are still lots of dumbphones on the market. https://www.dumbphones.org/ [dumbphones.org]
And best of all, they are really, really cheap. And they don't require cellular data.
Unfortunately, you can still be tracked, even if you have a dumbphone, so if tracking is your issue, you're out of luck.
Re: (Score:2)
Plus don't use Siri much. It's bad enough now when you do a Google search and it has a small disclaimer that results may not be accurate !!!!
To be fair, it arguably should always had had that disclaimer. They avoiding needing it by saying they were just providing links and it was on those web sites and their users to deal with any inaccuracies... but everyone knew that most users just took whatever the top link said as definitive truth.
Distilling allows plausible deniability (Score:3)
With progressive layer by layer distillation Apple can make aggressive changes in architecture, all while letting Google take all the blame for the piracy.
I think there is a lot of potential to improve architectures for local, beyond MoE and what Apple "pioneered" with LLM in a Flash (the low rank predictor approach was actually first described in a paper from 2013 they didn't cite). Google's spark transformer for instance is already far more elegant than MoE and low rank predictors, beyond that there is also unexplored potential of forced temporal coherence in the active set.
Only Apple and Tiny AI are likely to truly push sparsity in production. Going beyond MoE with sparsity and being forced to accept low single digit percentage compute utilisation during training on NVIDIA's expensive HBM based GPUs is too counter-intuitive for most researchers to accept, even if they really should.
This isn't the AGI you're looking for (Score:2)
Apple-Google-iNvidia(sic)
Move along.
No Thanks (Score:3)
Re: (Score:2)
Try a dumbphone, there are still lots of them available, for as low as $20. https://www.dumbphones.org/ [dumbphones.org]
Unfortunately, you can still be tracked, even if you're on a dumbphone. So if tracking is your issue, you're out of luck. But they DO let you make and receive calls, take voicemails, and text. For those who don't want sophisticated phones, it's perfect!
It's been a while.. (Score:4, Interesting)
Imagine a Beowulf cluster of those ;)
Siri continues to be Apple's shittiest product (Score:2)
Re: (Score:3)
Example, my 17 pro is pretty big and heavy, so you end up gripping it every time you pick it up. But with the extra buttons on the sides you end up engaging something you didn't want. So then you menu-dive into system settings just to turn off extra buttons.
I feel like I am fighting with this thing - Jobs used to say he was proud of the things Apple didn't do. Man, those days are gone.
Re: (Score:2)
i feel like that's been the problem with every phone i've ever had: apple or android, my last android was an HTC with big light-touch buttons on the side.
Re: (Score:2)
Example, my 17 pro is pretty big and heavy, so you end up gripping it every time you pick it up. But with the extra buttons on the sides you end up engaging something you didn't want. So then you menu-dive into system settings just to turn off extra buttons.
My kids call me a boomer when that happens to me. And yeah, it happens.
Though to be fair, I actually really like the side button -- the one on the lower right that is touch sensitive. I use it for activating and using the camera. I just ALSO sometimes activate it when reading in landscape mode. Oops.
Re: (Score:2)
Example, my 17 pro is pretty big and heavy, so you end up gripping it every time you pick it up. But with the extra buttons on the sides you end up engaging something you didn't want. So then you menu-dive into system settings just to turn off extra buttons.
This has been a problem ever since the iPhone 6, when Apple inexplicably decided to move the on/off/lock button from the top of the phone over to the right side, directly opposite the volume buttons.
So Google wins this round (Score:2)
So if you're using Android, you're using Google. And if you're using Apple, you're using Google.
Seems Google is in the dominant position here.
Apple works to increase privacy, very not-Google (Score:2)
So if you're using Android, you're using Google. And if you're using Apple, you're using Google.
Seems Google is in the dominant position here.
With Apple, we have increasing use of local ML processing for various reasons including privacy. Unlike Google, Apple sells hardware and software not you.
Re: (Score:2)
That's what Apple would like you to think. In recent years, Apple has started making some serious pushes into targeted marketing. https://www.wired.com/story/ap... [wired.com] Apple's revenue in the last year from targeted advertising was around $7 billion. https://www.businessinsider.co... [businessinsider.com] Now, with Apple, you pay extra AND you are the product.
Re: (Score:2)
Re: (Score:2)
Their targeted advertising occurs on device and they do not share your info with advertisers.
Well, not yet. Five years ago, would you have thought Apple would ever start pushing ads to app store users? I'll bet not, but they did.
The thing about Apple, is that their products are no longer cutting edge or innovative. People aren't buying new iPhones quite as frequently. So they need to find other ways to find revenue to build fancy circular buildings in the priciest real estate market in the country. First ads, next privacy. You watch, it will come.
Re: (Score:2)
Their targeted advertising occurs on device and they do not share your info with advertisers.
Well, not yet. Five years ago, would you have thought Apple would ever start pushing ads to app store users?
Yes, I would. Because there were doing so about 20 years ago. They had their own rival in-app ad platform. They dropped it around 15 years ago and most developers switched to the Google in-app ad platform.
Re: (Score:2)
Not especially. Remember, Google pays Apple about $20 billion/year to be the default search in Safari. The reports are that Apple pays Google $1 billion for Gemini.
And if we're honest, Gemini is not the clear-cut best model, it's just that Google and Apple already have a pretty good relationship. Given the amount of Capex Google is putting into AI/Gemini, they need to make money from SOMEWHERE, and Apple is a reliable partner. I'm sure they're extremely relieved that Apple is going with them instead of Anth
Re: (Score:2)
Yes, if you look at the arrangement in purely transactional terms, Google is paying Apple more than Apple is paying Google. But in the end, users are still using Google's software. Google wouldn't pay Apple if they didn't fully get their money back in that deal.
I agree on your characterization of Gemini. It's not the best, but it's pretty decent. It certainly beats Apple Intelligence.
Did anyone ask for this? (Score:3)
Are consumers really clamoring for this?
Re: (Score:3)
"
No, consumers are not clamoring for AI in iPhones.
While Apple, Google, and Samsung have spent billions marketing generative AI as the definitive reason to upgrade, multiple industry surveys and sales figures show a massive disconnect between tech company hype and actual consumer behavior.
Data reveals that the "AI revolution" on smartphones is largely supply-driven by Silicon Valley, rather than demanded by everyday users.
The Real Upgrade Drivers vs. AI Hype
According to market tracking fr
Re: (Score:2)
Oh wait,...
Is that answer now slop or not?
Re: (Score:2)
Half the posts: AI is dumb, do not want, is anybody actually asking for this?
Other half: Siri is dumb, it's Apple's worst product, it's why I use Android.
Apple tends to let you turn things off (Score:2)
Are consumers really clamoring for this?
No more than any other thing that Apple created and became wildly successful with.
Apple also has a track record of letting you turn features off.
Doesn't matter (Score:2)
Apple will fail.
Neat! (Score:2)
Now Apple can flatten my battery and use all my ram and bandwidth to make me part of a distributed AI service it can sell! Hurrah!
Most Important Feature: Ability to Opt Out (Score:2)
It's simple... (Score:2)
I already turn off Apple Intelligence, and don't use Siri. So I'll just disable all of this.
Now if I could only get rid of that "Build your ideas with Gemini" banner at the top of the Slashdot page...