Google Makes Your Pixel Screenshots Searchable With Recall-like AI Feature (theverge.com) 19
An anonymous reader shares a report: Google has announced Pixel Screenshots, a new AI-powered app for its Pixel 9 lineup that lets you save, organize, and surface information from screenshots. Pixel Screenshot uses Google's private, on-device Gemini Nano AI model to analyze the content of an image and make it searchable.
During a demo at its Pixel launch event, Google showed how you can take a screenshot and then save it to a collection, like "gift ideas." You can also search through all your other screenshots by typing in a keyword, like "bikes" or "shoes." Pixel Screenshots will then pull up all relevant results. Additionally, Pixel Screenshots can give you information about what's inside an image. Further reading: Microsoft Postpones Windows Recall After Major Backlash.
During a demo at its Pixel launch event, Google showed how you can take a screenshot and then save it to a collection, like "gift ideas." You can also search through all your other screenshots by typing in a keyword, like "bikes" or "shoes." Pixel Screenshots will then pull up all relevant results. Additionally, Pixel Screenshots can give you information about what's inside an image. Further reading: Microsoft Postpones Windows Recall After Major Backlash.
Apparently they didn't learn from Microsoft (Score:4, Interesting)
Microsoft faced a massive outcry over exactly this feature. I don't think google will fare well regarding this.
Re: (Score:2)
Re: (Score:3)
Microsoft faced a massive outcry over exactly this feature. I don't think google will fare well regarding this.
Microsoft's was automatically taking screenshots every few seconds. This sounds like it's at the very least user triggered. Though I wouldn't trust, with the infrastructure put in place, it wouldn't become an automatic every so many seconds thing somewhere down the line. This is coming to *EVERY* closed system eventually. Probably more than a few open systems as well, though there you will be able to turn it off. I imagine someone, somewhere, aside from marketing morons, likes the idea. But I'll be damned i
Day late, dollar short. (Score:4, Interesting)
iphone / ipad's been doing this for years. On the device, no network needed. No AI required. Dunno how it does it.
Try it. Put your iphone in airplane mode, then go to your photos app, tap on the search icon, and think of a word you know is on one of your pictures. It'll find it.
Or, search your pictures for "car." Or "plant." Or .. whatever you can think of. It'll find whatever it can match
Scared yet? It'll even find handwriting.
This isn't what Microsoft proposed with Recall.
And this google thing isn't Recall, either. It's just catching up. Big nothingburger. And it requires a separate app. App those appy apps! It's all about the apps, baby!
Heh. And when you're online with your iphone, take a pic of a weed or plant. Go to the pic, hit the "i" icon. If it recognizes the plant as a plant, the i will have a leaf or at least sparklies around it. It means it found a match online. I use it to ID weeds in my yard. No special app required. Been there for a year or more now.
Re: (Score:1)
Re: (Score:2)
You can ID food, animals, texts, etc. as well in iPhones' updated iOS (think it started in v16). However, third party apps are better like Google's Lens, etc. when online.
Re: (Score:2)
Android has been doing it for even longer. They were using image recognition to identify common objects, animals, and people. Text recognition that worked with handwriting.
It's not entirely clear what AI adds to it. Maybe some slightly better contextual understanding.
Google Photos does this for years (Score:2)
Yes, you need to back pictures up to Google Photos but they pester you enough that most people would do it anyway, and Google Photos is the gallery program for Pixels so that would be used anyway to browse local screen shots too. Also even if OCR and "things in the picture" recognition is done online the data is pulled back to the phone in some sqlite DB so you can actually search offline in the app.
Re: (Score:2)
"is done online the data is pulled back"
Thats something you dont expect to happen when you take a screenshot.
In fact, I think that you'd be quite vocal and displeased if your windows desktop computer sent an image of your primary display buffer to Microsoft every time you pressed a certain key on the keyboard.
Re: (Score:2)
You have a very special definition of "glossed over" when I used more than half the post stating that's a requirement and why it would be met in real life by most users.
It's a Google phone, with a Google OS, which you can't seriously consider using without a Google Account, where the picture browser is Google Photos, with a single backup option and that is online to Google (you CAN'T, as in you are not allowed to, use a third party for the vast majority of stuff, yes you can grab pictures and some other los
Google Knows Best (Score:2)
Google will analyze everything. Google will consume all. Google knows best.
No one wants this (Score:2)
Re: (Score:3)
Why do they keep shoving it down our throats?
they are malicious and are trying their hardest to normalize it, just wholesale reject them.
Re: (Score:2)
It's on-device. Whose privacy is being violated?
I've had this feature on my iPhone for a few years now. It's great. Pixel users will love it.
Re: (Score:2)
All phones have a steady stream of TLS 1.2 and TLS 1.3 encrypted data transmission from and to their manufacturer. All the time. Assume this includes "wanted = updates, messaging, synchronizing" and "unwanted = snooping" parts.
Neither you nor anyone else outside these companies has even the slightest idea what exactly is in there. You cannot tell wanted and unwanted background activity apart by any means. You only have some clues coming from the phone consuming power and data traffic and an idea if any "wan
Re: (Score:2)
An AI needs context to offer suggestions and respond to queries. The context is a history of what you did recently.
If you lose your wedding ring, Google wants their AI to be able to say go look in your car for it. How would it know? By taking lots of photos of everything you do during the day. Same with what you do on their phone. Same reason Microsoft wants Recall.
Those companies will not stop unless they change their business model to do something other than AI assistants. Fat chance
So, nothing like Recall? (Score:2)
Recall was a mess because it acted automatically on everything you view all day long. This is a feature that acts on photos and screenshots you explicitly took. They aren't the same. If anything, it's like Apple's Live Text, which has been out for a few years now.
year of the Linux smartphone (Score:2)