Google's Best Gemini Demo Was Faked (techcrunch.com) 49
Speaking of early-impressions of Gemini, users' confidence in Google might be shaken further to learn that the company pretty much faked the most impressive demo of Gemini. TechCrunch: A video called "Hands-on with Gemini: Interacting with multimodal AI" hit a million views over the last day, and it's not hard to see why. The impressive demo "highlights some of our favorite interactions with Gemini," showing how the multimodal model (that is, it understands and mixes language and visual understanding) can be flexible and responsive to a variety of inputs.
To begin with, it narrates an evolving sketch of a duck from a squiggle to a completed drawing, which it says is an unrealistic color, then evinces surprise ("What the quack!") when seeing a toy blue duck. [...] Just one problem: the video isn't real. "We created the demo by capturing footage in order to test Gemini's capabilities on a wide range of challenges. Then we prompted Gemini using still image frames from the footage, and prompting via text." So although it might kind of do the things Google shows in the video, it didn't, and maybe couldn't, do them live and in the way they implied. In actuality, it was a series of carefully tuned text prompts with still images, clearly selected and shortened to misrepresent what the interaction is actually like.
To begin with, it narrates an evolving sketch of a duck from a squiggle to a completed drawing, which it says is an unrealistic color, then evinces surprise ("What the quack!") when seeing a toy blue duck. [...] Just one problem: the video isn't real. "We created the demo by capturing footage in order to test Gemini's capabilities on a wide range of challenges. Then we prompted Gemini using still image frames from the footage, and prompting via text." So although it might kind of do the things Google shows in the video, it didn't, and maybe couldn't, do them live and in the way they implied. In actuality, it was a series of carefully tuned text prompts with still images, clearly selected and shortened to misrepresent what the interaction is actually like.
Google did unethical thing?! Shocking! (Score:1, Insightful)
Do no evil, assholes.
You forgot where you (claim) to have come from. You survive only on inertia now.
Re:Google did unethical thing?! Shocking! (Score:5, Insightful)
Don't be (merely) evil (Score:2)
Oh Google. It was impressive as hell even without the fakery. Why soil your reputation that way?
I think what has happened is the bussiness majors and kinds of scientists who'd rather publish fake results than modest good results have taken over from the founders.
Re: (Score:1)
Re:Google did unethical thing?! Shocking! (Score:5, Insightful)
You either go bust, or live long enough to see yourself be taken over by profit-maximizing assholes.
Re: (Score:2)
Must you refer to Sundar Pichai this way? It's very offensive to the shareholders.
Re: Google did unethical thing?! Shocking! (Score:2)
Re: (Score:1)
The keys to google's existence are:
1) spying on people
2) selling people's PII to corporations and governments
Every thing else they do leads to one of the above to generate revenue.
Re: Google did unethical thing?! Shocking! (Score:2)
Re: (Score:1)
Thumbs up.
Re: (Score:2)
The key to Google's existence are
If your business model is reliant on a specific model of browser, you should already know your own fate. Not like the rest of a planet is going to bow to that forever. Especially when you're caught faking your latest whizz-bang.
Be Evil (Score:2)
That's been the company motto for the longest time.
Re: (Score:2)
To be fair, that "do no evil" lie was a good way to get a lot of highly intelligent people with no wisdom to sign on with them.
But yes, Google has not really done anything remarkable for ages.
Re: (Score:2)
I admit to being one of them. I thought it was cool and actually believed it for a while until the unbreakable law of "look at what people do, don't listen to what they say" set in.
Re: (Score:2)
To me it always sounded like a variant of do not think about a pink elephant. If you constantly have to repeat to yourself you must not be evil must not be evil must not be evil... then all work and no play makes you a dull boy.
Re: (Score:2)
Re: (Score:2)
Classic Mechanical Turk* (Score:4, Informative)
(* Mechanical Turk [wikipedia.org])
Yo, AI. (Score:3, Insightful)
Do we hear a specific accent in the video? When we hear this accent, would you expect that we get a presentation full of exaggerations and bullshit?
As one Romulan put it... (Score:5, Funny)
It's a faaaaake [youtu.be].
Re: (Score:2)
Isn't that guy with the pointy ears a star wars math elf, what they call "bolkan" or somesuch?
Whooops (Score:4, Interesting)
ChatGPT was the opposite of this. People didn't first see it through ads. They pulled up the URL, and were amazed.
Re: (Score:2)
Re: (Score:2)
ChatGPT was the opposite of this. People didn't first see it through ads. They pulled up the URL, and were amazed.
Well there was the sparks of AGI paper and video presentation... several things in that paper were not reproducible including the svg unicorn drawing once GPT-4 was publicly released. Probably due to RLHF induced brain damage applied to avoid bad press.
Re: (Score:2)
Gotta keep the hype train on track (Score:2)
Honestly reporting the progress made would have been very impressive, but someone evidently believed it needed more
The evolution of AI should be a series of technological advancements, honestly reported by the tech press
Instead, it has turned into a cutthroat competition for investment dollars and the attention of the general public, by any means necessary
Re: (Score:3)
A manager probably demanded more and the company culture is probably diseased where avoiding confrontation with management is the common way to respond. Better to lie to the customer than say no to the manager ...
It's a cutthroat competition to suck up to the boss and ride the sinking ship down.
Re: (Score:2)
>The evolution of AI should be a series of technological advancements, honestly reported by the tech press
When has this been true of any technological advancement? Please let me know the golden era where this was happening.
Confirmed. (Score:2)
Not even the first time Google has faked AI (Score:4, Informative)
Most tech demos are faked (Score:1)
It's not necessarily that stuff doesn't work... it's a demo
Most companies want to avoid scenarios where the computer crashes mid-demo like Windows crashing on Bill Gates
Now how good it is... time will tell
Re: (Score:3)
If the summary is correct, then this wasn't just a case of "it crashes a lot so we made a video that's spliced together from multiple takes". That would be something like what you're talking about, in my view. That would be an actual demonstration of a capability that exists, even if not stable enough to dare show live.
This was more like "let's show a video of us feeding live video and audio to an AI, with the AI narrating what it sees and responding to verbal conversation about what's going on in the video
Re: (Score:3)
To add to my above comment, I decided to click through to the article and check the video itself. To Google's credit, they do explain a few things in a link in the video description. They also apparently responded to criticism with "The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers."
If they had named the video something like "here's what we hope Gemini will eventually do", that would have been perfectly fine. But they didn't. They
Re: (Score:2)
Re: (Score:2)
>It's not necessarily that stuff doesn't work... it's a demo
There's a difference between a demo and a scripted presentation. A BIG difference. Many companies have been pulled up and chastised for giving scripted presentations where they said they were giving demos.
>Most companies want to avoid scenarios where the computer crashes mid-demo like Windows crashing on Bill Gates
Yes, well, that's the whole point, or part of it. Is the thing good enough to actually do what they are claiming, as seen live on
Why is this unexpected? (Score:2)
I hate to break it to you guys, but Superbowl halftime shows are lip synched, too.
"Breaking news: Demo videos are edited to maker products look better, film at 11."
Re: (Score:2)
This doesn't seem so much like a demo as a scripted presentation. There is a big difference, and Google is touting this as if it was a demo, which again, it does not seem to fit the description of.
Re: (Score:2)
There's a difference between editing together a representative video of having the bugs worked out that came up during attempted recording, or taking several prerecorded runs at a demo to getting it right, and outright fabricating a video implying function that doesn't exist even in theory.
The concept video they imagined misrepresents the capability, by stitching together independent examples that they retroactively decided could fit a neat narrative if represented as a contiguous interaction in real time c
Re: Why is this unexpected? (Score:2)
The concept video they imagined misrepresents the capability, by stitching together independent examples that they retroactively decided could fit a neat narrative if represented as a contiguous interaction in real time conversation. Likely skipping over more verbose prompts and refinements made after a disappointing response, and outright discarding attempts that landed nowhere.
This literally describes editing to make the product look better.
It would be like if I showed up on stage and lip synced to Frank Sinatra and claimed it was my voice to get people to hire me to sing at events.
This is not the least bit analogous to what you described happening. The more accurate analogy would be Frank Sinatra editing together parts of different takes of his own singing and promoting it as an example of his live show. Which, I hate to break it to you, is how a lot of albums are made.
Bamboozled By Google (Score:2)
Google has denied engaging in disinformation campa (Score:4, Funny)
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Not Surprised. (Score:3)
Stomping Gemini's Namespace. Die Google. (Score:2)
What I do care about is that they are stomping on another project called Gemini [geminiprotocol.net] wit
All of it's faked (Score:2)