Google Assistant Is Smarter Than Alexa, Study Finds (cnet.com) 68
For the second year in a row, a study found that Google's digital assistant is smarter than Amazon's assistant, Alexa. The study does note that Alexa is catching up and was far and away the most improved from 2017 and 2018. CNET reports the findings: Digital marketing company Stone Temple released the results of its 2018 smart speaker quiz earlier this week. It did a similar test last year in which it asked digital assistants roughly 5,000 questions to see which assistants answer the most correctly. For the first time this year, Stone Temple asked the questions separately to Google Assistant on the company's Home smart speaker, and an Assistant-equipped phone. The study found that Google Assistant attempts the most responses, and gets the most attempted responses correct. Strangely, Assistant performed even better on a phone than on a smart speaker. Surprisingly, Microsoft's Cortana took second place, with Alexa trailing both and Siri lagging far behind the rest. Alexa doubled the number of questions it was able to answer from 2017 and Microsoft's assistant improved as well, with Google holding relatively steady at the top while its competition catches up.
bah (Score:3)
So the companies with their own search engines ranked first? Wow, a shocker!
Re:bah (Score:5, Funny)
When Jeff Bezos realizes that, he'll enter the search engine biz. Maybe he'll buy Microsoft, keep Bing and Azure cloud, and sell the rest to China.
Re: (Score:3)
Considering
* back in Feb. 2018 [cnbc.com] that Amazon's market cap was at $702.5 billion compared to Microsoft's at $699.2 billion (beating MS for the first time),
* but in March 2018 [marketwatch.com] Amazon was at $684.3 billion compared to Microsoft's $692.4 billion ...
Re: (Score:2)
A few days agao Amazon and Microsoft were neck and neck [usatoday.com]:
Re:bah (Score:5, Funny)
No, actually I think its because Google had already analyzed everything that all the Stone Temple employees said and did via email, chat, orders on the web, and past web search history.
Google Assistant already figured out who was doing the study and what questions they were going to ask, so they had the answers ready for them before hand. The web searches were just to make the competition look legit to the judges so they would not catch on.
Alexa on the other hand only knows what Stone Temple employees ordered online from Amazon, and last I checked you still could not order "test questions" from Amazon, even with Prime Membership.
Smarter Web searching (Score:4, Interesting)
Re:Smarter Web searching (Score:4, Interesting)
I think the difference between Google's products results shows that's not entirely true. Otherwise, they would've returned the same results.
AI/Machine Learning is playing the pivotal role here.
Siri's results prove that in my mind, she's dumb af. All she does is web queries.
Re: (Score:2)
Re: (Score:1)
I'm going to patent that process. Sounds obvious, but the Patent Office doesn't seem to notice such.
Zero surprise (Score:4, Interesting)
The entire point of Google Assistant seems, to me as a user, to be to actually assist me. It gives me answers, plays games, schedules things. It's pretty smart. I'm totally fine with Google mining all that data to target me with ads or whatever.
The point of Alexa, and most of Amazon's technology from phones and tablets to buttons and Alexa, is to make me buy things from Amazon. The other aspects are just as good as they have to be to keep up with the market, sorta.
The difference in design focus is apparent when using these systems.
Re: (Score:1)
Are you Leo Laporte in disguise?
You seem to be drawing a distinction without a difference. The point of both devices is to financially benefit their creators by getting you to purchase things. I really don't see how Google doing it is "good" while Amazon doing it is "bad".
Re: (Score:2)
Re:Confusing bar graphs... (Score:4, Interesting)
They are stupid. They have comparison bar graphs showing % correct of those attempted and bar graphs with % attempted. Which are both relatively useless metrics on their own, the most useful one for a comparison would be % correct over asked and they don't provide that!
I can't believe anyone with knowledge of statistics, data analysis or a person of sound logic in general had anything to do with this report.
All their graphs are like that, the year-over-year comparisons compare either the % attempted or the % correct of those, never the % correct overall, even their % wrong comparison is just on those attempted! Crazy-frustrating report! How confident can you be that their methodology is sound after that?
Re: (Score:2)
My guess would be it's:
X% of questions asked are attempted.
Y% of questions attempted are answered fully.
X and Y are thus not percentages of the same thing, and thus Y can be greater than X.
Re: (Score:2)
Re: (Score:2)
Product A attempted to answer 60 / 100 questions.
Product A fully answered 40 questions.
It attempted 60% and fully answered 66.666%.
The "fully answered" percentage is measured against the number attempted, not the number asked.
Not surprised (Score:2)
Google Assistant can be dryly funny (Score:5, Interesting)
I recently said to my phone, "OK, Google, is it supposed to rain today? When you answer, keep in mind that you got this question wrong yesterday."
To my surprise, it replied with a list of links to alternative digital assistants I could install.
Re: (Score:3)
Each morning I want to know whether the day will be warmer or colder than the day before. It seems like such a simple question but the best either Google or Alexa can do is give me today's temperature, expecting me to complete the research on my own.
Bullshit! (Score:2)
Alexa brings me beer and booze and everything else.
Good enough for me.
"Directions to Whole Foods" (Score:5, Interesting)
is a question Siri can't answer. Tried that yesterday eight times and had three coworkers try that each at least two times. One of the responses gave us directions to Austin, TX I assume because that is the home of Whole Foods. If AI isn't good enough to give us directions to a store a couple of blocks away, it's just useless and not even worth discussing.
Re: (Score:3)
is a question Siri can't answer. Tried that yesterday eight times and had three coworkers try that each at least two times. One of the responses gave us directions to Austin, TX I assume because that is the home of Whole Foods. If AI isn't good enough to give us directions to a store a couple of blocks away, it's just useless and not even worth discussing.
Wow, really?
"Drive to Whole Foods" (or to some market I actually use) with Google voice search on Android would result in immediate directions, or "you have a few to choose from", etc.
And that's just the plain old voice search, without the "assistant" window dressing.
Re: (Score:2)
Re: (Score:1)
Everyone is apples bitter rival; that explains why siri is so crappy.
Re: (Score:2)
Me: Give me directions to Whole Foods.
Siri: Which Whole Foods? Tap the one you want.
All the listed choices were nearby.
Define smarter (Score:4, Insightful)
Yes Google's assistant can process natural language far better.
But is it "smarter"? I'd define smarter as "able to do more" and it's clear that not only does Alexa win there, it has an insurmountable lead both in terms of marketshare and with the furious clip that Amazon has people developing "Skills" for Alexa.
Does anyone doubt eventually Alexa will handle natural language too? But then it will ALSO have a wide distribution of users, and vastly more skills... not to mention being wired into something people want, need and use (Amazon purchasing).
Re:Define smarter (Score:4, Interesting)
and vastly more skills... not to mention being wired into something people want
Yes I doubt these two parts. The rest of it seems okay but frankly I would be very amazed and actually quite let down if Google with it's phenomenal amount of data mining and large amounts of data research doesn't end up with the most useful and functional assistant.
As for being something people want, I would say that is trumped by something people have. Many people (myself included) don't want a digital assistant. None the less I have one in my phone. It would be quite phenomenal if a digital assistant people need to buy separately trumps the install base of a digital assistant that multiple people already have in their house in their phones.
Re: (Score:2)
frankly I would be very amazed and actually quite let down if Google with it's phenomenal amount of data mining and large amounts of data research doesn't end up with the most useful and functional assistant.
But it's too broad and not well focused. All of that doesn't matter for the basics. Alexa can tell me what the weather is just as good as Google can - but Google doesn't have the integration to order me my usual toothpaste when it gets low, or at least not something with zero friction the way Amazon
Re: (Score:2)
but Google doesn't have the integration to order me my usual toothpaste
So what you value in a device is an on demand shopping system rather than a digital assistant.
I don't either, and that makes us the absolutely worst judge of what people want in terms of digital assistants.
No it doesn't. It only makes us a bad judge if we aren't interested in viewing the target market. Interesting too that your friends all have an Alexa. The only person I know with an Alexa is my sister. It's still in the box though because she already has the little ugly Google puck looking thing in her living room (she won the Alexa). I know a lot of people with the Google assistant though (as in the standalone one
Forget the best part (Score:2)
Google may have a lot of data but Amazon has literally HUNDREDS OF PEOPLE working on just Alexa. I don't think it's crazy to think that Amazon will shoot way, way ahead of Google on this effort. Even if Amazon didn't have a massive lead now I would expect this to put them on top.
Re: (Score:2)
Google may have a lot of data but Amazon has literally HUNDREDS OF PEOPLE working on just Alexa.
Implying that Google's assistant is just some programmers side hobby?
Implying that throwing people at something solves a data set / deep learning problem?
Honestly I fail to see how you got to your conclusion and while I can't find numbers I will eat my hat if Google doesn't have several times the number of people in the deep learning field working on everything from natural language analysis to how to integrate answers together in relevant results as Amazon.
You forget that analysing data and serving up resu
Google is not Google (Score:2)
Implying that Google's assistant is just some programmers side hobby?
Nope, just that Google does not have nearly as many resources on it, which is 100% true. I know a few people that work at Google...
Implying that throwing people at something solves a data set / deep learning problem?
I've been doing some deep learning stuff over the past few years, your problem is implying deep learning can address this generally and/or better than 500 people thinking of ways they can improve and expand a service TODAY. A
The world's tallest Midget (Score:2)
And smartest of all (Score:3)
Re: (Score:2)
Smarter than Alexa? That's not saying much. (Score:1)
I bought 3 Home devices and 2 Alexa devices along with a few Hue lights and strips for the kids to play with.
Alexa immediately annoyed the crap out of me. Most questions got an answer, and a good number of them also ended with "We have XYZ in stock, would you like to order them now?"
"Alexa, what kind of food does a marmot eat? Mostly vegetation. We have marmot food in stock for $15.95, would you like to order some?"
While Google do
Re: (Score:2)
please fix search engines ! (Score:2)
The voice assistant is remarkably responsive when you understand the technology necessary. But please don't neglect ordinary search engines!
Amazon can't even do a product search. I searched for a product by brand name, a name unique among all the products available. I got 10,000 results, only one of which actually had that unique word. I searched for "paper towels" and got 10K results, only a handful with my term and far outnumbered by 'toilet paper' and soaps and totally unrelated products.
Google is almost
Smarter? (Score:2)
Not enough jokes on the quiz (Score:2)
If there had been, Siri would have come in first, of course!
And...I'll bet Alexa was the best at ordering stuff from Amazon.
Alexa, ... (Score:2)
Can't you just say, "Alexa, Enable the Google Assistant Skill" ?
Seems like that ought to fix any problems!
What is Alexa good for? (Score:2)
My understanding is that Alexa is still superior for answering questions about whether cats can eat pancakes.