Fresh off its recent voice search study, Stone Temple Consulting compared the relative accuracy of Google Home and Amazon Echo. The SEO firm posed 5,000 questions to each unit “to see which device was smarter.”
In its discussion of results, the firm makes clear that the test was not a comparison of the full range of virtual assistant capabilities. Rather it sought to determine which device more often gave a correct answer to a wide range of factual questions. The specific list of questions wasn’t provided.
Percentage of 5,000 questions device attempted to answer
Source: Stone Temple Consulting
There were some scoring nuances that Stone Temple describes in its own blog post, which I won’t reproduce here. As one might anticipate, however, Google was able to answer many more questions than Echo/Alexa. That’s consistent with my personal experience and makes sense given Google’s search index.
Google Home answered more than three times the number of questions that Amazon Echo could. The surprise is that Amazon Echo had nearly as good a percentage score for those questions it was able to answer.
Of the roughly 3,400 questions Google Home answered, 89.5 percent were correct, according to the study. By comparison, Amazon Echo got right roughly 87 percent of the just over 1,000 questions it was able to answer.
Percentage of questions answered correctly
Source: Stone Temple Consulting
For Stone Temple Consulting, Google Home was the clear winner. However, it’s not clear to me that consumers are drawn to these devices simply to replicate the search experience or to answer factual questions.
Ultimately, they’re entertainment devices and transactional tools to help people accomplish tasks such as ordering takeout, renting a car, buying an iPhone charger, making dinner reservations, creating shopping lists and so on.
I don’t think the battle of the intelligent assistants will be won or lost on the basis of index size. Rather a range of considerations, including general utility, cost and aesthetics (design and UX) will be significant factors. In these other categories, Amazon has more than a fighting chance, especially if it can build out a rich developer ecosystem.
Comments