NCSA Compares Google and Yahoo Index Numbers 395
Posted
by
ScuttleMonkey
from the searching-for-truth dept.
from the searching-for-truth dept.
chrisd (former Slashdot editor and now Google employee) writes "Recently, Yahoo claimed an increase of index size to "over 20 billion items", compared to Google's 8.16 billion pages. Now, researchers at NCSA have done their own, independent, comparison of the two engines. "
Yahoo pants down, egg on face, no WMD either. (Score:3, Interesting)
Honestly, when I first heard the news over the weekend I thought "rubbish, they must be ignoring requests for spiders to go no further or something." I guess NCSA can either 1) Expect no gifts from Yahoo OR 2) Report significantly different results after a sizable gift to NCSA.
75% less truth than other leading brand
Accurate results? (Score:5, Interesting)
Try searching for the word, "failure" in Google and check the results.
This brings into question *accurate* results. In this case it appears that's left to interpretation.
The results (Score:5, Interesting)
In other words, they believe Google indexes more items based on their own tests of searching.
Hrmm (Score:3, Interesting)
Queries with 1,000 results (Score:4, Interesting)
That makes sense, but it does stand to reason (or, at least, to my reason) that these queries that garner large numbers of results could have had a significant impact on the bottom line of the survey.
Those could be the larger sites, where Yahoo is perhaps digging deeper, requesting data from forms, ignoring robots.txt, etc. It could be where they're getting those big claimed numbers of indexed documents.
Conflict of interest? (Score:1, Interesting)
It seems to me that when Slashdot publishes an article that is favourable to Google, that was submitted by a Google staff member, one might question whether someone involved has a conflict of interest. It's not astroturfing, because his employment at Google was clearly mentioned. It might be an ad (or more correctly, a press release) masquerading as news. I wonder if the article would have been published had it been submitted anonymously...
Re:Yahoo pants down, egg on face, no WMD either. (Score:4, Interesting)
More please! (Score:5, Interesting)
OK, it is yet another Google piece, but it's not "some junior analyst predicts Google will buy Apple and release OSX86box 720".
Google parses plurals differently. (Score:3, Interesting)
The net result is that for the same index size, Google will return more results. (And, IMHO, more meaningful ones.)
Re:Accurate results? (Score:2, Interesting)
Proper name samples (Score:5, Interesting)
Search: Valerie Plame
Google: 908,000
Yahoo: 2,580,000
Search: "Boulder, Colorado"
Google: 1,600,000
Yahoo: 5,880,000
Search: "Linus Torvalds"
Google: 2,560,000
Yahoo: 5,870,000
I assume it goes on like this. Of course these exceed the 1000 maximum hit limit given in the study.
not so fast (Score:2, Interesting)
Yahoo returns 0 results, and Google returns... 4 different links to the ispell dictionary (or variants thereof).
('carbolization clambers')
Re:Conclusion (Score:1, Interesting)
Re:Proper name samples (Score:3, Interesting)
Search: "Dirk Bradford"
Google: 11
Yahoo: 15
Search: "Ronald Hendrickson"
Google: 170
Yahoo: 418
Search: "centerville baptist church" iowa
Google: 43
Yahoo: 37
Well that's less certain. It's hard finding words that return over zero but less than a thousand results...
Are more search results "better"? (Score:3, Interesting)
What I care about is actually getting the information I went out to find. There's only a certain amount of hits I'm willing to explore. That's probbably on the order of 100-200 or so if I _really_ need the information. The implication by Yahoo is that more hits == better top ranked hits. Is that true? Really what should be done is just compare the top few hundred hits between the two search engines and see how they differ. Those are the only ones that matter anyway.
Where more results might prove usefull is obscure searches with less than 100-200 hits. But if this study is true, Yahoo does a worse job on obscure searches that google.
The problem of course is the type of obscure searches that this study performed. Two random words out of a dictionary just isn't what your typical person conducting a search engine query is looking for.
Re:Conclusion (Score:2, Interesting)
1. Make a list of 999 sites.
2. Set up website with a query input form.
3. Upon query, return the entire list.
A major problem with this study is that the number of results returned depends on two variables: (a) the number of sites in the index (so far so good) and (b) the accuracy and sensitivity of the search algorithm. The latter is the very point of a search engine. Yahoo may, who knows, be more selective in returning results.
I'm a google fan, but these results prove nothing.
Flaws in methodology (Score:2, Interesting)
2. Assumes, as many people have stated, that by using an English dictionary for its seeds, the study assumes that Yahoo's expansion has been in English. If Yahoo has expanded it's database in non-English pages with few words that overlap into English, those pages will not show up in the study.
This study essentially determines that Google has a larger database of random, obscure English language words. Consequently, they demonstrate that Google is the superior search engine for finding obscure, random English words.
One additional check that they could have thrown in would be how many of the pages in the links presently deliver 404 errors. That would have been far more interesting to me than how well the search engines do at finding obscure and random English words.
Re:you are WRONG (Score:3, Interesting)
Also, you can see something to similar to stemming when you search for certain acronyms. Try searching for [lotr] or [ada]. It also performs searches for the full version of the acronym, as you can see by the bold query in the snippets and title.
Holy lack of IR stastics understanding, Batman! (Score:5, Interesting)
Precision is how many of the results that you return are correct. e.g. If Google returns 100 results and 10 of them are correct, then the precision on that query is 10%.
Recall is how many of the correct results you return. e.g. If Yahoo returns 100 results out of a total 1000 correct matches, then the recall on that query is 10%.
Information retrieval systems such as search engines balance these two metrics -- which are fundamentally at odds with each other -- to give the "best balance" in the eyes of the system's designers.
The NCSA study basically misses the effect this decision would have on perceived size of index.
A simple demonstration shows how it works.
First let's say both search engines have the same index size: 10B pages. Second, let's say both search engines have exactly the same apriori capability for precision and recall, but can tune for a preferred performance. Yahoo decides it wants to favor more precise results over more results recalled, at a 2:1 relative ratio compared to Google.
In that case, any given query will show half the hits from Yahoo as compared to Google. Concluding Yahoo's index to be half the size of Google's, given this result, would be incorrect.
Furthermore, without knowing the precision/recall performance of either system, they can only demonstrate a lower-bound on index size, and that certainly doesn't predict average or max index size.