The test, which consisted of over 5,000 individual queries, looked at each search engine to determine how well it covered the top 100 Alexa-rated news, sports, TV, music and movie websites that have video content.
The basic procedure is simple:
- Select a video to search for.
- Search for it in a video search engine.
- Record if the video was found.
- Repeat steps 1 to 3 for more videos and more video search engines.
To develop the list of 100 websites Truveo selected the top 100 Alexa-ranked (http://www.alexa.com) video content websites with the highest traffic rank (as recorded on August 6, 2008) for each of the following categories: news, sports, TV, music and movies. These five lists were combined and sorted by traffic rank. Sites were then checked to ensure they had a reasonable number of videos and were appropriate for the study. The top 100 sites on this list were then selected for testing.
Then, the study selected 10 representative videos from each of the websites and queried each search engine for each video. If a search engine returned at least five of the 10 representative videos, it was counted as covering that website. Because the test was designed to measure coverage, the search engine needed to return the exact video from the original site in order to be counted. The queries were run between August 8, 2008 and August 28, 2008.
.