Web search engines are the primary method for finding information in the vast World Wide Web, yet little empirical research of them has been conducted. This study examined how well search engines' relevancy ratings of Web sites matched users' ratings. Eighty-one people rated Web sites in three different topic areas that were delivered as top sites by Infoseek and/or HotBot. To ensure fair ratings, participants were unaware of the ratings assigned by the search engines, or even that search engines had generated the sites. Individual difference characteristics of the raters (e.g., browser-searcher, level of knowledge and interest in topic) were also collected. Agreement as to the top sites was quite low among search engines. However, people showed very high levels of agreement and reliability. Search engines' ratings did not correspond well with the raters' ratings. Nevertheless, Infoseek performed better than HotBot on every measure of agreement with user ratings.