Web Page Relevance

How do search engines rank relevance? While this is a closely guarded secret, there are some generalities that can be determined. Location of a word or phrase is a factor that most search engines will consider relevant. Pages with keywords appearing in the title are assumed to be more relevant than others to the topic, as well as pages with keywords that appear near the top the page. 

Frequency is the other major factor in how search engines determine relevancy. A search engine will analyze how often keywords appear in relation to other words in a web page. Those with a higher frequency are often deemed more relevant than other web pages.

Proximity is another indicator of relevance. If those words are found close together in a document, that document is assigned a higher weight than one in which the words appear scattered farther apart. When a user searches for a number of words, and In the document in which the words are found close together, it's more likely that they are being used in the same context as the user meant. For example, the query "What is love?" is more likely to be answered by a document including the phrase "what love is" or "love is what" than a document in which the three words appear widely separated.

One must consider other variables in addition to the position, frequency, and proximity mix. Some search engines index more web pages than others, and that some search engines also index web pages more often than others. The result is that no search engine has the exact same collection of web pages to search through. Search engines may also give web pages a "boost" for certain reasons. For example, Excite uses link popularity as part of its ranking method. It can tell which of the pages in its index have a lot of links pointing at them. These pages are given a slight boost during ranking, since a page with many links to it is probably well-regarded on the Internet. Some Internet search engines are accepting payment for a boost in the relevance ranking of a site. 

Some hybrid search engines, those with associated directories, may give a relevancy boost to sites they've reviewed. The logic is that if the site was good enough to earn a review, chances are it's more relevant than an unreviewed site. Meta tags are what many web designers mistakenly assume are the "secret" to propelling their web pages to the top of the rankings. HotBot and Infoseek do give a slight boost to pages with keywords in their meta tags. But Lycos doesn't read them at all, and there are plenty of examples where pages without meta tags still get highly ranked.