If your pages can’t be seen or read by search spiders, then those pages can’t show up in search results! Find out more from The Search Guru.

In a previous post, we covered the basics of how search engines work, and how they use search spiders to index the pages on your site. It’s very important that your pages become indexed because, if they aren’t, then those pages can’t show up in search results.

The number of pages that have been indexed by the search engines is referred to as your level of “visibility,” and this is another crucial part of your search engine optimization strategy – to become more visible.

Here are the types of questions that we get at The Search Guru – and their answers.

1. I want to become more visible. How can I tell which pages search spiders have indexed?

Check Google and Yahoo! separately. The numbers will rarely be exactly the same, but they should at least be in the ballpark of one another. If they’re not, then it could indicate a deeper issue.

To check the number of pages indexed by Google, go to http://www.google.com. In the search box type in site:yourdomain.com (omit the www reference).

You will see in the upper right hand corner a reference to “Results 1-X of about X.” The number that comes after “about” indicates the total number of pages Google has found on your site.

To find out how many pages Yahoo! has indexed, go to http://siteexplorer.search.yahoo.com and type http://yourdomain.com into the search box (again omit the www reference).

It’s important to note that these numbers will fluctuate constantly, as search spiders are always crawling and indexing. Don’t be alarmed if you see these numbers going up and down slightly – it’s just how search engines work.

2. Are those the only search engines I should check as part of my search engine optimization strategy?

Google and Yahoo! are going to give you the most reliable figures on indexed pages, so those are the most important to check – and, Google can provide even more detailed data.

Google has two different indices that it pulls results from: the main index and the supplemental index. Google pulls results for searches from both indices, but gives preference to the main index; therefore you want as many pages as possible to be in the main index.

Pages are placed into the supplemental index when Google considers them to be weak or not as important. These pages also tend to have deeper issues, such as a lack of linking, little content or duplicate content.

To look at the number of pages you have in the supplemental index, type site:domain.com/* into the Google search box. This will give you the number of pages in the main index. Subtract that number from the total number of pages indexed to find the number of pages in the supplemental index. Then, calculate the percentage of your pages in the supplemental index; you want to shoot for no more than 80% of your pages being there.

Watch for our next post, where we’ll address the factors that can affect whether or not the search spiders will crawl your pages.