This is the last post in our multi-part series on how search engines work; read the basics of how search engines work and the importance of visibility to the search spiders.

Search engine optimization strategy #1: Add a robots.txt to your site.

A robots.txt is a file that tells search spiders where they are and are not allowed to go on a site. Having a robots.txt, even a basic one, is considered to be a “sign of quality.” It shows that you acknowledge that search engines are allowed on your site and that they may have free access to it. Learn how to add a robots.txt file to your site.

Search engine optimization strategy #2: Use search engine-friendly CSS navigation.

These days, all of the cool navigational effects previously accomplished with JavaScript can be replicated using text-based CSS (Cascading Style Sheets). CSS navigation is much more search engine friendly in that the links, which sometimes look like images, are actually HTML text that can easily be crawled and indexed by the search spiders. Many types of JavaScript links cannot be read or followed by the spiders. If the links can’t be read, the pages can’t be indexed.

Search engine optimization strategy #3: Strengthen internal linking with a site map.

Having a good site map is one of the most important things you can do for your site visibility. A site map makes it easy for the search spiders to find all of the pages in your site and index them efficiently. Link to the site map in the footer, and be sure to include every page on your site in the map. Remember to update the site map whenever new pages are added to your site.

We hope you’ve enjoyed this series about how search engines work. Contact The Search Guru to plan your search engine optimization strategy today!

In a previous post, we covered the basics of how search engines work, and how they use search spiders to index the pages on your site. It’s very important that your pages become indexed because, if they aren’t, then those pages can’t show up in search results.

The number of pages that have been indexed by the search engines is referred to as your level of “visibility,” and this is another crucial part of your search engine optimization strategy – to become more visible.

Here are the types of questions that we get at The Search Guru – and their answers.

1. I want to become more visible. How can I tell which pages search spiders have indexed?

Check Google and Yahoo! separately. The numbers will rarely be exactly the same, but they should at least be in the ballpark of one another. If they’re not, then it could indicate a deeper issue.

To check the number of pages indexed by Google, go to http://www.google.com. In the search box type in site:yourdomain.com (omit the www reference).

You will see in the upper right hand corner a reference to “Results 1-X of about X.” The number that comes after “about” indicates the total number of pages Google has found on your site.

To find out how many pages Yahoo! has indexed, go to http://siteexplorer.search.yahoo.com and type http://yourdomain.com into the search box (again omit the www reference).

It’s important to note that these numbers will fluctuate constantly, as search spiders are always crawling and indexing. Don’t be alarmed if you see these numbers going up and down slightly – it’s just how search engines work.

2. Are those the only search engines I should check as part of my search engine optimization strategy?

Google and Yahoo! are going to give you the most reliable figures on indexed pages, so those are the most important to check – and, Google can provide even more detailed data.

Google has two different indices that it pulls results from: the main index and the supplemental index. Google pulls results for searches from both indices, but gives preference to the main index; therefore you want as many pages as possible to be in the main index.

Pages are placed into the supplemental index when Google considers them to be weak or not as important. These pages also tend to have deeper issues, such as a lack of linking, little content or duplicate content.

To look at the number of pages you have in the supplemental index, type site:domain.com/* into the Google search box. This will give you the number of pages in the main index. Subtract that number from the total number of pages indexed to find the number of pages in the supplemental index. Then, calculate the percentage of your pages in the supplemental index; you want to shoot for no more than 80% of your pages being there.

Watch for our next post, where we’ll address the factors that can affect whether or not the search spiders will crawl your pages.

This is the first post in our multi-part series that deconstructs the technical how and why of SEO in a way that non-technical users can understand.

How search engines work

The major search engines (Google, Yahoo and Bing) use what are commonly called “search spiders” or “search engine spiders” to examine web pages. They “crawl” or “spider” the web and index their results so that people can search through what they have found.

Search spiders and engines are very simple creatures, overall, and search engine optimization strategy – or SEO – is an umbrella term for the group of techniques that make it easier for the spiders to find and properly index pages on your website. But, first – more about the spiders.

What do the search spiders do on my site?

A search engine spider visits a web page, reads it, and then follows links to other pages within the site. The spider returns to the site on a regular basis to look for additions and changes.

Spiders are given names by the search engines so that they can be easily identified in web site logs. It lets you know which spiders are crawling your site and how frequently. The more frequently a spider visits a page, the more important the spider thinks the page is. Common search engine spider names are:

Google: Googlebot
Yahoo: Yahoo Slurp
Bing/MSN: MSNbot

If you go into your site logs, you’ll see a string of text that likely has a date, time, a reference to the URL crawled, and a citation as to what was searched. So if Bing crawled a certain URL, you’d see a log of it along with a reference such as: (+http://www.bing.com/msnbot.htm).

What happens after my page is crawled by one of the search spiders?

The spider places the information into the search engine’s index. The index functions as a master repository of every web page that the spider finds. Until a page is indexed, it is not available to those searching with the search engine.

When a user searches and gets results, they are essentially getting a ranked list of what spiders have found. Each search engine has a program that sifts through the millions of pages recorded in the index to find matches to a search and rank them in order of what it believes is most relevant, based on the words the searcher typed in.

Now that we have a foundation for understanding how search engines work, we will delve further into indexing, and highlight the important role that search engine optimization strategy plays in getting your site properly indexed.

It has been more than a year and a half since Google launched Google 2.0, otherwise known as Google Universal Search. If that term doesn’t mean anything to you, don’t worry, you’re not alone! If you’re like most people, you probably didn’t even know it existed. And maybe that’s just what Google had in mind.

What is the Google Universal Search Engine anyway?

Google’s universal search engine now integrates search engine results from its regular “horizontal” search engine along with its many vertical search engines, which you may not have known existed. If you take a look at Google’s home page, you will notice the links along the top left for Images, Maps, News, etc. These are specialized, or “vertical,” search engines that deliver more specific search results from within a particular topic.

With Google Universal Search, the listings presented are a blend of search engine results, consisting of a mix of “regular” results and more specialized results from the vertical search engines.

So what does this mean with my search engine results?

Basically, Google has done us all a favor by choosing the most relevant order of Google search results for us from all its engines behind the scenes. Have you ever searched for a particular topic and gotten a bunch of frustrating, non-relevant sites? Google’s universal search engine attempts to place the most relevant results first, since most searchers will generally choose to click on one of the first few results.

The order of Google search results will vary based on a formula that determines the relevancy of the search results. Sometimes the vertical search engine results will be at the top, or they may be further down the page, below a few of the regular results.

Chances are you won’t notice either way. And it appears that the folks at Google don’t mind – as long as you keep using Google to get your search results.

I will admit, the financial world is a scary place right now. The markets look more like roller coaster schematics than financial statements. Not to mention the doom and gloom from every news source on the planet. It is enough to make anyone want to take their online marketing budget and crawl into a deep hole. But I am here to tell you: Don’t Do It. The impacts of the internet on economy outlooks are huge and you can benefit from them.

Why you should be increasing your online marketing budget right now.

Businesses all over the world are struggling because the general public has tightened its belt around its collective wallet. But, that is not to say that people have stopped buying things. They have not. They are simply looking to buy the same things cheaper. While high end and mid level restaurants are losing sales, places like McDonald’s and Heinz are seeing an increase in profits.

Why is this? Because people are opting to eat either at the cheaper restaurants or to cook at home. What else will these same people do to save money? One of the major impacts of the internet on the economy will be that they will be doing more of their shopping online. Will they be able to find you when they go shopping?

Established sites should take a look at their online marketing budget in segments. Where is your money going?

The internet will be the best, last place that many shoppers will go to find the best price. This means that your online marketing budget is the worst, last place to cut money from. The rule of the internet economy is that the more you cut your budget, the less likely you will be found.

Look closely at your online marketing budget and make sure you are making the best use of the money. A sample online marketing budget might contain 30% for SEO, 60% for PPC and the remaining 10% for local and social marketing. This will make sure that when your customer price shops for your service or product, you will at least be in the running. Cut your online budget, and you will not have the chance to woo the customer.

New websites should have internet marketing money built into their web design budget as well.

If you are a company launching a new site in the current economic environment, don’t skimp on internet marketing in your web design budget. While SEO may seem like a convenient place to save some money, it will hurt you in the long run.

The internet is not the “Field of Dreams.” If you build it, they will not come unless you do some internet marketing. Make sure the money is there for this.

The economy is scary right now, but the internet economy is still going strong. Use this to help the rest of your company weather this economic storm. Let me know what you are doing to help stretch your online marketing budget.

Do you know that a single dissatisfied customer looking for where to file an online consumer complaint can rob your company of profits? Learn how to use a little brand marketing online to protect your company’s good name.

Last time, I talked about why your online brand matters. In response, a reader asked:

Will you please give me suggestions to push down negative content in SERP?

This is an excellent question about how business owners can combat stores’ online complaints

Reputation management for brand marketing online starts with putting your company’s name on as many sites as possible.

Some possibilities include:

• Setting up a MySpace, FaceBook or LinkedIn page under your company’s name
• Uploading a video about your company at YouTube, AOL Video or Yahoo! Video
• Opening an eBay, Amazon or Flickr account with your company’s name as the username

Constantly check sites about where to file an online consumer complaint to head off bad reviews.

If you find a bad review, deal positively and publicly with your store’s online complaints to resolve the problem.

When potential customers are comparing online brands and they see a bad review, the damage to your brand will be less harmful if they see that you have taken the time to respond positively to your store’s online complaints. When they are comparing online brands, they are looking for good customer service, which includes resolving any problems they might have.

Brand marketing online is easier if you are proactive about it.

Online brand protection can help keep your company’s good name safe and on top when your customer’s are comparing online brands. What have you done to keep your online brand looking positive in the search engines?

On the internet you will find many articles on how to search the internet, but many are not related to finding out more about your own site. Do you know things like:

• How many quality links Google thinks you have coming into your site?
• How many pages from your site Google has indexed?

These bits of information can be useful to you in your SEO efforts and can be found by using advanced Google search operators. Let’s look at how to effectively use a search engine to find out more about your site.

Instructions on how to search the internet for the number of quality links coming into your site.

The link: query will tell you how many quality links Google thinks you have coming into your site and what those links are. Note that we said quality links, not all links. The link: query will not show all links to your site, just some of the good ones.

To use the advanced Google search link: query, simply go to Google’s search box and type in link:yoursite.com and hit enter. The list of links will then be displayed.

How to effectively use a search engine to find the number of pages indexed.

To find the number of pages on your site indexed by Google, go to Google’s home page and type in site:yoursite.com. All of the pages that Google has indexed from your site will be displayed.

It is important to check the number of pages that Google reports as being indexed against your own knowledge of the site. If you know your site has roughly 500 pages and a site: query shows that Google only has 100 pages indexed, then Google may be having trouble crawling your site. If the site: query reports 1000 pages indexed, then you may have a duplicate content or a canonical issue on your site. Both of these problems need to be addressed.

There are many other available articles on how to search the internet, to learn more about advanced Google search, and how to effectively use a search engine. Learning more about how search engines work will make using them to enhance your SEO more effective.

There are several methods to create an html drop down menu (also called a fly-out menu) on a site. The three main ways are:

• Flash menus
• javascript drop down menu
• CSS drop down menu

Flash menus
Flash menus are the poorest choice as search engine friendly drop down menus. While search engines have gotten better at reading links and text in Flash, they still have trouble with it. Because of this, you should not use Flash menus for navigation on a site.

JavaScript drop down menu
JavaScript is the most popular way to create drop down menu code. But, with JavaScript, there is a right way to create an SEO friendly JavaScript drop down menu and a wrong way. If you are not familiar with programming, the easiest way to tell a good JavaScript drop down menu from a bad JavaScript drop down menu is to look for the code “href.” If the JavaScript drop down menu code your developer wants to use contains href codes, then it is the kind of JavaScript a search engine can follow. If the href codes are not there, then a search engine will not be able to follow the links in the drop down menu.

There is one issue, though, with the search engine friendly JavaScript. The code used can be complicated and, the more complicated the code, the more chances you have that a search engine robot will fail while trying to decipher that code.

CSS drop down menu
The best option is to use a CSS drop down menu. To a search engine, the links in the code appear to be no different than normal links. The CSS drop down menu effect is done with styles on the style sheet, meaning that much of the code that powers the drop down effect is not even on the page.

Adding an html drop down menu to your site can be an excellent way to give your visitors more options without taking up additional space. Just make sure that your drop down menu code is created so that search engines can help visitors find you in the first place.

Why Google universal search affects your company
Since Google has replaced traditional search with Google Universal Search (which is a mixture of tradition search, Google Local maps, video, images and music related to a search) many changes have occurred in Google’s search results pages. One of those changes is that Google now strives to show the address of a company along with the listing of a company in the Google Universal Search results. Occasionally, if a company has more than one location, this can result in the wrong local Google listing being identified as the main location for the company. This can cause confusion for your customers and visitors. This is something you need to fix.

Steps to fix your company’s address in Google Local Maps
It is a simple procedure to fix this in both Google Universal Search and Google Local. First, you will need to make sure that you have declared your ownership of the Google Local listing for the headquarters. This can be done by going to Google Local and doing a search for the headquarters’ location in Google Local. Once you have found your company headquarters’, follow these directions:

1. Click on the link for “more info” in your company’s headquarters listing.
2. Then click on “Add or edit your business.”
3. Choose “Edit my business information” and click “continue.”
4. Fill out the information required by Google and the description field as well. You may also fill out any other fields you would like that are offered.
5. In either the company name field or the description field, indicate that this location is the headquarters. You must use the word “headquarters” in one of these two fields.

After you have completed the forms, Google Local will send a postcard to the company’s headquarters location to verify that the company did indeed authorize these changes. Be sure to look for this postcard and, when it arrives, enter the verification code according to the instructions on the card. Once the verification code has been entered, Google will update the listings in both Google Local and in the Google Universal Search results page.