I have been a cyber citizen now for many years. I was lucky enough to be part of the internet revolution from quite early days as far as the UK goes. I have a lot of experience of using the internet as a 'surfer'.

What amazes me is the growth of the market relating to the provision of search engines services.
I remember when Yahoo! was just about the only search. These where the days of unbiased search results. The search engine market is now big business, and payment to be listed in one way or another is common.

Google, although apparently providing unbiased results, allows PPC or Pay Per Click advertising by its AD Words campaign. When searching on Google you may notice the small advertisement boxes on the right of the returned results. These are free to display but incur a cost to the website owner/advertiser when clicked. Hence pay per click!

Other search engines like Yahoo provide a pay to be included service. Where you may not actually be paying to be listed in search engine results you will be manually approved (or not!) to be included in there yahoo directory for a fee.

But things look like they are due to change. With income now viable for search engine providers, other big players are on the horizon all wanting a slice of the cake. MSN has recently stopped using other search engines results on its pages and has replaced them with its own search engine.

Both MSN and Yahoo now appear to be cataloguing the internet like crazy trying to catch up, and over take, the biggest search engine we all loving call Google.

As a web designer/web application builder and search engine optimisation fanatic I watch closely the log files of my many websites. These files can tell you a host of things. One interesting thing to follow is the visits your site gets from search engine bots (or spiders). These are the automated parts of a search engine that travel around the internet "reading" pages and deciding what they contain, how important they are, and how they sit in relation to other pages and sites in the tangled web called the internet.

MSN for one appears to be visiting all my sites very frequently. Most of the time visiting up to twenty pages per visit. This compares to googles one to three pages when it can feel like it. Yahoo too seems to be doing the same as MSN.

With these two search engines spidering the internet like crazy, it could mean that google has to start worrying about its top spot.

This my friend, should start you thinking. If you own or run a web site, and search engine traffic is important to you, is it friendly to Yahoo/MSN two search engine spiders? Do they like the same things Google does? If Google disappeared tomorrow, would your business still be able to exist?

Food for thought!

About the Author
From the early days of the internet, Darren has always been a part of it in one way or another. A typical "how's it work" kinda guy, he likes to take web pages apart to see how they work. This understanding, and the relationship of how search engines look at web sites, Darren now spends most of his time "Optomising" his many sites. His constantly changing site lives at http://www.netneo.co.uk.