About search engines
This was originally written for one of our customers. Because it is frequently
asked for by many parties it is here provided
Any remarks or suggestions are welcomed by e-mail.
Search engines
When addressing this subject one should realise that the Internet is a dynamic, developing and growing entity. As this is written it may already be outdated or overcome, bypassed or obsolete. Nevertheless, some basics.
- What are search engines?
- How do search engines index the Internet?
- How to control search engines?
- How to get a better ranking in searc hengines?
- Which search engines should a site be added to?
- Technical details
What are search engines and how do they work?
Search engines are basically huge databases that help you to find information
on the Internet. Since it is not possible for you to know exactly where the
information that you are looking for resides, one can always query these databases
to find and locate information.
The bigger global engines are www.google.com, www.hotbot.com, www.lycos.com,
www.altavista.com, www.infoseek.com, www.webcrawler.com, www.excite.com and
the like. National, global or specific area's of interest all have their own
search engines, there are quite a lot of them.
How do search engines index the Internet?
Well, basically by starting at the homepage of a site, any site, and indexing
that page. All the links that are found will be traced and indexed. So, if
the homepage has a link to a page with contact information, then that page
will be indexed as well. If this contact page offers a link to the company
that built the site, than that entire site will be indexed as well. You understand
this goes everywhere. This process is called spidering a website,
when a spider comes along the web and obtains information about linked pages.
Once your page or site is linked to from any other page on the Internet,
than it's just a matter of time and your site will be visited by search engines...
Once an engine has indexed a site it may take 2 weeks till 6 months before
it is actually shows up in the result listing depending on the search engine.
Every now and then, typically every two weeks, the search engines will revisit
all the indexed pages to check if they still exist and if so if the content
has changed. The index will be updated accordingly and any changes will be
available instantly. This implies that it takes time to get listed in search
engines, and also to get out, even though the site or pages may be non existing.
How to control search engines?
Various tools and methods are available to influence search engines in if,
and how a site is indexed. One is by means of a robots.txt file
in the root of the site, or META tags in the individual pages. Good reference
on his matter can be found at AltaVista or
at searchenginewatch.com and
many other search engines offer background information on this subject.
How to get a better ranking in search engines?
In other words, "How to get your site or pages best positioned in the
results to a query issued to a search engine?"
Specially here one faces the dynamic character of the Internet. There is no absolute answer to this question yet some basic guidelines are these.
- Title of the page
The title of the pages should specifically reflect the content of that page. Don't name all the pages "Company website ". - Links to your site
When another website links to your site, it is seen as an endorsement, a vote, for your site. Also, it will be more frequently visited by search engines, and be better positioned in the index.
Don't forget links inside your site to your other pages. A navigation bar should be standard on all pages, so if a search engine throws most of your pages out of it's index, then the one's that remain offer links to all other pages to be reindexed. - Page content
This may look simple and yet don't forget that search engines can't read images, javascript or flash movies. If you use images for headings and navigation buttons than be sure to add the ALT tag.
Those are the main ingredients that determine your ranking. Of course some fine-tuning can be done:
- Don't use frames
Search engines are primitive creatures and technologically compatible to NN2.0 or IE3.0 and these browsers can't read framed sites. If you do use them anyway be sure to add content to the <NOFRAME> container with links to all of the important individual pages.
Other con's to frames;- You can't bookmark a framed page, you'll end up bookmarking the homepage.
- Visitors coming from search engines end up in a page that is supposed to be framed yet isn't because they are referred straight to the right page. No navigation or context is available then.
- When having multiple browsers open at the same time, your page might turn up in the site of someone else...
- These items can be overcome with some clever techniques utilising JavaScript and PHP.
- Make the content accessible
Beware that SE's wont accept cookies, won't execute JavaScript, find frames a horde simply too high and can't submit forms. Several SE's refuse addresses that have parameters in the URL. Therefore, in case any of the before mentioned is used, navigation options have to be provided to all the pages you want to see indexed. - Frequency of update
A frequently updated site stands a better chance of not being forgotten or bypassed than those that are'last revised Aug 15, 1985' . You may refresh the files on the webserver regularly even though the content may not be changed. - Manually add your site to search engines
Nowadays companies offers services to promote your website to the leading 400 search engines for free! Beware, the rumour goes that several search engines have started refusing sites that are added this way, even deleting them from their index as they see it as a form of spamming. Therefore if you want a site added to a specific search engine do it manually, search their site for a 'add your site' link or use our listing. - Keywords META tag
In the keywords META tag one puts keywords associated to the content, and are the words likely to be used by others trying to find your content. Don't forget to put the antithesis there to not just let people find the bad PR sites.
You may find sites putting the same word several times in the META tag hoping for a better ranking, mind you this no longer works yet once did. If you put the same word more than twice in the keywords META that keyword will be ignored... this as an answer to sites trying to improve their ranking by keywords "sex, sex, sex, sex, sex, sex". - Description META tag
Typically these one or two sentences are shown by search engines in their result listing to a query. Be concise and simple, remember that people hate to read a lot.
Which search engines should a site be added to?
Well, this is usually overemphasised. The question "From where do you expect the majority of visitors?" should be leading in this consideration.
As guideline try this;
- The major American/global search engines;
People tend to grab the bigger ones and use them all the time. The top-ones are listed on our AddUrl page. - The major national search engines;
See above, the Dutch most famous are listed on our AddUrl page. - Specific interest search engines, if applicable.
If a site covers all aspects of dancing then it might be wise to find search engines that have specialised in that field. If the site is a 'general business site' then don't bother.
Remember that sooner or later your site will be indexed, whether you add it manually or not.
Technical details
For surfers with graphics turned off, you want both a background color and a background image. But if fonts have the same hex value, some search engines (HotBot is one) will penalize for spamming their index with hidden keywords, even with a different colored background image.
Example: if your background color is white and you use white letters in a navy table, or on a navy background image, you may trigger spam penalties that will reduce your page rankings.
META name="keywords" use up to 1,000 characters, including spaces and commas";