Have you ever wondered how a search engine works? It must be fascinating figuring out how this search tool could direct you to several websites that are relevant to your keywords. Or, have you experienced instances where the link that supposedly contains your keywords is not exactly what you have in mind? You would probably think that there must be something wrong with the search engine that it generated irrelevant results.

How does a search engine work?
Two things figure greatly in making search engines work effectively and efficiently: the electronic search spider and the sitemap.

What is a sitemap?
A sitemap is basically a page or pages that serve/s as a directory by listing all the links to all documents and files found in a website. It is not merely a random listing of links, but organized in such a way that it gives the web user an idea of how all the information that can be found in the site fits into an outline or framework. It is like viewing the table of contents of a book, or viewing the “concept map” of the site’s content.

What is a spider?
In SEO language, spider is not an animal found in your closet. This electronic search spider is actually a bot which collects data and copies content to be stored in the search engine’s database when keywords are fed into the search dialogue box. The spider reads the content of the site and sends another bot to follow the links and copy the content contained in them.

What purpose does a sitemap serve?
A sitemap like any other map gives directions to a navigator. It primarily targets search engine spiders so that they are properly directed to your site and to the links where keywords entered in the search dialogue appears.  As such, it is actually a useful tool in search engine optimization. A well organized site map would guide the spider to find the information it needs when keywords are entered during a search operation.

As an additional beneficial consequence, sitemaps have proven to be useful even to web users. Since a sitemap displays all the links to information found in a website, it helps the user to search for a particular topic in mind. Many users also use the sitemap to navigate between pages in a site.

What are the benefits of having a sitemap for my website?

1.    No page would be left unturned
Going back to the purpose of sitemaps, having one would mean faster and easier tracking and crawling of spiders all over your site. As a result, search engines would surely get to the view all the pages of your site and not just the pages containing random keywords.

2.    Easier navigation for site visitors
Once a web user has accessed your sitemap, they need not go back to the search engine page to look for what they need. If what they are looking for is in your site, then they would have an easier and faster way of locating it.

3.    Potential advertising value
If it so happens that a relevant product or service company reaches your site, then it would be easier for them to see how best they can position themselves in the different pages of your site as a paid page advertisement.

4.    Encourage greater traffic to your site
If your company website has a sitemap then potential buyers would have an easier time in accessing your latest products and services. Moreover, they would not miss out on any product that might be off future interest to them since the sitemap would display all information found the site.

How are sitemaps formatted?
There are at least three major types of sitemaps: indexed, full categorical, and restricted categorical. An indexed site map appears as an alphabetical listing or directory.
A full categorical map displays all links classified into categories; while a restricted categorical sitemap displays all links listed in a chosen category at a time. The full and restricted sitemaps are very similar except that the former displays all links in all categories all at once in a page, while the latter focuses only the links under the selected category for easier and less eye-straining viewing.

The most widely used format is the full categorical. Based on the results of a 1999 SURL study on sitemap designs, the full categorical format is most preferred by users since it is easier to search for topics within the site and it allows easier comparison between and among categories.

Some tips in setting up your sitemap

1.    Link the sitemap only to your homepage.
This is to ensure that the spider starts searching from your homepage down to all the pages listed in your sitemap. In this way, no page would be left unvisited by the spider.

2.    Do not go beyond 30 pages for a sitemap.
Large websites having 50 or more pages should not go beyond 30 since this adds more pages to the site, and might make search engines think that the sitemap is a link farm. Also, this prevents overcrowding of links which could be tiring to view.

3.    Check all the links listed in your sitemap.
It can be discouraging when you click on a link only to find out that nothing is displayed. Test your sitemap; click all links in every page to make sure that all links are indeed linked to the right page.

4.    Give keyword-rich titles to sitemap links.
Keyword-rich titles give your site more advantage in being searched properly under the right category. But be sure to have this sitemap link linked back to the sitemap (e.g. back to sitemap).

5.    Provide a short description for the links in the sitemap.
Doing so would give readers a better idea of what to find in the link and save them time on surfing.

6.    Be consistent in designing your sitemap with the other pages of the site.
Employ a recurring design and the same HTML template for all pages to establish identity and build character to your website.

Now that you have learned basic things about sitemaps, maybe it is time for you to build one for your site.

The Federal Trade Commission (FTC) has decided to abandon net neutrality and allow telecoms companies to charge websites for access.

The FTC said in a report that, despite popular support for net neutrality, it was minded to let the market sort out the issue.

This means that the organisation will not stand in the way of companies using differential pricing to make sure that some websites can be viewed more quickly than others. The report also counsels against net neutrality legislation.

“This report recommends that policy makers proceed with caution in the evolving dynamic industry of broadband internet access, which is generally moving towards more, not less, competition,” FTC chairman Deborah Platt Majoras wrote.

“In the absence of significant market failure, or demonstrated consumer harm, policy makers should be particularly hesitant to enact new regulation in this area.”

The report has caused outrage in the online community. Many are worried that any abandonment of net neutrality will harm competition, since it will allow big companies to outspend start-ups.

“Mostly the FTC suggests ways that the telephone and cable companies could have new ways to make money from content and applications providers,” said Art Brodsky, of internet advocacy group Public Knowledge.

“Or lower-income subscribers could be charged lower prices, subsidised by ‘prioritization revenues’ much as supported email services now provide free email accounts. Nowhere is there discussion of what the consumer gets out of the deal.”

      Now, if you havent been paying attention, Net Neutrality was/is an issue about access. With Net Neutrality in place, everyone has equal access to the Internet and the information on the Internet. With the FTC dropping its support of Net Neutrality, now the ‘big wheels’ on the internet, such as Google, Yahoo, AOL, will have the opportunity to basically pay so that their content gets to you faster than the average mom and pop website. The heavy hitters will have the ability to pay to make sure that their content is given priority through your ISP to get to you. So your gaming, your bit-torrents, your file downloads will have their bandwith somewhat throttled back so that you can get those fresh new ads from AOL, or so your Yahoo Toolbar will be able to make its oh so important conections back to their content servers to bring you all those bright and shiny buttons on your toolbar that you probably dont use anyway, if you even have it installed at all.

     In short, this is not a good thing for the consumer. Be prepared for a ruckus on the Internet, when people realize that their Internet enjoyment is now being choked down so that the admakers of the world can have better access to shove their shit down your throat, people will not be happy. I hope that people DO wake up and not just stick their head back in WoW and hope it all goes away or that someone else does the work for them.

If you need to do a little reading to catch up, here ya go:
Wikipedia on Net Neturality
Hands Off