XML Sitemap

A list of your pages written for search engines, saying what exists on the site and when each page last changed.

Also known as sitemap.xml site map file search engine sitemap

Definition

An XML sitemap is a list of your pages written for machines. It usually sits at an address ending in sitemap.xml and names every page you want found, often with the date each one last changed. Nobody browses it for pleasure. Its only reader is the program deciding which parts of your site to visit next.

Robots.txt is a different file with the opposite job. The sitemap says here is everything worth reading. Robots.txt says please stay out of these folders. One invites, the other refuses. They sit side by side at the top of the site, and often the same person edits them on the same afternoon. That is where the trouble starts.

The accident is always the same. A developer blocks a section while it is being tested. Two lines in robots.txt, and the whole shop folder is off limits. The work finishes, the block stays, and months later the owner cannot understand why the products vanished from search while the About page is fine. Nothing broke. The site politely asked Google to stay out. Linkysoft reads that file first on any site handed over to us, because it explains a surprising number of mysteries.

A sitemap does not force anything. Listing a page is a suggestion, and Google is free to ignore it if the page says little. The other common fault is a sitemap made once and never touched again, still naming pages deleted last year. Every visit a crawler spends on a dead address is a visit it does not spend on your new ones. The site should write its own sitemap, every time a page is added or removed.

Small sites live happily without one. If every page can be reached by clicking links from the home page, a crawler will find them on its own. A sitemap earns its keep on a large catalogue, a news section, or pages nothing links to yet. Any website Linkysoft designs and builds writes its own sitemap automatically, and the same applies to a bigger web application where pages appear and disappear every day. Before you sign off a new site, open the sitemap address and the robots file in your browser and read them yourself. Both are plain text, and both are worth two minutes of your time.

Questions about XML Sitemap

Do I need an XML sitemap for a small website?
Often not. If every page can be reached by clicking from the home page, search engines will find them. A sitemap matters most on large or very new sites.
What is the difference between a sitemap and robots.txt?
A sitemap lists the pages you want found. Robots.txt names the places you want left alone. One invites, the other refuses.
Where should the sitemap file be?
At the top level of the site, usually at an address ending in sitemap.xml, and named at the bottom of robots.txt so search engines find it on their own.
Will a sitemap make my pages rank higher?
No. It helps pages get found and checked sooner. Where they rank depends on the page itself, on its words and on who links to it.
Do I have to update the sitemap by hand?
You should not have to. A well built site writes its own sitemap every time a page is added, changed or removed, and a hand-made list goes stale fast.

Still not sure how this applies to your project?

Tell us what you are building and we will answer in plain language.

Start a project Contact us