XML Sitemaps: Guidelines on Their Use – Moz

Note: This is body-only HTML prepared for direct publishing. External source links and unwanted publishing artifacts have been intentionally removed.

If XML sitemaps had a dating profile, they would describe themselves as “helpful, misunderstood, and not here to do all the work.” That is the right energy, honestly. Too many site owners treat a sitemap like a magic SEO coupon. Upload file, summon rankings, retire early. Sadly, search engines remain stubbornly committed to reality.

An XML sitemap is not a ranking trick. It is a structured file that helps search engines discover important URLs, understand when meaningful updates happen, and crawl large or complicated websites more efficiently. Used well, it is a practical communication tool. Used badly, it becomes a beautifully formatted list of URLs you never wanted indexed in the first place. A sort of self-inflicted technical SEO comedy.

This guide breaks down what XML sitemaps do, when they matter most, what belongs inside them, which tags still deserve attention, and how to avoid the classic mistakes that make search engines sigh heavily into the void. If you have ever wondered whether your sitemap should include every URL, your tag archives, your filtered faceted pages, and that one staging page nobody admits exists, pull up a chair.

What an XML Sitemap Actually Does

An XML sitemap is a machine-readable file that lists URLs you want search engines to know about. In plain English, it is a roadmap for crawlers. Not the whole car, not the destination, and definitely not the engine. Just the roadmap.

Its job is to help search engines discover content more efficiently, especially when your site is large, new, media-heavy, or not perfectly linked together internally. It can also provide extra signals, such as when a page was last meaningfully updated. On modern websites, that matters because crawl budget and crawl scheduling are not random acts of destiny. Search engines prioritize based on what looks useful, fresh, and worth revisiting.

What a sitemap does not do is guarantee indexing. If a page is thin, duplicate, blocked, redirected, broken, or marked noindex, your sitemap does not get to overrule that. A sitemap is a suggestion file, not a royal decree. Search engines still evaluate the quality, accessibility, and relevance of the page itself.

When You Truly Need an XML Sitemap

Technically, not every website needs one. A small website with excellent internal linking and a clean structure can often be crawled just fine without an XML sitemap. But “can survive without one” is not the same as “should ignore one.” In practice, most websites benefit from having a well-maintained sitemap because it improves discoverability and gives site owners a cleaner way to monitor indexing behavior.

XML sitemaps are especially useful when:

• Your site is large and contains thousands of URLs.
• Your site is new and has few external backlinks pointing to it.
• Your internal linking is weak, uneven, or complicated.
• You publish lots of images, videos, or news content.
• You run an e-commerce site with constant inventory changes.
• You manage multiple content types, languages, or regional versions.

Imagine an online store with 120,000 product URLs, seasonal collections, pagination, blog content, and localized pages. That site should not rely on hope and menu navigation alone. It should use segmented XML sitemaps for products, categories, blog posts, and international content, all tied together with a sitemap index. That setup is cleaner, easier to audit, and much friendlier to search engines than one giant digital junk drawer.

What Belongs in Your XML Sitemap

This is where a lot of websites wander into trouble. The best XML sitemap is not the one with the most URLs. It is the one with the right URLs.

Include pages that are indexable, canonical, valuable, and return a proper 200 status. Those are the pages you genuinely want search engines to crawl and consider for search results. If a URL is not good enough to rank, it is probably not good enough to live in your sitemap either.

Good candidates for inclusion:

• Core landing pages
• Category pages
• Product pages you want indexed
• Blog posts and evergreen resources
• Important image, video, or news content when relevant
• Canonical regional or language versions

URLs you should usually leave out:

• Redirected URLs
• 404 or soft-404 pages
• Noindex pages
• Blocked-by-robots pages
• Duplicate URLs and parameter variants
• Internal search result pages
• Login, account, cart, or checkout pages
• Thin archives or tag pages with little standalone value

One of the most common sitemap mistakes is including every URL a CMS can possibly generate. That approach is like inviting search engines to a dinner party and then seating them in the garage, laundry room, and supply closet. Be selective. A sitemap should highlight your best crawl targets, not every technical byproduct of your platform.

The Tags That Matter Most

The essential XML sitemap tag is the URL location itself. Without the page location, there is no sitemap, just a philosophical gesture. Beyond that, there are optional tags that can still be useful, but not all of them carry equal weight.

<loc>

This is the star of the show. It tells search engines the exact canonical URL you want listed.

<lastmod>

This tag is useful when it reflects a real, significant page update. That means changes to the main content, meaningful structured data, or core links. It should not change because the footer year updated itself at midnight like an overachieving intern. If your lastmod values are inaccurate or constantly refreshed for trivial edits, search engines are more likely to ignore them.

<changefreq> and <priority>

These tags are the former prom kings of sitemap conversations. They still appear in plenty of examples, but they are no longer the stars people think they are. Search engines, especially Google, do not treat them as powerful directives. That means you should not waste energy hand-tuning them as though you are playing chess with an algorithm from 2009. Keep your attention on crawlable, canonical, high-value URLs and accurate lastmod data instead.

File Size Limits and Sitemap Indexes

XML sitemaps have limits, and search engines are quite serious about them. A single sitemap file should contain no more than 50,000 URLs and should remain under 50 MB uncompressed. If your site exceeds that threshold, split the URLs into multiple sitemaps and organize them with a sitemap index file.

That is not a punishment. It is actually a good thing. Multiple sitemaps make your setup easier to manage and easier to diagnose. For example, you can separate:

• Products
• Categories
• Blog posts
• Videos
• Images
• Regional or language sections

This helps you spot issues faster. If your blog sitemap shows healthy indexing but your product sitemap looks like a haunted house, you instantly know where to investigate. Segmenting sitemaps is not just tidy. It is strategic.

Best Practices That Still Hold Up

1. Generate your sitemap dynamically whenever possible

Static sitemaps age badly. The moment pages are added, removed, redirected, or set to noindex, a manually maintained file starts drifting away from reality. Dynamic sitemap generation keeps your sitemap synced with your site. That is especially important for publishing sites, e-commerce stores, and anything with regular content turnover.

2. Keep the sitemap in a logical location

Many websites place the file at /sitemap.xml or use a sitemap index such as /sitemap_index.xml. Simplicity helps. Search engines like predictable file locations, and site owners like not playing hide-and-seek with critical infrastructure.

3. Reference it in robots.txt

Even though you can submit your sitemap directly in webmaster tools, it is still smart to reference the sitemap in robots.txt. It gives crawlers an additional path to discover it and keeps your technical setup more transparent.

4. Submit it through Google Search Console and Bing Webmaster Tools

Do this once when the sitemap goes live, and then monitor it. You do not need to keep ceremonially resubmitting the same sitemap every month like you are renewing a library card. Search engines will revisit it. Your job is to keep it accurate and review the reports.

5. Use sitemap extensions when they genuinely help

If your website relies heavily on image search, video visibility, news content, or multilingual versions, sitemap extensions can help search engines better understand those assets. They are particularly useful for media publishers, publishers in Google News workflows, and international sites managing alternate language versions at scale.

6. Monitor sitemap reports like a grown-up SEO

The sitemap itself is only half the story. The reports in Search Console and Bing Webmaster Tools can show whether submitted URLs are being discovered, crawled, and indexed the way you expect. If the ratio looks ugly, the sitemap may not be the problem. The real issue could be quality, duplication, canonical conflicts, rendering problems, or crawl restrictions.

Common XML Sitemap Mistakes

Here is the part where good intentions go to fight redirect chains.

Including non-canonical URLs

If the sitemap lists a URL but the page points to a different canonical version, you are sending mixed signals. Search engines dislike mixed signals the way accountants dislike mystery receipts.

Listing pages blocked by robots.txt

This is classic mixed messaging. You are saying, “Please crawl this page,” and “Do not crawl this page,” in the same technical breath. Pick a lane.

Keeping deleted pages in the sitemap

When pages are gone, remove them from the sitemap. Old 404s, discontinued products, and retired campaign pages should not linger there forever like party guests who will not leave.

Using fake freshness in lastmod

If every page claims it was updated five minutes ago, search engines may stop trusting the signal. Accuracy beats enthusiasm.

Submitting sitemaps and ignoring internal linking

A sitemap is helpful, but it does not replace a strong site architecture. Search engines still rely heavily on internal links to understand structure, hierarchy, and importance. Think of the sitemap as support staff, not the CEO.

XML Sitemap vs. HTML Sitemap

An XML sitemap is built for search engines. An HTML sitemap is built for people. They are related, but they are not twins. More like cousins who show up to the same family gathering wearing very different shoes.

Most modern SEO strategies rely more heavily on XML sitemaps than HTML sitemaps. That said, HTML sitemaps can still be useful on large sites, help centers, archives, or complicated content hubs where users may benefit from an alternate navigation layer. Just do not mistake an HTML sitemap for a substitute for technical crawl guidance. It is not.

A Practical Example of a Smart Sitemap Setup

Let’s say you manage a national furniture retailer. Your site contains category pages, product pages, buying guides, store-location pages, videos, and seasonal campaigns. A smart XML sitemap setup might look like this:

• /sitemap_index.xml
• /sitemaps/products.xml
• /sitemaps/categories.xml
• /sitemaps/guides.xml
• /sitemaps/locations.xml
• /sitemaps/videos.xml

Each file contains only canonical, indexable, 200-status URLs. Discontinued products that now redirect are removed. Filtered URLs such as color, size, or sort-order parameters are excluded. Buying guides include accurate lastmod values when major content changes happen. The sitemap index is submitted to both Google Search Console and Bing Webmaster Tools, and the file is referenced in robots.txt. That is not glamorous, but it is clean, scalable, and effective.

Final Thoughts

XML sitemaps are not flashy. Nobody throws a party because the sitemap is valid and up to date. They probably should, but they do not. Still, this file remains one of the most practical tools in technical SEO when used with discipline.

The real value of a sitemap is clarity. It tells search engines which URLs matter, when important content changed, and how your site is organized at scale. Done right, it supports faster discovery, cleaner indexing analysis, and fewer technical misunderstandings. Done poorly, it becomes a tidy little file full of URLs you should have cleaned up three migrations ago.

So the goal is not “have a sitemap because SEO said so.” The goal is to maintain a sitemap that reflects the truth of your website. Keep it current. Keep it selective. Keep it aligned with canonicals, crawlability, and content quality. Then let it do what it does best: make a search engine’s job easier without pretending it can do yours.

Experience: What XML Sitemaps Teach You in the Real World

After working with enough websites, you start noticing that XML sitemap problems are rarely “sitemap problems” in isolation. They are usually site truth problems. The sitemap simply exposes them with less mercy. That is why sitemap audits are oddly revealing. You begin by checking a file, and twenty minutes later you are staring at redirected product URLs, orphaned blog posts, parameter chaos, inconsistent canonicals, and a robots.txt file that seems to have been written during a minor emotional event.

One common experience is discovering that a site owner believes every published URL deserves a place in the sitemap. In real projects, that almost never holds up. The sitemap should represent the best version of the site, not the entire history of every experiment, filter, tag page, and temporary landing page. Once teams start trimming low-value URLs from their sitemaps, they often realize the real issue was not “Google is ignoring us.” The issue was “we keep sending mixed signals and then acting surprised.”

Another lesson shows up during redesigns and migrations. Sitemaps become one of the fastest ways to catch damage early. If a new site launches and the sitemap suddenly includes noindex pages, broken canonicals, or huge numbers of redirected URLs, that is your warning flare. In that sense, the sitemap is more than a crawler aid. It is a quality-control checkpoint. A clean sitemap after launch is a reassuring sign. A messy one is the SEO equivalent of hearing a weird sound in your car and turning the radio up.

There is also a practical human lesson here: automation wins. Teams that rely on manual exports tend to forget updates, miss removals, and create bloated files full of technical leftovers. Teams that generate sitemaps dynamically through a trustworthy CMS, plugin, or custom logic usually perform better because the process reflects reality more quickly. Search engines like fresh, accurate data. So do the humans who have to explain indexing patterns in meetings.

And perhaps the biggest real-world takeaway is this: the best XML sitemaps are boring. They do not try to game search engines. They do not stuff in every possible page. They do not scream with fake priority values like everything is equally important. They quietly list the right URLs, stay updated, and support a healthy site structure. That kind of boring is beautiful. In SEO, boring systems often outperform dramatic ideas. A disciplined sitemap will not make headlines, but it can save a site from months of crawl waste, confusion, and self-sabotage. Which, in technical SEO terms, is basically a standing ovation.

SEO Tags