Your Sitemap Might Be Lying to Google: The Technical SEO Audit We Ran on Our Own Site

Most SEO advice is about content: keywords, headlines, internal links. That stuff matters. But underneath it sits a layer of plumbing that most site owners never look at, and when that plumbing is wrong, good content gets discounted for reasons that never show up in a page editor.

We recently audited our own site, webglo.org, and found several of these problems. None of them were visible to a human visitor; every page looked fine in a browser. This post walks through exactly what we found, how to check for the same issues on your site in about 20 minutes, and how to fix them.

Problem 1: Every URL in Our Sitemap Was a Redirect

Your XML sitemap (usually at /sitemap.xml) is a list of URLs you are asking search engines to index. The implicit promise is: these are the real, final addresses of my pages.

Our site is served from www.webglo.org. Visitors who type webglo.org get a permanent redirect to the www version. That’s a normal setup. The problem was that our site generator was configured with the bare domain as the site’s base URL, so every single entry in the sitemap looked like this:

https://webglo.org/blog/

…which immediately redirected to:

https://www.webglo.org/blog/

Google follows redirects, so nothing was technically “broken.” But a sitemap full of redirects tells Google the sitemap is stale or carelessly maintained. In Search Console, these URLs show up under Pages → “Page with redirect” rather than as indexed pages, and Google has to do extra work to figure out which address you actually mean.

How to check yours

Open your sitemap in a browser, copy one URL, and check its status code. The easiest way without installing anything is your browser’s developer tools (Network tab → reload → look at the first request’s status). Or, if you are comfortable with a terminal:

curl -sI https://yourdomain.com/some-page/ | head -1

You want HTTP/2 200. If you see 301, 302, 307, or 308, the sitemap is pointing at a redirect.

How to fix it

Find where your platform defines the site’s canonical address and make it match the domain visitors actually land on:

Jekyll / Hugo / Eleventy: the url (or baseURL) setting in your config file.
WordPress: Settings → General → WordPress Address and Site Address. Both should match the final domain, including www or not.
Hosted builders (Wix, Squarespace, Google Sites): set your primary domain in the domain settings. The builder generates the sitemap from it.

Then resubmit the sitemap in Google Search Console.

Problem 2: Canonical Tags Pointed at the Wrong Host

A canonical tag is a line in a page’s <head> that says “this is the official URL of this content”:

<link rel="canonical" href="https://www.example.com/pricing/" />

Because ours were generated from the same misconfigured base URL, every page on www.webglo.org declared its official address to be a URL on webglo.org, which redirects back to www. Each page was effectively saying “the real version of me is somewhere else,” and that somewhere else said “no, it’s back there.”

Google usually resolves this loop sensibly, but you are leaving the decision to Google. Search Console reports it as “Duplicate, Google chose different canonical than user.” When you see that status, it means Google decided your canonical tag was wrong and ignored it.

We also found that our pages were emitting the canonical tag twice: once from our template and once from an SEO plugin. Two identical tags are harmless, but two different tags (common after a theme change) make Google ignore both.

How to check yours

View the page source (Ctrl+U / Cmd+Option+U) and search for canonical. Confirm that:

There is exactly one canonical tag.
Its URL uses the same protocol (https), host (www or not), and trailing-slash style as the address in your browser’s address bar.

Problem 3: Files That Were Never Meant to Be Public

This one was a little embarrassing. Our site’s repository contains a short Markdown file of internal notes for the tools we use to maintain the codebase. Static site generators like Jekyll will happily turn any Markdown file into a web page unless you tell them not to, so that internal file was live at a public URL and listed in our sitemap.

It wasn’t sensitive, but it was exactly the kind of page that makes a site look unmaintained: a page of build notes, with no navigation context, sitting next to our real articles.

We also had about a dozen demo template sites (a sample restaurant, a sample dog walker, and so on) that we use to show clients design options. They are useful as demos, but to a search engine they look like thin, fake business pages. A crawler has no way of knowing they’re demos.

How to check yours

Read your sitemap from top to bottom. Every URL should be a page you’d be happy for a stranger to land on from Google. Look especially for:

Old test pages, “sample page,” “hello world,” or staging copies
README, changelog, or notes files rendered as pages
Thank-you / order-confirmation pages (these should be noindex)
Demo or template content
Duplicate legacy URLs (for example /privacy-policy.html and /privacy-policy/)

How to fix it

You have two tools, and you usually want both:

Remove the page from the sitemap. In Jekyll, add sitemap: false to the page’s front matter, or exclude the file from the build entirely in _config.yml. In WordPress, most SEO plugins have a per-page “exclude from sitemap” toggle.
Tell search engines not to index it. Add <meta name="robots" content="noindex"> to the page, or, for whole folders of static files you don’t want to edit one by one, send an X-Robots-Tag: noindex HTTP header from your host for that path.

Don’t use robots.txt alone for this. robots.txt stops crawling, but a blocked URL can still appear in results if other pages link to it. noindex is the instruction that actually keeps a page out of the index, and Google has to be able to crawl the page to see it.

Problem 4: Debug Output in Production HTML

Our blog index contained a leftover HTML comment from a developer debugging pagination:

<!-- DEBUG: nil -->

Visitors never see HTML comments, and on its own this is trivial. But it’s a symptom worth looking for, because debug output sometimes includes far more than nil: full data dumps, internal paths, or API responses. View source on your key templates (home page, a blog post, the blog index) and search for “debug”, “todo”, and “test”.

Problem 5: ads.txt (If You Run Ads)

If you run display ads, ads.txt is a plain-text file at the root of your domain that lists which ad networks are authorized to sell your inventory. It exists to stop fraudsters from selling fake ad space that claims to be on your site. A Google AdSense entry looks like this:

google.com, pub-0000000000000000, DIRECT, f08c47fec0942fa0

Two things to verify:

The file is reachable at https://yourdomain.com/ads.txt and returns plain text (a redirect from the bare domain to www is fine; ad crawlers follow it).
The publisher ID exactly matches the one in your ad account. A typo here won’t produce an error anywhere; your ads just quietly earn less, or not at all.

The 20-Minute Checklist

Here is the whole audit condensed:

Pick your canonical host (www or not) and confirm the other one redirects to it with a single 301/308.
Check five sitemap URLs and make sure each returns 200, not a redirect.
View source on three page types (home, article, listing) and confirm there’s exactly one canonical tag, pointing at the URL in the address bar.
Read the whole sitemap and remove anything a stranger shouldn’t land on.
noindex utility pages like confirmation pages, demos, and internal tools.
Search your templates’ rendered HTML for leftover debug output.
Verify ads.txt if you run ads.
In Search Console, open Pages and read the “Why pages aren’t indexed” table. Each reason links to example URLs, and it’s the fastest way to find problems you didn’t know to look for.

💡

Pro Tip

Fix the source, not the symptom. All three of our biggest issues came from a single wrong setting: the site’s base URL. When Search Console shows the same problem on many pages, look for the one shared setting or template that generates all of them.

Why This Matters More Than It Seems

None of these problems will tank a site on its own. They matter because search engines and ad networks are trying to answer a broader question: is this site actively and carefully maintained? A sitemap full of redirects, conflicting canonicals, and stray internal pages all point toward “no,” even when the content itself is good.

The fixes are small and mostly one-time. If you write content regularly, spend 20 minutes on this checklist once a quarter. It’s cheaper than writing another article, and it makes every article you’ve already written easier for Google to trust.

❓ Frequently Asked Questions

Is www or non-www better for SEO?

Neither. Google treats them as two different hosts, and what matters is that you pick one and make everything agree with it: the redirect, the canonical tags, the sitemap, internal links, and the property you verify in Search Console.

How long does it take Google to pick up sitemap and canonical fixes?

Usually days to a few weeks, depending on how often Google crawls your site. Resubmitting the sitemap in Search Console and using URL Inspection → Request Indexing on your most important pages speeds it up.

Do I need a sitemap if my site is small?

Google can find pages on a small, well-linked site without one, but a sitemap is still cheap insurance. A wrong sitemap is worse than none, though, so if you have one, make sure every URL in it returns a 200.

Want a Second Pair of Eyes?

We run this same audit on client sites before every launch. If Search Console is showing errors you don't understand, we can help you read them.

Ask About an Audit See Our Services