Sitemap lastmod: why false freshness is worse than none

A sitemap lastmod is a promise that something changed on a date. It is easy to emit, hard to keep true, and uniquely damaging when it is wrong: a sitemap that always claims freshness teaches a crawler that the field carries no information, and the site loses the one signal it added the field to send.

This article is about the three ways a lastmod goes wrong, why "no date" can be better than a false one, and how this site made the stale case fail a test instead of waiting for a search-console graph to notice.

What lastmod is for

A sitemap lists URLs. lastmod adds one claim per URL: when the page last changed. A crawler that trusts the claim can prioritise pages that changed since the last crawl and skip pages that did not. That is the whole mechanism — efficiency in recrawling, not a ranking boost.

The mechanism only works if the claim is accurate. A crawler that finds the field unreliable learns to ignore it, at which point the sitemap still lists URLs and still gets read, but the freshness hint is gone. The loss is quiet: no error, no warning, just a field that stops being used.

The three ways it goes wrong

The second one is the hardest to notice because nothing fails. A computed date is at least visibly noisy; a frozen date looks disciplined. This site lived through exactly that: the constant sat at one date through a week of real content changes, and Search Console ended up reporting 27 rule pages as discovered but not indexed while the sitemap told Google nothing had moved.

Three designs, and what each costs

1. A single hand-maintained date for the whole sitemap. Simple, and accurate only if someone remembers. The cost is discipline; the mitigation is a test that fails when content changes and the date does not. That is the design this site uses.
2. A per-URL date from the content system. More accurate, and more work: the date has to come from the same source that decides the page changed, not from the build clock. If the content is file-based, a frontmatter date; if it is a database, aupdated_at that is only written on real edits.
3. No lastmod at all. Honest when the pipeline cannot produce a truthful date. A sitemap without the field still lists URLs; it just does not make a freshness claim it cannot keep.

What does not work is the fourth option people reach for: a date that moves with every deploy because it is easy. That is not a freshness signal; it is a build artifact wearing one.

Making the stale case fail

The test in this repository does four things, and each one exists because the naive version failed in practice. It is scripts/test-sitemap.mts, run as npm run test:sitemap.

// 1. LAST_MODIFIED must still be a literal, not a computed date.
const literal = source.match(/const LAST_MODIFIED = new Date\(\s*'([^']+)'\s*\)/);

// 2. No bare new Date() anywhere in the sitemap (comments stripped first,
//    because the comment forbidding it contains the same characters).
const bareCalls = (code.match(/new Date\(\s*\)/g) ?? []).length;

// 3. The declared date must not be in the future.
// 4. Content under app/ or lib/ must not have changed after the date.
git log -1 --format=%cs -- app lib

Two details are worth stealing. The comment-stripping in step 2 is not tidiness: the comment above the constant says "Do NOT use new Date() here", and the first version of the check matched its own prohibition and reported a failure that did not exist. A check that fires on the rule's explanation teaches people to ignore red lines.

The git comparison in step 4 is the other one. File mtimes would be stamped by the checkout on every CI run, so the test would fail constantly and be disabled within a week. Git history records when the content actually changed; the check compares against that, with a half-day grace for timezone overlap.

How to audit a sitemap you did not write

# 1. Does every URL carry a lastmod, and are they all identical?
curl -sS https://your-domain.com/sitemap.xml \
  | grep -oE '<lastmod>[^<]+' | sort | uniq -c | sort -rn | head

# 2. Is any date in the future?
curl -sS https://your-domain.com/sitemap.xml \
  | grep -oE '<lastmod>[^<]+' | sed 's/<lastmod>//' | while read d; do
      [ "$d" \> "$(date -u +%Y-%m-%d)" ] && echo "FUTURE: $d"
    done

# 3. Does the sitemap contain URLs that redirect or 404?
curl -sS https://your-domain.com/sitemap.xml \
  | grep -oE '<loc>[^<]+' | sed 's/<loc>//' | head -5 \
  | while read u; do curl -sS -o /dev/null -w "%{http_code} %{url_effective}\n" "$u"; done

The first command is the tell. If every URL carries the same date because the date is global, that can be honest — but only if a test moves it when content changes. If every URL carries the build date, the field is decorative.

What lastmod does not do

It does not make a page rank, it does not guarantee that a changed page is recrawled, and it does not make an AI engine treat the content as new. It is a hint that costs a line and returns a small efficiency gain, which is exactly why the failure mode — a false claim — costs more than the gain is worth.

The standard is the same as for the date fields on a page: a claim about history should be true, checkable, and not moved by a deploy. If it cannot be, omit it. A sitemap withoutlastmod is a list; a sitemap with a false lastmod is a list with a lie attached.