The technical SEO audit checklist for 2026

A technical SEO audit checks whether search engines can find, read and index your pages. It is about removing the things that quietly stop good pages from showing up: a stray noindex tag, a robots.txt rule left over from a staging site, a chain of redirects, text that only appears after JavaScript runs.

You don't need to be a developer. Google's technical requirements are short: Googlebot isn't blocked, the page returns an HTTP 200 (success) status code, and the page has indexable content. This checklist confirms those three things for the pages that matter, then tidies up the signals that help Google understand them.

You need Google Search Console, a browser and a few free Google tools. Set aside an afternoon and note what fails.

Before you start

Verify your site in Google Search Console. It is free and most checks use it.
List your 10 to 20 most important URLs: homepage, product pages, pricing, best articles.
Search Google for site:yourdomain.com. If results appear, you are in the index. Google's SEO starter guide suggests this as the first check.

The checklist at a glance

Check	Why it matters	How to test	Free tool
robots.txt doesn't block key pages	Blocked pages can't be crawled	Open /robots.txt in a private window	Search Console robots.txt report
Sitemap lists the URLs you want indexed	Helps Google find pages	Open /sitemap.xml	Search Console Sitemaps report
No unwanted `noindex`	Drops a page from Google entirely	Check "indexing allowed"	URL Inspection tool
Canonicals point to the right URL	Tells Google which duplicate to show	Compare declared and Google-selected canonical	URL Inspection tool
Key pages return 200	Google only indexes 200 pages	Inspect the URL	URL Inspection tool
Redirects are permanent and short	Chains add delay	Visit old URLs	Page indexing report
Non-indexed pages are excluded on purpose	Shows what Google skipped and why	Read each reason	Page indexing report
Unique, descriptive titles	Main input for the title in results	View source	Browser
Unique meta descriptions	Sometimes used as the snippet	View source	Browser
One clear main heading	Shows what the page is about	Look at the page	Browser
Every key page has an internal link	Google finds pages through links	Click from the homepage	Browser
Images use `<img>` with alt text	CSS background images aren't indexed	Inspect the image	Lighthouse
Structured data is valid	Needed for rich results	Test the URL	Rich Results Test
Pages work on mobile	Google indexes the mobile version	Mobile audit, real phone	Lighthouse
HTTPS everywhere	Google prefers HTTPS	Look for HTTP URLs	Search Console HTTPS report
Core Web Vitals are good	Used by Google's ranking systems	Check the report	PageSpeed Insights
Content is in the rendered HTML	Not all bots run JavaScript	View the tested page	URL Inspection tool
Site files load	robots.txt, sitemap, optional llms.txt	Open each file	Browser

Crawling and indexing

robots.txt

robots.txt sits at your site root and tells crawlers which URLs they may access. Google says it is "not a mechanism for keeping a web page out of Google": a blocked URL can still be indexed if other pages link to it, just without a description.

Look for Disallow: / under User-agent: * or User-agent: Googlebot (it blocks the whole site, a common staging leftover), rules blocking CSS or JavaScript your pages need, and a Sitemap: line. A healthy file:

User-agent: *
Disallow: /admin/

Sitemap: https://www.example.com/sitemap.xml

Then check the robots.txt report in Search Console for errors.

XML sitemap

A sitemap lists the pages you want search engines to know about. Google says a well-linked site of about 500 pages or fewer might not need one, but "in most cases, your site will benefit". Check that it:

Uses full URLs (https://www.example.com/pricing, not /pricing).
Lists only URLs you want in results: no redirects, error pages or noindexed pages.
Stays under 50MB uncompressed or 50,000 URLs per file.
Has accurate <lastmod> dates. Google ignores <priority> and <changefreq>.

Submit it in the Sitemaps report or with a Sitemap: line in robots.txt. Google calls submission "merely a hint".

noindex

A noindex rule, as <meta name="robots" content="noindex"> or an X-Robots-Tag: noindex header, makes Google drop the page. It is easy to leave on by accident after a redesign. And if robots.txt blocks the page, Google never sees the noindex, so the URL can still appear. Confirm in URL Inspection that indexing is allowed on every key URL.

Canonical tags

When the same content lives at several URLs (with and without www, with tracking parameters), Google picks one canonical version to show. Google treats redirects and rel="canonical" tags as strong signals and sitemap inclusion as a weak one. Its advice: put a self-referencing canonical on the canonical page, don't send mixed signals (one URL in the sitemap, another in the tag), don't use robots.txt or noindex to choose, and link internally to the canonical. None of this is required; without it, Google picks.

Status codes and redirects

What Google does with each status code:

200: considered for indexing, not guaranteed.
301 or 308: permanent redirect, a strong signal. 302 or 307: temporary, a weak signal.
4xx (404, 410): not indexed; indexed URLs are removed.
5xx and 429: Google slows crawling and eventually drops URLs that keep failing.

Watch for soft 404s, "not found" pages served with a 200 code. Use server-side 301 or 308 redirects for permanent moves. Googlebot follows up to 10 hops, but Google advises going straight to the final URL, or "ideally no more than 3 and fewer than 5". Check that http://, http://www. and https:// versions of your homepage reach your preferred URL in one hop.

Read the Page indexing report

The Page indexing report in Search Console shows which URLs are indexed and why others aren't. Google says sites with fewer than 500 pages probably don't need it; site: searches are enough. Use it when key pages are missing.

Check each "not indexed" reason and decide whether it is intended. Duplicates, filtered pages and deleted pages shouldn't be indexed.
Fix the mistakes, then click Validate fix. Google says this typically takes up to about two weeks.

Reason	What to do
URL marked 'noindex'	Remove the rule if the page should be in search
Soft 404	Return a real 404, or add real content
Duplicate, Google chose different canonical than user	Check the pages really are duplicates
Crawled - currently not indexed	It may be indexed later; no need to resubmit
Discovered - currently not indexed	Google put off crawling to avoid overloading your site; usually wait

Titles and meta descriptions

Google builds each result's title link from several sources, including the <title> element, headings and links to the page. Its title link guidance: give every page a descriptive, concise <title> (not "Home"), avoid keyword stuffing and near-identical titles, and add your brand at the start or end with a separator. There is no length limit, but titles are truncated "typically to fit the device width".

Google says snippets are "primarily created from the page content itself", using the meta description when it describes the page better. Write a unique one per page, starting with your homepage and popular pages. Google ignores the meta keywords tag.

Headings

Google's starter guide says headings in semantic order are "fantastic for screen readers, but from Google Search perspective, it doesn't matter if you're using them out of order". There is no ideal number. Google does look at the main visible title and headings when building title links, so give each key page one obvious main heading, for example in the first visible <h1>.

Internal links

Google's link guidance says: "Every page you care about should have a link from at least one other page on your site." Check that key pages are reachable by clicking from the homepage, that links are real <a href="/page"> elements (Google can't reliably follow links that only work through a JavaScript click handler), and that link text is descriptive, not "click here".

Images and alt text

Use standard <img> elements. Google "doesn't index CSS images".
Use a supported format: BMP, GIF, JPEG, PNG, WebP, SVG or AVIF.
Use descriptive file names (invoice-dashboard.png, not IMG_0042.png).
Write alt text that describes the image in context, without keyword stuffing. It also serves as link text when an image is a link.
Set width and height so the page doesn't jump while images load.

Structured data

Structured data is code, usually JSON-LD (data in a <script> tag), that labels what a page contains. Google recommends JSON-LD and uses it for rich results such as breadcrumbs.

Only features on Google's supported list produce rich results. For a typical B2B site, the likely candidates are Organization on the homepage (no required properties), Article on blog posts and Breadcrumb.
FAQ rich results are gone: Google stopped showing them from 7 May 2026.
Mark up only content visible on the page.
Google "does not guarantee that your structured data will show up in search results", even when it is valid.

Test templates with the Rich Results Test.

Mobile

Google uses the mobile version of your pages for indexing and ranking (mobile-first indexing) and recommends responsive design. If your mobile pages differ, they need the same content, structured data and robots meta tags as desktop. Google retired the Mobile-Friendly Test on 1 December 2023; use Lighthouse in Chrome DevTools or PageSpeed Insights, and a real phone.

HTTPS

Google "strongly recommends using HTTPS" and prefers to index the HTTPS version of a page. The HTTPS report in Search Console should show no HTTP URLs. Every http:// URL should redirect to https://, your certificate should be valid, and canonical tags and sitemaps should point to HTTPS URLs.

Page speed and Core Web Vitals

Core Web Vitals measure real visitors' experience: LCP (Largest Contentful Paint, loading), INP (Interaction to Next Paint, responsiveness) and CLS (Cumulative Layout Shift, visual stability). Google says "Core Web Vitals are used by our ranking systems", and that good scores don't guarantee top rankings. Open the Core Web Vitals report in Search Console, fix "Poor" groups first, then test key URLs in PageSpeed Insights. Thresholds and fixes are in Core Web Vitals explained.

JavaScript rendering

Google crawls, then renders, then indexes JavaScript pages. Pages wait in a render queue "for a few seconds, but it can take longer than that". Google's JavaScript SEO basics say server-side or pre-rendering "is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript."

In URL Inspection, run a live test and view the tested page. Is your main text in the rendered HTML?
Use "View page source". If the raw HTML has no text, ask your developer about server-side rendering.
Check pages have real URLs, not #/ fragments, and missing pages return a real 404.
Set the title and canonical in the HTML, and don't let JavaScript change the canonical.

Site files: robots.txt, sitemap.xml and llms.txt

Each should load at your site root with a 200 code.

robots.txt also controls AI crawlers. OpenAI says OAI-SearchBot is "used to surface websites in search results in ChatGPT's search features", while GPTBot crawls content that may be used for training. To appear in ChatGPT search but opt out of training:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

llms.txt is a proposal, first published in September 2024, for a Markdown file that gives AI tools a short summary of your site. It is not an official standard. Google says Google Search ignores it and that it "will neither harm nor help your site's visibility or rankings in Google Search"; other tools may read it. See our llms.txt guide and what GEO is.

How to prioritise what you find

Anything stopping key pages being indexed: site-wide robots.txt blocks, stray noindex, server errors, 404s.
Canonical and redirect problems.
Missing or duplicate titles and descriptions on top pages.
Key pages with no internal links.
"Poor" Core Web Vitals groups.
Structured data and tidy-ups.

Google says some changes take effect in hours and others take months, so wait a few weeks before judging.

How the BeforeQuery AI Marketer helps

The AI Marketer's SEO tab audits your homepage and gives it a health score out of 100. It checks title and meta description length, a single H1, heading order, canonical, noindex, viewport, alt text, page language, Organization schema, Open Graph and X card tags and render-blocking files, and shows Lighthouse scores from PageSpeed Insights for mobile and desktop. It checks whether robots.txt allows Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot and PerplexityBot, whether the page can be read without JavaScript, and whether robots.txt, sitemap.xml and llms.txt exist. It writes an llms.txt for you.

It covers the homepage only, so site-wide checks such as the Page indexing report, internal links and redirects are still yours.

The free plan includes one homepage audit at sign-up. On paid plans the SEO Agent audits your homepage every day and writes 2 fixes a day, each with steps and a copy-ready snippet. The Coding Agent opens a GitHub pull request with fixes such as titles, meta descriptions, canonical tags, alt text, robots.txt and sitemaps. It never merges; you review and merge. Sign up free.

Frequently asked questions

How often should I run a technical SEO audit?

Google doesn't set a schedule. In practice, a full audit a few times a year suits most small sites, plus a check after any redesign, migration or CMS switch, when robots.txt rules, redirects and noindex tags most often go wrong.

Do I need an XML sitemap if my site is small?

Not always. Google says you might not need one if your site has about 500 pages or fewer and every important page is reachable by links from the homepage. It also says most sites benefit, and many CMSs create one automatically.

Can robots.txt keep a page out of Google?

No. robots.txt controls crawling, not indexing. Google can still index a blocked URL if other pages link to it. To keep a page out of results, use a noindex rule on a page Google is allowed to crawl, or put it behind a password.

Why does Google show a different title from the one I wrote?

Google generates title links automatically from several sources, including the <title> element, headings and links to the page. It may choose another source if your title is missing, repetitive or doesn't match the page. Snippets work the same way: they come mainly from page content and can change with each search.

Does llms.txt help SEO?

Not for Google. Google says Google Search ignores llms.txt files and that having one neither helps nor harms your rankings. It is a proposal aimed at AI tools and agents, so treat it as optional rather than as an SEO fix.