Noindex Tag: Website Not Showing in Google? 5-Minute Check

A physical-therapy practice in Plantation has a clean site, real reviews and a Google Business Profile that gets calls. Search the practice name and it comes up. Search physical therapy Plantation and it does not exist. Not page three. Not anywhere.

We found the reason in under a minute during a recent audit sweep of Broward wellness practices: the homepage carries a noindex tag. One line of HTML, almost certainly left over from whoever built the site, has been telling Google to drop the page. Nobody noticed because the site loads fine, the map listing still rings the phone, and the tag is invisible unless you go looking.

It is the most expensive small mistake we see, and it takes five minutes to rule out.

What a noindex tag actually does

A noindex directive is a request to search engines to leave a page out of their index. It lives in one of two places: a meta tag in the page’s <head> (<meta name="robots" content="noindex">) or an HTTP response header called X-Robots-Tag. Google’s documentation on blocking indexing with noindex says both have the same effect, and it adds a caveat that trips people up constantly: for noindex to work, the page must not be blocked by robots.txt, because a blocked crawler never sees the tag.

The tag has legitimate uses, like a staging copy while the site is being built. The problem is that it is easy to switch on and easy to forget.

The four ways a live business site ends up de-indexed

1. The staging tag ships to production

Developers noindex the staging copy, then the site launches and the setting comes with it. Search Engine Journal’s guide to Search Console’s excluded pages calls this out directly: a spike in “Excluded by ‘noindex’ tag” often means a site went to production with the staging site’s noindex tags still attached. That is our best guess for the Plantation practice.

2. The WordPress checkbox

Settings > Reading has a box labeled “Discourage search engines from indexing this site.” Since WordPress 5.3, ticking it outputs a noindex,nofollow meta tag on every page instead of editing robots.txt, a change the WordPress core team made because robots.txt blocking never actually kept pages out of search results. Semrush’s list of reasons small-business sites vanish from Google puts this checkbox first, and it stays checked on plenty of sites that were built on a temporary domain and then pointed at the real one.

3. An SEO plugin toggle or a header you cannot see

Yoast, Rank Math and similar plugins have a per-page “Allow search engines to show this page” switch, plus site-wide defaults for post types. A theme or a page builder can add its own. And some hosts or security layers send an X-Robots-Tag: noindex header. Ahrefs’ glossary makes the point that the X-Robots-Tag cannot be found in the HTML code of the page at all; you have to inspect the response headers. An audit that only reads the source will miss it.

4. A bad robots.txt or a sitemap pointing at the wrong host

A robots.txt with Disallow: / under User-agent: * blocks every crawler from every page; Semrush notes that a single missing slash is the difference between blocking nothing and blocking everything. And a sitemap generated on staging can list staging.yourdomain.com URLs. Google says it will attempt to crawl sitemap URLs exactly as listed, so you are handing it a map to the wrong building.

That last one is ours. Not long after launch, our own sitemap was still pointing at our staging host after launch. The pages were indexable, and we were feeding Google a list of URLs nobody should visit. We caught it in Search Console. It happens to people who do this for a living.

The part nobody checks: AI crawlers

There is a newer version of the same mistake. “Block all AI bots” robots.txt snippets are everywhere, and a lot of them disallow every OpenAI user agent. OpenAI’s publisher FAQ separates them: GPTBot is the training crawler you can block if you want, but for your content to appear in ChatGPT summaries and snippets you must not block OAI-SearchBot. Block both and you have removed your practice from a search surface your patients are already using.

Google’s side is less dangerous. Its Google-Extended token controls whether your content is used for Gemini training and grounding, and Google’s crawler documentation states it does not affect inclusion in Google Search or act as a ranking signal. Blocking it will not hurt rankings, but it can keep you out of AI answers. This is the technical floor under everything we do in AI search and GEO work: if a crawler cannot read you, no amount of schema or content will get you cited.

The 5-minute check

Search Google for site:yourdomain.com. If nothing comes back, or only a handful of pages, keep going.
Open your homepage, view source, and search the page for noindex. Do the same on your top service page.
Load yourdomain.com/robots.txt. Look for Disallow: / under User-agent: *, and look for OAI-SearchBot or Google-Extended being disallowed.
Open yourdomain.com/sitemap.xml (or sitemap_index.xml). Every URL should be on your live domain, not staging.
In Search Console, run URL Inspection on the homepage. It shows the HTML Googlebot actually received, which catches header-level noindex that view-source misses. Then open the Page indexing report and look for “Excluded by ‘noindex’ tag” and “Blocked by robots.txt.”

WordPress users: also confirm the Settings > Reading visibility box is unchecked, then purge every cache layer. A WordPress.org support thread shows an owner who flipped the setting off and still served noindex on every page but the homepage, because the caching plugin held the old HTML until purged.

How Phoenix handles it

Technical checks are the first line of our free audit, before content, citations or reviews. There is no point building authority for a page Google has been told to ignore. We pull the homepage headers and source, robots.txt, the sitemap, and the Search Console indexing report if you give us access. If you run a practice in Plantation, our Plantation SEO page explains what we look at locally.

When we find a block, the fix order is boring and it works:

Remove the directive at its source: the plugin toggle, the theme option, the WordPress checkbox, the server header. Then purge caches and confirm the live response is clean.
Fix robots.txt so it controls crawling only, and make sure OAI-SearchBot and Google-Extended are allowed unless you have a reason to say otherwise.
Regenerate the sitemap on the live host and resubmit it in Search Console.
Request indexing on the homepage and the money pages through URL Inspection. Google says this typically takes a day or so but can take longer, and there is a daily quota, so we prioritize pages and let the sitemap carry the rest.
Then start the authority work: entity and schema fixes, Google Business Profile alignment, and the 40 to 60 real citations where rank lift actually plateaus.

On timelines, we will be straight with you. A homepage that was indexed before usually comes back within days of the fix. Competitive terms like physical therapy or med spa in Broward still take the normal 60 to 90 days to move in the map pack, and 4 to 6 months in the toughest categories. The noindex fix removes the ceiling; it does not do the climbing. If your site is being rebuilt, our web and app development team runs the indexability check at every launch so the staging tag never ships.

Frequently asked questions

Why is my homepage not showing in Google after a redesign?

Nine times out of ten the new site went live carrying the staging site’s noindex tag or its “discourage search engines” setting. Google reads that tag as an instruction to drop the page. Open Search Console, run URL Inspection on the homepage, and look for “Excluded by ‘noindex’ tag.” Remove the tag, clear the cache, and request indexing.

I unchecked “Discourage search engines” but Google still sees noindex. Why?

Usually a cache or a second source. Caching plugins can keep serving the old HTML with the noindex line until you purge every page, not just the homepage. If the cache is clean, an SEO plugin, the theme, or a hosting-level X-Robots-Tag header may be adding its own noindex. Check the live source and the response headers, not the settings screen.

How long does it take Google to reindex a page after removing noindex?

Google says indexing after a request “typically takes only a day or so, but can take much longer in some cases,” and a request is not a guarantee. In our experience a homepage that was previously indexed comes back within days; rankings can take longer to settle. Submit the sitemap too, since request-indexing has a daily quota.

Does blocking a page in robots.txt remove it from Google?

No. Google’s own documentation says robots.txt “is not a mechanism for keeping a web page out of Google.” A blocked URL can still be listed if other sites link to it, just without a description. Worse, if a page is blocked in robots.txt Google can never see a noindex tag on it. Use noindex for removal and robots.txt only for crawl control.

Can a robots.txt file stop my business from showing up in ChatGPT?

Yes. OpenAI’s publisher FAQ says that for your content to be included in ChatGPT summaries and snippets, you must not block OAI-SearchBot. Many “block all AI” robots.txt snippets copied from blogs disallow that crawler along with the training bot. Decide deliberately: block GPTBot if you want, but leave the search crawler open.

Find out in five minutes, or let us do it

Not sure whether your site is indexable? Send us the URL. We will run the technical, AI-crawler and sitemap checks as part of a free website + AI-visibility audit and tell you plainly what we found, even if the answer is “nothing wrong, you just need more authority.” Or call (786) 719-7997.