Noindex tells a search engine not to include a page in its index. The page can still be crawled and still be read by a visitor, it just will not appear in search results.
Most of the confusion around it comes from mixing it up with two other things that sound similar and do something different. This covers what noindex is, how to apply it, the robots.txt mistake that silently breaks it, and which pages are actually worth hiding.
The two ways to apply it
Google documents exactly two implementations, and says they “have the same effect”.
- A meta tag in the page’s
<head>, which is the usual method for normal HTML pages. - An X-Robots-Tag HTTP header returned with a value of
noindexornone. Google notes this “can be used for non-HTML resources, such as PDFs, video files, and image files”, which the meta tag cannot reach.
Source: Google Search Central.
That second one matters more than people expect. If you have PDFs ranking that you would rather were invisible, the meta tag is not an option and the header is the only route.
The mistake that silently breaks it
This is the single most important thing on this page, and it catches experienced people.
Google is unambiguous: “For the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file.” If robots.txt blocks the URL, Googlebot “will never see the noindex rule, and the page can still appear in search results”.
Read that again, because it is backwards from how most people reason. Blocking a page more aggressively does not hide it harder. It hides your instruction to hide it.
The two directives work at different stages. Robots.txt controls whether a page gets crawled. Noindex controls whether a crawled page gets indexed. A rule in the page can only be read if the crawler is allowed to fetch the page.
Google also states that specifying noindex inside robots.txt “is not supported”, so putting it there achieves nothing at all.
If you want a page out of the index: allow crawling, serve noindex, and wait. Once it has dropped out, you can block it in robots.txt if you also want to stop the crawling, though by then there is rarely a reason to.
Noindex, nofollow and disallow
Three things people use interchangeably, which do three separate jobs.
- noindex is a page-level instruction that says: crawl this if you like, but do not list it in search results.
- nofollow is a link attribute that says something about a specific link rather than about your page. Google documents it alongside
ugcandsponsored, and it is covered in do nofollow links help SEO. - Disallow in robots.txt says do not fetch this at all. It is about crawling and says nothing about indexing, which is why a disallowed URL can still show up in results, usually with no description.
Short version: use noindex to keep something out of search, disallow to keep a crawler away from something, and never expect either one to do the other’s job.
How long it takes
Longer than people expect, and Google gives no fixed timeframe.
If the page is still showing, Google says that most likely it “haven’t crawled the page since you added the noindex rule”. It adds that “depending on the importance of the page on the internet, it may take months for Googlebot to revisit a page”.
So a page that is still in results a week later is not evidence that the tag is broken. Before changing anything, confirm the tag is actually being served, then request a recrawl with the URL Inspection tool, which Google names as the way to speed this up.
If you need something gone urgently rather than eventually, the Removals tool in Search Console hides a URL temporarily while the noindex takes effect. It is a stopgap, not a fix.
Which pages are worth noindexing
The useful test is not “is this page low quality”. It is “would anyone ever want to arrive here from a search result”.
- Thank-you and confirmation pages. Nobody searches for them, and they look like errors when indexed.
- Internal search result pages. Endless, thin, and generated on demand.
- Thin tag and category archives. These compete with the articles they list, which is a common source of keyword cannibalization.
- Attachment pages on WordPress, which exist for every uploaded image and contain nothing, as covered in WordPress attachment pages.
- Staging and duplicate environments. Although these are better protected with authentication than with a tag.
- Paginated archive pages beyond the first, where they add nothing a reader would search for.
What is generally not worth noindexing is a real article that happens to be underperforming. A page with no traffic is usually a page to improve or merge, not to hide, and that decision belongs in a content audit.
On WordPress
You rarely write the tag by hand. Yoast and All in One SEO both expose a per-page setting, usually phrased as whether to allow search engines to show this page, and both offer site-wide switches for archive types such as categories, tags, author pages and media.
Two things to watch. First, check the rendered page source rather than trusting the setting, because a theme or second plugin can also emit robots directives and the result is not always what the checkbox says. Searching the source for noindex takes a few seconds.
Second, remember the robots.txt rule above. Some security and caching plugins add disallow rules, and if one of those blocks a URL you have noindexed, the tag will never be read.
Checking it worked
- View the page source and confirm the robots meta tag is present and says what you think.
- Check robots.txt does not block that URL, since that is what quietly defeats the tag.
- Run a live URL inspection in Search Console to see what Google fetches right now rather than what it saw last time.
- Request indexing to prompt a recrawl, which Google suggests as the way to speed removal up.
- Wait, then re-check. Search Console will eventually list the URL under excluded by noindex, which is confirmation rather than a problem.
That last point is worth stating plainly: pages listed as excluded by a noindex tag in Search Console are working as intended. It is a report of your own decisions, not an error list.
Frequently asked questions
What does noindex do?
Noindex tells search engines not to include a page in their index, so it will not appear in search results. The page can still be crawled and is still reachable by visitors. Google supports two implementations, a robots meta tag in the page head and an X-Robots-Tag HTTP header, and says both have the same effect.
Why is my noindexed page still in Google?
Usually because Google has not recrawled it yet. Google says that if a page is still showing, it most likely has not crawled the page since the rule was added, and that depending on the page’s importance it may take months for Googlebot to revisit. Confirm the tag is actually being served, then request a recrawl with the URL Inspection tool.
Can I use robots.txt and noindex together?
Not for the same URL, because blocking it defeats the tag. Google states that for the noindex rule to be effective the page must not be blocked by robots.txt, since the crawler will never see the rule and the page can still appear in results. Allow crawling until the page has dropped out of the index, and only then consider blocking it.
What is the difference between noindex and nofollow?
Noindex is a page-level instruction about whether a page appears in search results. Nofollow is a link attribute that qualifies a specific link rather than describing your page, alongside the ugc and sponsored values. They operate on different things, so one is never a substitute for the other.
Can you noindex a PDF?
Yes, but not with a meta tag, since a PDF has no HTML head. Use the X-Robots-Tag HTTP header instead, which Google says can be used for non-HTML resources such as PDFs, video files and image files. This is the main reason the header method exists.
Should I noindex pages with no traffic?
Usually not. A real article with no traffic is normally a candidate for improvement or merging rather than hiding, and removing it from the index gives up any impressions it was quietly earning. Reserve noindex for pages nobody would ever want to find through a search, such as thank-you pages, internal search results and thin archives.