Why the NoIndex Tag is GOOD for SEO (If You're Careful)

on under Search Engine Optimization.

wrong wayIf you're having trouble getting into the search results, the very first thing you should do is check your pages for the NoIndex tag.

It looks like this:

<meta name="robots" content="noindex">

This code tells search engines that they shouldn't show a page in their search results. More often than you might expect, webmasters have the noindex tag on their pages without even realizing it, because it came with one of their themes, or because of some other misunderstanding.

So, if you want a page to show up in search results, make sure you ARE NOT using the NoIndex tag.

However...

The NoIndex tag can be GOOD for SEO.

Yes, you read that right.

While you need to be very careful with how you use it, the NoIndex tag plays an important part in search engine optimization, and you would be wise to understand it, rather than simply assuming that it's always a bad idea to use it.

Why NoIndex Can Improve SEO

At this point, I can imagine you might be wondering how an instruction to ignore pages in search engines could help optimize your performance in search engines.

Here's the thing: there are some pages you just don't want showing up in search results.

Does anybody really want to land on .../page/27/ of your site? Is that helpful for users? Are any of them going to find what they're looking for there?

I know, I know, "But at least their landing on my site, and some of them might find what they're looking for, and eventually they might become customers."

There's some truth to this statement, but things aren't that straightforward. Here's the problem. What if .../page/27/ outranks the page that's actually a better fit for the searcher? How do they react when they see your site listed again in the search results? If they recognize your domain name, are they going to bother clicking, or are they going to assume it's another junk result?

Here's something that happens more often than you might think. A tag page, archive page, or something similar ends up ranking for the query in question, and Google simply decides not to display the more relevant page in search results. It may be hiding in the index somewhere, but it never shows it on the front page, because Google's algorithm tends to avoid placing multiple links from the same domain on the front page (unless you're branding is very good).

The end result is that the more relevant page never ends up ranking, and the less relevant archive page ends up taking its place.

By noindexing archive pages, tag pages, and similar pages, you avoid this problem.

How to Use NoIndex to Boost Visibility in Search Results

A case study by Harrison Jones reveals how using the NoIndex tag can actually send more referral traffic, when used correctly.

Harrison's experiment worked like this:

  • He NoIndexed the taxonomy pages on one site (such as category and tag pages). In isolation, this was a bad move. It caused traffic to drop by 20 percent. However, after optimizing the pagination using the rel=next and rel=prev tags, traffic skyrocketed.
  • On a second site, he tried optimizing the pagination without using the NoIndex tag. The result? No dramatic changes.
  • On two final sites, he optimized the pagination and NoIndexed the taxonomy pages. This approach worked. Traffic increased by 30 percent on one site and 20 percent on the other.

So, the NoIndex tag on its own can hurt your visibility in search results, but when it is combined with optimized pagination, it actually increases your visibility in the search results.

We have ourselves a winner, folks.

Image credit: David Goehring

  •  
  •  
  •  
  •  
  •  
  • Thanks for this Carter. I'd actually be reticent to noindex either paginated results or tag/category pages - especially the latter, as (at least if they're properly constructed) they do collapse topically-related items in a single collection page, and so are legitimate search targets (that is, they can certainly be perceived as being useful to users).

    For more a cut-and-dried example of a good page to noindex, I'd instead suggest a filtered result on an ecommerce page. Category page of running shoes? Good URL to index. Category page of running shoes filtered to show Nike shoes between $75 and $100 in red, and each those filtration attributes encoded with parameters? Not such a good URL to index. :)

    • Carter Bowles

      I definitely agree these aren't cut and dry, especially in the case of category pages. (I tend to question whether tag pages, on the other hand, should exist at all, or at least whether they should exist simultaneously with category pages.)

      I don't personally think that archive pages are all that useful for users, though, at least not on a SERP. But our disagreement on that is a perfect example of how subjective this is, and why you shouldn't just let some SEO WordPress plugin make the call for you.

      Completely agree on filtered result pages. Very clear cut case.

      • "But our disagreement on that is a perfect example of how subjective this is, and why you shouldn't just let some SEO WordPress plugin make the call for you."

        Big +1 on that sir. And to plugins I'd add platforms, software services, content management systems and third party hosts that also shouldn't be trusted to "make the call for you." Seven terrifying words? "We take care of SEO for you."

  • Hey Carter,

    Yeah
    from time to time I might noindex something I don't want to appear in the
    search engines (like a special offer page). It definitely has its uses, you
    just have to know what you're doing.

    Cheers!

    Dave at NinjaOutreach

  • Couldn't agree more.

    Also worth highlighting that there's a subtle but crucial difference between "noindex,nofollow" and "noindex,follow". I never nofollow internal links.

    • Carter Bowles

      Good point. I actually meant to touch on that, but the post was getting a bit long as is. By default, noindex without any specification is usually treated as follow, but it's probably best to spell it out, just in case.