Noindex and nofollow - how to exclude pages from Google search?

Indexing a large number of pages in search engines is very tempting for many novice site owners.

Especially if these are new sites, and we are trying to improve the position of our site as quickly as possible.

While it's true that by posting more pages that are relevant to a given keyword (assuming they're also of good quality) we'll improve our rankings for the chosen keywords, sometimes it's better to keep some pages away from the search engine index.

Why?

I'll explain everything now. This post will show some situations in which it is better to remove a given subpage from the search engine index, and what exactly it looks like from a technical point of view.

Why should you de-index certain subpages of your site?

There are a number of cases where a subpage should be excluded from a search engine.

The main reason for using noindex and nofollow is to prevent duplicate content from being indexed (a situation where the same text is on two different pages).

Another good reason?

For example, a thank you subpage. Usually, the user can see it immediately after downloading your e-book or subscribing to your newsletter.

As a site owner, you only want a user to go to your site if they take some action, such as signing up for a newsletter. You don't need direct links from the search engine to thank you pages.

Why?

A simple example - if you measure conversions using such a page, you will get false information, because someone can visit such a subpage directly from the search engine.

Another example would be the ability to download a free e-book in exchange for filling out a form. Upon proper completion, the user is redirected to a thank you subpage with a link to the promised ebook.

Now imagine that anyone can get to this subpage by typing "e-book positioning" in a search engine.

Thank you and a link will appear on your subpage. The user got what he wanted, and you were left without information and the desired email.

In addition, you may find that the thank you subpage ranks first in the SERPs for your main keyword. As a result, you again lose a lot of blank forms and information.

These are pretty good reasons to use noindex and nofollow tags.

So, how to disable the indexing of unwanted pages?

There are two ways, so let's move on to the next point.

Ways to not index selected subpages of the site

1. Attach the robots.txt file to your site.

Use the robots.txt file (before continuing, I advise you to read the article at the link) if you want to have more control over what should be indexed and what should not.

Attention: if you don't have basic technical knowledge, take the advice of an experienced specialist, as incorrect robots.txt file setting can completely "cut off" your site from any search engine.

In this regard, one way to remove a specific subpage from search results is to create and add a robots.txt file to your site.

The advantage of this method is that you can have complete control over what appears in the search engine.

Inside the robots.txt file you can specify whether you want to block indexing robots or not.

There are a number of rules. You can exclude one subpage, such as a thank you page, block an entire directory, or even get rid of indexing a single image.

I also recently came across a rule that says you shouldn't index a subpage if Google AdSense isn't already running on that site.

If you don't have technical knowledge and don't need full control over what appears in search engine results, there is a much easier way.....

2. Add noindex and nofollow tags to the subpage in question.

Use noindex and nofollow tags as an easier solution to exclude unwanted subpages from search engine results.

Using meta tags to prevent the selected subpage from appearing is efficient and simple.

It's actually a simple "copy/paste" if you're using the right content management system (CMS).

The tags that allow you to do this are called noindex and nofollow.

Before explaining how to use noindex and nofollow correctly, I will explain the differences between these two tags.

They look similar, but they are two completely different directives. They can be used together or separately, but what do they really mean?

What is the noindex tag?

When you add a noindex tag to a web page, you are telling the search engine that it can index this page, but under no circumstances can it add it to the search results.

What is a nofollow tag?

When you add a nofollow tag to a web page, you prevent search engines from indexing links on that page. This also means that page authority (page authority) will not be passed to any link that is on that page.

In this regard, any links on the page with the 'nofollow' directive will be ignored by Google and other search engines.

When to use noindex and nofollow together and when separately?

You can use the noindex tag alone or in combination with nofollow. This rule also applies to the nofollow directive.

Add one noindex tag when you want the search engine to not index a particular subpage.Remember that a page with a noindex directive will be searched across the length and width of the page for links, and will derive some of its power from the authority of the subpage.

Add one nofollow tag when you want to index a specific subpage but don't want to pass subpage authority to all links on that subpage.

There aren't many examples of adding nofollow to an entire subpage without adding the noindex tag at the same time.

When you think about what to add to this or that subpage, it's more a matter of what to add: noindex or noindex and nofollow.

Add the noindex and nofollow tags if you don't want the search engine to index the selected subpage and pass authority to the links within it.

Thank you pages for signing up or filling out a form are perfect examples of this kind of situation. You don't want browsers to index such a subpage, and you don't want them to know about links leading from it to a particular offer or to a link to an e-book, for example.

How to use noindex and nofollow tags?

Step 1: Copy the code below:

For "noindex":

For "nofollow":

For "noindex and nofollow":

Step 2: Add the above code to the head section of your subpage that you want to exclude.

If you are using WordPress, the easiest way is to install Yoast SEO plugin:

If you don't know what Yoast SEO plugin is yet, I invite you to check out our article on how to optimize a post with Yoast plugin SEO . If your site is pure HTML, then unfortunately things won't be that easy. You need access to hosting or FTP.

Please note that the code must be placed somewhere between the and tags.

H2 hocus pocus, hocus mary - your subpages are gone

Your excluded subpages will no longer appear in search engine results. Now you can get back to capturing lost emails and valuable information.

Keep in mind that the changes you make will not be visible immediately. It may even take several weeks.

In this regard, the best way to track changes in the index is to study the statistics in Google Search Console.

If you notice that the page you selected is still showing up in the search results, it's likely that Googlebot hasn't visited your site yet, or the noindex and nofollow directives were used incorrectly.