Meta robots

[Mat_C]

Hi,

I am checking a website for SEO and I've noticed that a lot of pages from the blog have the following meta robots:

meta name="robots" content="follow"

Normally these pages should be indexed, since search engines will index and follow by default. In this case however, a lot of pages from this blog are not indexed.

Is this because the meta robots is specified, but only contains follow? So will search engines only index and follow by default if there is no meta robots specified at all?

And secondly, if I would change the meta robots, should I just add index or remove the meta robots completely from the code?

Thanks for checking!

[Mat_C]

Thanks, this is a really helpful answer.

[GastonRiera]

Hi Mat_C

There is no issue with that Meta Robots tag. This is not the reason why those pages aren't indexed.

I'd look a little deep trying to understand why Google didn't want to index that pages. 
Do you have access to that website Search Console? What does index coverage report say? 
Have you tried looking for one of those URLs in the "URL Inspection Tool"? There you might find why Google chose not to index it.

That said, assuming that the site has as CMS Wordpress, the widely known YOAST plugin allows you to configure to be non-indexable many "known to cause issues" pages, such as tag or archive pages.
Have you checked that this is not the case?

Also, there is another common reason why pages aren't indexed: Canonicals chosen by Google. This happens when some pages are almost identical and/or serve for the same user intent, so Google's Algorithms consider them as the same and just set one as the canonical for other, even when there isn't any canonical tag present.

Hope it helps.
Best luck.
GR

[effectdigital]

I am pretty sure that's not how Meta robots tags work. If you fail to specify something, Google assumes they are allowed to index by default. By the way, search engines do not index pages which they don't think users will like or be interested in. Just because a search engine 'can' index a URL, that doesn't mean it will!

Follow directives and index directives actually operate on two entirely different sub-sets of data. Follow / nofollow directives are link-level (meaning they apply only to the hyperlinks on a page, not to the page itself). Index / no-index directives are page-level, and apply to the entire page upon which they are situated

Due to this, I don't believe they could or would interfere with each other in the way you described

Interesting experiment though. To test, I'd recommend adding index instead of removing follow. If hat doesn't make any kind of difference, it's not the issue