Meta Robots "Index, Follow"
-
In my MozBar under "General Attributes" it says "index, follow" next to Meta Roberts for one of our client's websites. I've never seen "index, follow" before. I've seen it say "not found." What does index, follow mean and is that a bad thing? I know the reason should be obvious but this site has had a lot of problems and I'm wondering if this is related.
-
Hello!
Index/Follow means that you are allowing search engines to index your pages and follow links through those pages. If you didn't have this tag on the page, search engines would do it anyway. Basically, this meta tag isn't necessarily a bad thing and the search engines would be doing it even if the tag wasn't there.
You can use meta robots to do good things for your site, such as stop the crawling of blog archive pages to limit duplicate content. There's tons of resources here on MOZ about the robots file and I would suggest looking into it. This is a good one: http://moz.com/learn/seo/robotstxt
What type of problems is your site having?
-
Thank you! That was very helpful!
-
What you're seeing is the page's meta robots tag, which instructs the search engine bots to (in this case) index a specific page and follow links on the page. The opposite, "noindex,nofollow" instructs the search bots to not include this page in its index or follow links on the page. You can also use "noindex, follow" so the search engine keeps the page out of its index, but still follows the links on the page. I personally have never come across an example of "index, nofollow" but I imagine that it's possible - probably more of an outdated practice though related to hoarding links.
If you've never seen this attribute before, it may be because the page doesn't include the tag (in which case the bots, by default, assume "index, follow"). And as for whether or not having it is a bad thing - probably not, unless it's a page you really don't want visible in search results.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz claims we have meta noindex but we don't
Hi, I'm encountering an issue where moz scan says we have meta noindex, but I have confirmed across several of our pages that this simply isn't true. I have confirmation that the below tag is present: name="robots" content="index, follow" /> I also verified our https header through https://www.webconfs.com/http-header-check.php and see nothing indicating that we are sending any no index headers. Why would the crawler report this when it doesn't seem to be the case? Let me know if I need to provide more information.
Moz Bar | | charper_floqast0 -
"New" issues not previously found being shown?
I'm not sure what logic Moz is using for its reporting of Site Crawl issues, but it appears to be pretty flawed (unless I'm missing something, which is possible). I've got a client site that has been in Moz for about 6 months now. Every time the crawler runs, the same number of pages are reported as having been crawled. However I'm consistently getting "New Issues" reported that should have been reported during previous crawls. Example: A redirect chain was reported several month ago. The referring URL was the homepage of the website, and we tracked it down to an old link in the header. This was fixed, marked as resolved, and the issue was not shown on the next crawl. Several weeks later, the same issue was reported for a different page on the website - a page which has existed since 2014 and was already crawled many times. Again, we fixed. Fast-forward to the report that just ran on 12/1 and we have the same issue reported, for a different page, which has also existed for years and has been previously crawled. It's very hard to explain to a client "this item you are seeing has been resolved", only to have it continually crop back up in future reports. Note this is not limited to redirect chains - that's just an example. I'm seeing this for other items such as missing canonicals, duplicate titles, etc.
Moz Bar | | RucksackDigital0 -
Hald the keywords in my keyword list say "Gethering Metrics". How do I get that to complete?
I'vwe waited about three hours, with no change. Why does this happen — is it something I did? Will those metrics ever be gathered? Or do I need to re-do buildingthe list all over again? And how can I be sure it will be any dirfferent?
Moz Bar | | btreloar0 -
605 : Page banned by robots.txt
Hello everyone, I need experts help here, Please suggest, I am receiving crawl errors for my site that is , X-Robots-Tag: header, or tag. my robots.txt file is: User-agent: * Disallow:
Moz Bar | | bhomes0 -
Do exact keyword matches exclude "in", "based" etc?
I am trying to build a landing page for the search term "web design london" and I have included this search term as well as some variations such as "web design in london", "web design based in london" as the content doesn't really read well if I don't put in a connector word (I can't remember what the term for the use of "in" etc is). However I am using the Moz On-Page Grader to make sure I'm dotting every i and crossing every t, but it doesn't seem to pick up on the search term when "in" or "based" is used. Now is this a limitation of the On-Page Grader or should I expect Google and other search engines to not pick up on the search term when it contains these sorts of words?
Moz Bar | | mickburkesnr0 -
On Page Grader - We have keyword in Meta Description But does not show up
The on-page grader is not counting/picking up the keyword in the meta description. We have double checked the meta description and re-uploaded the page, and checked the source code on-line. The keyword is there in meta description, but not showing up in on-page-grader. https://www.trustdeedrealestateinvesting.com/trimarkfunding/buy-first-trust-deeds.html keyword buy first trust deeds
Moz Bar | | Manifestation0 -
Moz crawl sees meta description but there are none
I have a new site I have run a starter crawl on. The crawl came back saying some of the pages do indeed have meta descriptions. When I go to the same page and use the Moz Chrome toolbar it says they do not have meta descriptions. I also know they do not have meta descriptions. Are there any instances in which the Moz crawler would see them when they are not there?
Moz Bar | | SBXMedia0 -
Why still the moz index showing "Next Update on August 26, 2013"?
I have been waiting to see the updated authority metrics from Opensiteexplorer.org. But still it is showing old data. and showing "next index update: August 26, 2013" as the date is 27th August today. Is there any delay in update?
Moz Bar | | Bala_K2