Meta Robots "Index, Follow"
-
In my MozBar under "General Attributes" it says "index, follow" next to Meta Roberts for one of our client's websites. I've never seen "index, follow" before. I've seen it say "not found." What does index, follow mean and is that a bad thing? I know the reason should be obvious but this site has had a lot of problems and I'm wondering if this is related.
-
Hello!
Index/Follow means that you are allowing search engines to index your pages and follow links through those pages. If you didn't have this tag on the page, search engines would do it anyway. Basically, this meta tag isn't necessarily a bad thing and the search engines would be doing it even if the tag wasn't there.
You can use meta robots to do good things for your site, such as stop the crawling of blog archive pages to limit duplicate content. There's tons of resources here on MOZ about the robots file and I would suggest looking into it. This is a good one: http://moz.com/learn/seo/robotstxt
What type of problems is your site having?
-
Thank you! That was very helpful!
-
What you're seeing is the page's meta robots tag, which instructs the search engine bots to (in this case) index a specific page and follow links on the page. The opposite, "noindex,nofollow" instructs the search bots to not include this page in its index or follow links on the page. You can also use "noindex, follow" so the search engine keeps the page out of its index, but still follows the links on the page. I personally have never come across an example of "index, nofollow" but I imagine that it's possible - probably more of an outdated practice though related to hoarding links.
If you've never seen this attribute before, it may be because the page doesn't include the tag (in which case the bots, by default, assume "index, follow"). And as for whether or not having it is a bad thing - probably not, unless it's a page you really don't want visible in search results.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does "Disallow: /xmlrpc.php" in robots.txt affect moz tools ability to fetch DA?
Just checked a website for Domain Authority using Moz' tool, however it returned 1 for DA, which should be unlikely. I have been trying to find the problem and found "Disallow: /xmlrpc.php" in robots.txt. Could this affect Moz' tools ability to get the required data?
Moz Bar | | Foli0 -
Is the update site crawl feature following robot.txt rules?
I noticed that most of the errors would not be occurring if Moz's tool followed the rules implemented in sites robots.txt. Has anyone else seen this problem and do you know if Moz will fix this?
Moz Bar | | jamestown0 -
Moz is reporting weird email address URLs as 'Meta refresh' errors? Anything to worry about?
Under site crawl, Moz is reporting weird email address URLs as 'Meta refresh' errors. The URLs are: http://support@ihasco.co.uk and http://enquiries@ihasco.co.uk Once clicked, they redirect to our homepage. Anyone else ever had this? Is it anything to worry about? I don't think it is, but would be good to get some reassurance.
Moz Bar | | iHasco0 -
How does the grader tool treat keyword "stuffing" in ecommerce
We recently started using Moz on our ecommmerce site because I'm concerned that our SEO company doesn't really know what they are doing and I want to see what I can do on my own with the little bit of knowledge I have. It's helping in a number of ways but here's a big question mark: The Grader Tool keeps telling us that our product category pages have too many keywords on them. We are only using them in the content once or twice, but the sub-category buttons on the page show the category + sub repeated. Could this be what's causing it? Does Google distinguish this for ecommerce? We've taken a huge hit in rankings for key phrases and keywords over the past six months and I'm wondering if this is part of it?
Moz Bar | | Creative-Web-Stores0 -
Google Analytics keeps disconnecting, requires "Reauthorization"
I have many campaigns connected to our Google Analytics account (all websites in same GA account) but I keep getting the message "Our connection to your Google Analytics account was lost. Don't worry, you won't lose any data. Please reauthorize now.". I go through the motions of "reauthorizing" and it tells me I'm successful. Minutes later it gives me the same message. I even keep my GA account logged in on the same computer, trying to avoid this. As a result, I'm not getting my Analytics data in my campaign. Help!
Moz Bar | | TotalMarketExposure0 -
How does an index page have a higher Authority than the root domain?
So just curious, but on my domain, http://www.bulwarkpestcontrol.com the Page Authority is 59 and the root domain authority 52. That seems odd as it is the same page. Explanation?
Moz Bar | | Thos0030 -
Moz crawl sees meta description but there are none
I have a new site I have run a starter crawl on. The crawl came back saying some of the pages do indeed have meta descriptions. When I go to the same page and use the Moz Chrome toolbar it says they do not have meta descriptions. I also know they do not have meta descriptions. Are there any instances in which the Moz crawler would see them when they are not there?
Moz Bar | | SBXMedia0