Meta Robots Tag - What's it really mean?
-
I used on a handful of pages recently and noticed that they're still popping up in the Google search index. I'd like to keep these from appearing, so I figured I needed a directive statement with stronger semantic meaning.
From what I understand, is what I'm looking for. Using this will keep Google from not only crawling the page, but indexing the page, as well.
I decided to see what the official robotstxt.org website said about it, so I checked (link here):
the NOFOLLOW directive only applies to links on this page. It's entirely likely that a robot might find the same links on some other page without a NOFOLLOW (perhaps on some other site), and so still arrives at your undesired page.
So, is their explanation saying that the page itself will be indexed, but the content / links on it won't be followed / indexed? Let me hear your thoughts, mozzers.
-
You can find out your desired meta robots setting by asking yourself two questions:
noindex or index: do you want this particular page to show up as a listing in the Google results for a query? Yes: index, No: noindex.
nofollow or follow: do you want the links on this page to be crawled (followed) by the search engine robots, thereby allowing them access to the deeper pages it links to (even though that particular page might even be noindexed). Yes: follow, No: nofollow
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
We have a client that wants to apply UTM URL tagging to track local organic traffic in Google Analytics. Is there any benefit in doing this?
One of our clients requested that we apply UTM URL tagging to better track organic traffic in Google Analytics. We found this to be an odd request because we are most familiar with UTM tracking for special campaigns (referral tracking, PPC, email tracking, etc). Is there any benefit of applying UTM tags to urls to analyze local organic traffic in Google Analytics? Are there any resources out there about this? Thanks!
Reporting & Analytics | | RosemaryB0 -
Why do WMT's Search Analytics & GA's Real Time Keywords Differ?
I took a screen shot of the keywords from Google Analytics Real Time Keywords for Google / Organic. Then I waited a week and filtered Google Webmaster Tools' Search Analytics for queries with those keywords. None of them showed up. Why not? What I missing? Why do some keywords show up in Google Analytics RT for Google / Organic while most others are in Not Provided? I see WMT's Search Analytics says that it doesn't always show queries that "are made a very small number of times or those that contain personal or sensitive information." But most of my queries are like that. Anybody know any patterns / criteria that leads WMT to show some but not others? Thanks
Reporting & Analytics | | GilReich0 -
Canonical Tags & GWT Parameters
A site I'm working on has canonical tags which I find to be accurate, regardless of tracking parameters or anything else added to the url. The tag looks like: And we have alot of parameters in Google Search Console that look like Parameter Crawl page Let Googlebot Decide destination Let Googlebot Decide filters Let Googlebot Decide Since all of our parameters follow a question mark, like http://www.examplesite.com/questions/avocados?source=ad12345 and all of our pages have canonical tags showing the representative url without the additional parameters, why wouldn't we just have the one parameter in GWT as Parameter Crawl ? Representative URL I ask because I find that Google analytics shows pages with parameters as landing pages in search, which has me concerned about Google seeing it as duplicate content. Thanks! Best... Darcy
Reporting & Analytics | | 945010 -
What will be configuration for new version of tag manager for given below code?
Hello Expert, I am using new version of tag manager for enhance ecommerce. Now i have post related to enhance ecommerce for old version of tag manager this one - https://developers.google.com/tag-manager/enhanced-ecommerce In this post, below is the configuration of "Measuring Views of Product Details" for old version of tag manager, can you please tell me what will be configuration for new version of tag manager? ( mainly basic setting and firing rule ) Tag type : Universal Analytics
Reporting & Analytics | | bkmitesh
Track type : Pageview
Enable Enhanced Ecommerce Features: true
Use Data Layer: true
Basic Settings - Document Path: {{url path}}
Firing Rule: {{event}} equals gtm.js Thanks! BK Mitesh0 -
Megamenu: Too many links really bad?
Hi there! Our site hosts paid training videos, and has a javascript menu that lists EVERY video on the site, and it is our most-used method of navigation. The menu is structured to look like the business software our training videos cover, so it's very intuitive for users. That said, since we currently have so many videos EVERY page has more than 250 links. The only way to get this down to under 100 as SEOMOZ recommends is to delete/hide the link from being seen by search engines. What should I do? Is the menu worth being visible to search engines? businessonetraining.com
Reporting & Analytics | | TigerSheep0 -
Google Webmaster Tools - spike in 'not selected' under Index Status
Hi fellow mozzers Has anyone seen a huge shift in the number of pages 'Not Selected' under Index Status in Google WMT, and been able to identify what the problem has been? My new client recently moved their site to wordpress - and in doing so the number of pages 'not selected' rose from ~200 to ~1100, It was high before but is ridiculous now. I am thinking there must be a new duplicate content issue which should be cleaned up in my quest to improve their SEO. Could it be the good old WP tag/category issue? In which case I won't worry as Joost is doing its job of keeping stuff out of the index. There are loads of image pages which could well appear as dupe as have no content on them (i do need to fix this), but Google is already indexing these so doesn't explain the ones 'not selected'. I've tried checking dupe title tags but there are very few of them so that doesn't help Any other ideas of how to identify what these problem pages maybe? Thanks very much! Wendy
Reporting & Analytics | | Chammy0 -
Google's New Privacy Policy and Analytics
Does anybody know if Google's new privacy policy allows it to use data gathered by Analytics to be used as a ranking factor in the SERPs?
Reporting & Analytics | | Jolora0 -
Tracing Google Analytics 'goal' back to original search phrase
I added Goals to my Google Analytics tracking. It's working; I get visitors who have completed Goals showing up in the reporting. My question is: Is it possible to trace backwards from a completed Goal to the original search phrase a user entered in Google to come to my site (for those who entered from Google.com via organic search result)? I'm trying to answer the question of which search phrases are resulting in completed Goals (as opposed to bouncing off the site or just any behaviour other than completing a Goal). It seems like this should be one of Analytics' default reports -- help identify which search phrases are converting well. It's probably there and I'm just not seeing it... Thanks.
Reporting & Analytics | | scanlin0