Meta Robots Tag - What's it really mean?
-
I used on a handful of pages recently and noticed that they're still popping up in the Google search index. I'd like to keep these from appearing, so I figured I needed a directive statement with stronger semantic meaning.
From what I understand, is what I'm looking for. Using this will keep Google from not only crawling the page, but indexing the page, as well.
I decided to see what the official robotstxt.org website said about it, so I checked (link here):
the NOFOLLOW directive only applies to links on this page. It's entirely likely that a robot might find the same links on some other page without a NOFOLLOW (perhaps on some other site), and so still arrives at your undesired page.
So, is their explanation saying that the page itself will be indexed, but the content / links on it won't be followed / indexed? Let me hear your thoughts, mozzers.
-
You can find out your desired meta robots setting by asking yourself two questions:
noindex or index: do you want this particular page to show up as a listing in the Google results for a query? Yes: index, No: noindex.
nofollow or follow: do you want the links on this page to be crawled (followed) by the search engine robots, thereby allowing them access to the deeper pages it links to (even though that particular page might even be noindexed). Yes: follow, No: nofollow
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sudden drop in traffic since website redesign – can't spot double firing sessions pre-update
Hi all, We launched an updated website around mid-April and have seen a long-term drop in unique page views and sessions since then. Between 25-35%. Page views returned to normal within a month, however. In terms of acquisition, we've not lost referrals or direct traffic, just organic sessions. I haven't seen any significant Google updates except for 'Fred' but that was in March (pre-website update). We did have to do some 301s and fix up an HTML and XML sitemap, etc, post launch – but that's been in place for a while now so shouldn't still have such an effect? We are also struggling with load speed at the moment, working on this at the moment. Didn't think this had a such a significant impact on ranking however. We've lost mostly non-UK traffic, especially USA. We're actually climbing rankings for a lot of new keywords I'm targeting, so I can't see how our SEO has been hampered. Frustratingly, I wasn't using Search Console or Moz Pro before this update, so can't compare much in the way of keyword traffic (I know I feel very silly now). I don't know if it's a reporting issue (I wish it was). I mean we had both GTM and GA tags on the website (I didn't know this until I looked into it) but these were present post website-launch. And I haven't found enough double firing sessions or page views to suggest we used to be getting double figures anyway. Before this, we were ranking brilliantly and getting more organic traffic than ever. All the best,
Reporting & Analytics | | landport0 -
Canonical Tags & GWT Parameters
A site I'm working on has canonical tags which I find to be accurate, regardless of tracking parameters or anything else added to the url. The tag looks like: And we have alot of parameters in Google Search Console that look like Parameter Crawl page Let Googlebot Decide destination Let Googlebot Decide filters Let Googlebot Decide Since all of our parameters follow a question mark, like http://www.examplesite.com/questions/avocados?source=ad12345 and all of our pages have canonical tags showing the representative url without the additional parameters, why wouldn't we just have the one parameter in GWT as Parameter Crawl ? Representative URL I ask because I find that Google analytics shows pages with parameters as landing pages in search, which has me concerned about Google seeing it as duplicate content. Thanks! Best... Darcy
Reporting & Analytics | | 945010 -
No Data in Custom Report set to 'Hit' Scope
Hi Guys, Been having a problem recently with a custom report I have set up... I want to find out number of sessions, bounce rate, session duration etc for different dimensions on my site - store area, store name, product type etc but I cannot seem to get the data to filter through to the report I have set up when 'Session' scope is selected. If I set it to 'hit' then I do get the data but this will only record the first instance of a dimension being triggered (from what I can gather) rather than all dimensions that might be triggered during a complete session. Has anyone experienced similar problems? Thanks, Dan
Reporting & Analytics | | SEOBirmingham810 -
Ok. I'm just going to cut loose with my stupid question. What is internal link equity? What distinguishes an internal link with equity from one without equity?
What distinguishes an internal link with equity from one without equity? Is there a limit to how many of these I want? What's the rule-of-thumb? Cheers, Wes
Reporting & Analytics | | wrconard0 -
What is s.ytimg.com in google analytics?
My clients GA reports 273 visits from s.ytimg.com. I go to the site, it doesn't exist. I googled it, there were some code with s.ytimg.com in it, but nothing I could understand. Anybody have an idea where this comes from?
Reporting & Analytics | | endlessrange0 -
What's the difference between landing pages and entrance pages on Google Analytics?
I'm confused about the difference between entrance pages and landing pages on Google Analytics. If I compare our search traffic to entrance pages with search traffic to landing pages it seems very similar -- but not identical. But this is probably because all of our GA analytics is sampled (we're a huge site). Can anyone help?
Reporting & Analytics | | CecilyP0 -
Has Google changed its algorithm? My traffic has almost doubled and I don't know why.
My traffic has nearly doubled over the past few days, and my conversion rate has doubled as well. It looks like our rankings haven't changed... We haven't done anything to the site recently, although we did submit a couple of press releases through the wire not too long ago (within the last few weeks). I do not think that this is a cyclical/seasonal jump in traffic because last year this did not occur. Any ideas?
Reporting & Analytics | | deuce1s0 -
Tracing Google Analytics 'goal' back to original search phrase
I added Goals to my Google Analytics tracking. It's working; I get visitors who have completed Goals showing up in the reporting. My question is: Is it possible to trace backwards from a completed Goal to the original search phrase a user entered in Google to come to my site (for those who entered from Google.com via organic search result)? I'm trying to answer the question of which search phrases are resulting in completed Goals (as opposed to bouncing off the site or just any behaviour other than completing a Goal). It seems like this should be one of Analytics' default reports -- help identify which search phrases are converting well. It's probably there and I'm just not seeing it... Thanks.
Reporting & Analytics | | scanlin0