Blocked by Meta Robots.
-
Hi,
I get this warning on my reporting.
- Blocked by Meta Robots - This page is being kept out of the search engine indexes by meta-robots.
what does that means ? and how to solve that, if i using wordpress as my website engine.
and about rel=canonical , in which page I should put this tag, in original page, or in copy page ?
thanks for all of your answer, it will be means a lot
-
There are wordpress plugins you can use to modify your robots.txt, wordpress makes it difficult.
http://yoast.com/example-robots-txt-wordpress/
Also, make sure it is an important page for your blog- Google is just being proactive on your behalf, it might be an irrelevant page to your overall plan.
-
Actually it would not be in the meta robots noindex. The meta tag does not prevent Google from crawling the page it is on. If it did that, then Google would not be able to crawl the page and then it would not be able to read the tag :-). The meta robots will tell Google to remove the page from the index and so it is very effective for that application.
That said, the GWT warning, is probably related to you robots.txt file located at
http://www.yourdomain.ext/robots.txt
Put that in your browser and see if you have any of your files, pages Disallowed in that file. If that is the case, then Google will not be able to spider a page to start with, let alone read the meta tags. Do some searching on Google on how robots.txt works Moz obviously has one
http://moz.com/learn/seo/robotstxt
Here is a video on how to use Wordpress and robot.txt - it may or may not relate to your config, but will show a plugin that you can use to adjust
http://www.youtube.com/watch?v=JY9A5OqHTvw
You can figure out how to understand it and then what you need to update it. Get with your IT person or whoever admins your site
-
It means there is a meta tag on the page that is blocking the page. Look in the head section of the page for a tag. Remove this and you should be good to go. Check your WordPress settings, sometimes these tags are automatically assigned to pages as a default. You could also download a SEO plug in to help manage the meta robots tags.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Only Our Homepage Can Be Crawled Showing a Redirect Message as the Meta Title
Hello Everyone, So recently when we checked our domain using a Moz Crawl Test and Screaming Frog only the homepage comes up and the meta title says “You are being redirected to…”. We have several pages that used to come up and when submitting them to GSC no issues come up. The robots.txt file looks fine as well. We thought this might be ‘server’ related but it’s a little out of our field of expertise so we thought we would find out if anyone has any experience with this (ideas of reasons, how to check…etc.) or any potential suggestions. Any extra insight would be really appreciated. Please let us know if there is anything we could provide further details for that might. Looking forward to hearing from all of you! Thanks in advance. Best,
Moz Pro | | Ben-R0 -
Should I set blog category/tag pages as "noindex"? If so, how do I prevent "meta noindex" Moz crawl errors for those pages?
From what I can tell, SEO experts recommend setting blog category and tag pages (ie. "http://site.com/blog/tag/some-product") as "noindex, follow" in order to keep the page quality of indexable pages high. However, I just received a slew of critical crawl warnings from Moz for having these pages set to "noindex." Should the pages be indexed? If not, why am I receiving critical crawl warnings from Moz and how do I prevent this?
Moz Pro | | NichGunn0 -
Should I block .ashx files from being indexed ?
I got a crawl issue that 82% of site pages have missing title tags
Moz Pro | | thlonius
All this pages are ashx files (4400 pages).
Should I better removed all this files from google ?0 -
Moz says I am missing titles and meta tags and have duplicate content
I just redesigned my website and suddenly traffic has dropped. Moz says I am missing titles, meta tag descriptions and have a lot of duplicate content. My site is http://skigenie.com and is full of unique and hand written content,. Are there any wordpress plugins that will add titles etc to my pages (some of the added ones are custom)? Any help would be much appreciated!
Moz Pro | | flexy0 -
Updating Meta Keywords
Hi I am going through the process of cleaning up the SEO on my blog www.shoottokyo.com. Someone recommended that I can use ScreamingFrog to find the location of 4xx errors and I noticed that there are Meta Keywords on about 200 of my posts but some of them are wrong such as it mentions my old city I lived in or my old camera I used to use. I want to clean these up. If I look on the post itself in Wordpress I don't even this this information. Where can I edit it? Is there a way to easily edit across multiple posts? I previously used All in One SEO perhaps these came from that and I need to reinstall that to clean this up? I'm new to all of this expect a lot of questions. Thanks Dave
Moz Pro | | ShootTokyo0 -
Does the page authority data also considers the on page factors like the presence of keyword in the title,meta text, and keyword frequency ??
The moz difficulty score considers four factors for the top websites. are the on page factors included in the page authority data ?
Moz Pro | | iQuanti0 -
Sites Blocking Open Site Explorer? Penguin related.
Last week I was looking at a competitors site who has a link scheme going on and I could actually check the links for each anchor text. This week they don't work at all, do you think they're blocking the rogerbot on their domains? Or is there a problem with open site explorer? http://www.opensiteexplorer.org/anchors?site=www.decks.ca If you're interested in the background, all the links are to instant-home-biz . com which then redirects decks . ca - it's a tricky technique. Pretty much all of the links are from sketchy sites like: airpr23.xelr8it.biz/ airpr23.anzaland.net/ airpr23.vacation-4-free.com/airpr23.blogfreeradio.net/airpr23.blogomatik.com/http://www.morcandirect.com/mortgages/resources2.php which I thought Penguin was supposed to catch…
Moz Pro | | BeTheBoss0 -
Robots review
Anything in this that would have caused Rogerbot to stop indexing my site? It only saw 34 of 5000+ pages on the last pass. It had no problems seeing the whole site before. User-agent: Rogerbot Disallow: /default.aspx?*
Moz Pro | | sprynewmedia
//Keep from crawling the CMS urls default.aspx?Tabid=234. Real home page is home.aspx Disallow: /ctl/
// Keep from indexing the admin controls Disallow: ArticleAdmin
// Keep from indexing article admin page Disallow: articleadmin
// same in lower case Disallow: /images/
// Keep from indexing CMS images Disallow: captcha
// keep from indexing the captcha image which appears to be a page to crawls. general rules lacking wildcards User-agent: * Disallow: /default.aspx Disallow: /images/ Disallow: /DesktopModules/DnnForge - NewsArticles/Controls/ImageChallenge.captcha.aspx0