Description tag not showing in the SERPs because page is blocked by Robots, but the page isn't blocked. Any help?
-
While checking some SERP results for a few pages of a site this morning I noticed that some pages were returning this message instead of a description tag,
A description for this result is not avaliable because of this site's robot.s.txt
The odd thing is the page isn't blocked in the Robots.txt. The page is using Yoast SEO Plugin to populate meta data though.
Anyone else had this happen and have a fix?
-
I am having the same issue currently with one of my clients sites. No robots.txt is or was ever in place. I can not find any issues at all that would cause the problem. Was wondering if you every had any resolution to this? Client also uses Yoast.
-
Hi there! Has this issue been resolved? We'd love to have an update, thanks! (Christy)
-
Could the pages have been blocked in robotos.txt in the past? It can take an age for Google to re-index pages a proper after having been blocked previously.
-
Certainly doesn't appear to be any issues there. It's difficult to determine the issue without taking a look closer but its not the robots.txt file and if your site is fully indexed then I can't offer more without seeing the site.
Would you be willing to share the URL?
-
No, it's not on. The site is indexed, just certain description tags aren't.
-
It's pretty standard
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
-
I've seen people with similar issues before and it's been a small but fatal error in the robots.txt file itself. Can you please share it here and I'll advise further if needed.
-
Can you check something for me please?
Log in to wp-admin
Go to Setting then Privacy
Check if “Ask search engines not to index this site.” is set to ON
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Are there detrimental effects of having multiple robot tags
Hi All, I came across some pages on our site that have multiple robot tags, but they have the same directives. Two are identical while one is for Google only. I know there aren't any real benefits from having it set up this way, but are there any detrimental effects such as slowing down the bots crawling these pages? name="googlebot" content="index, follow, noodp"/> Thanks!
On-Page Optimization | | STP_SEO0 -
Subdomain SERPS are different from the root domain SERPS
I am fairly new to SEO so bare with me. I created two campaigns within Moz, one tracking the sudomain www.mysite.com and another tracking just the root domain mysite.com. When I look at the keyword rankings I get different results from Google, Bing and Yahoo. For instance, the root domain shows rankings in Bing and Yahoo, but "Not in top 50" for Google. For my subdomain, it is reversed. I see rankings for Google, but "not in top 50' for Bing and Yahoo. What can I do to solve this issue? I am using canonical tags, but it is a domain redirect issue? Thanks in advance
On-Page Optimization | | BouldersInn0 -
10 Mobile Pages. 5 Desktop Pages. Canonnical to where?
I have a mobile site with more pages than the desktop site. Normally I would just point the page equivalents to the desktop site using the rel canonnical tag. What about the 5 pages? Do I just leave them be? Thanks
On-Page Optimization | | Imajery0 -
Different Title Tag and Page Headline
My editorial team won't budge with their headlines which are excruciatingly vague ... But I have managed to convince them to let me optimize the title tags and the URLs. Is this sub-optimal or are there some benefits to having a title tag that varies from the page headline or what our dev team calls the "reader friendly" title? For example... Their headline: Increase Your Retirement by 20% with This Safe, Simple Strategy My title tag: Compound Returns: How to Increase Your Retirement 20% Thanks for the help, E
On-Page Optimization | | essdee0 -
Wordpress pages URL's redirection.
I was checking W3C Markup Validation and in report it was shown that that pages (not post or any other URL's just PAGES) at investmentcontrarians.com are 301 redirected. e.g. original URL "http://www.investmentcontrarians.com/debt-crisis" which is redirected to "http://www.investmentcontrarians.com/debt-crisis/" I know that its not that serious issue, but still want to know why only pages are being redirected and how can we avoid it.
On-Page Optimization | | NumeroUnoWebSolutions0 -
Lots of Listings on Page 2 SERP, but none on the first?
For one of our most important keywords we have several pages ranking highly on the second page of search results in google. Yet none on the first page, is there anyway to concentrate our rankings to bump one to the first page? We would rather have even just one link on the first page than the 4 or 5 we have on the second now.
On-Page Optimization | | absoauto0 -
Does a page's url have any weight in Google rankings?
I'm sure this question must have been asked before but I can't find it. I'm assuming that the title tag is far more important than the page's url. Is that correct? Does the url have any relevance to Google?
On-Page Optimization | | rdreich490 -
Why isn't SEOMoz using File Extensions (*.html etc) on any of their web page URLs?
...and what is the SEO benefit of this? This video from Matt Cutts suggests using file extentions, except for a directory.
On-Page Optimization | | magicrob0