Description tag not showing in the SERPs because page is blocked by Robots, but the page isn't blocked. Any help?
-
While checking some SERP results for a few pages of a site this morning I noticed that some pages were returning this message instead of a description tag,
A description for this result is not avaliable because of this site's robot.s.txt
The odd thing is the page isn't blocked in the Robots.txt. The page is using Yoast SEO Plugin to populate meta data though.
Anyone else had this happen and have a fix?
-
I am having the same issue currently with one of my clients sites. No robots.txt is or was ever in place. I can not find any issues at all that would cause the problem. Was wondering if you every had any resolution to this? Client also uses Yoast.
-
Hi there! Has this issue been resolved? We'd love to have an update, thanks! (Christy)
-
Could the pages have been blocked in robotos.txt in the past? It can take an age for Google to re-index pages a proper after having been blocked previously.
-
Certainly doesn't appear to be any issues there. It's difficult to determine the issue without taking a look closer but its not the robots.txt file and if your site is fully indexed then I can't offer more without seeing the site.
Would you be willing to share the URL?
-
No, it's not on. The site is indexed, just certain description tags aren't.
-
It's pretty standard
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/ -
I've seen people with similar issues before and it's been a small but fatal error in the robots.txt file itself. Can you please share it here and I'll advise further if needed.
-
Can you check something for me please?
Log in to wp-admin
Go to Setting then Privacy
Check if “Ask search engines not to index this site.” is set to ON
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
.htaccess file uploaded, website won't load
I uploaded the .htaccess file with the below, and now my website won't load at all? Then I deleted the htaccess file and it still won't load? But then it would load on my phone when I took it down, not on chrome, or explorer? Then I put it back up and looked again on my phone, wouldn't load on phone. Then deleted file and it still won't load on my phone? What is going on? RewriteEngine on
On-Page Optimization | | dwebb007
RewriteCond %{HTTP_HOST} !^http://freightetc.com$
RewriteRule ^(.)$ http://www.freightetc.com/$1 [R=301]
RewriteCond %{THE_REQUEST} ^./index.php
RewriteRule ^(.*)index.php$ http://www.freightetc.com/$1 [R=301]0 -
Meta titles and description for subcategory pages in CMS websites
I have a CMS based site and for that I am getting duplicate page title and description error message, for my all sub-category pages. What can I do for this because write Meta title and description for each page for CMS based website is not possible.
On-Page Optimization | | 1akal0 -
Help with the indexation of my page
Hi all, I have a problem with my website. When writing site:www.pinesapiensa.com there're no pages indexed although the webmaster tools tells me that the sitemap file has been processed in 13 May and the number of indexed paged are 21. ¿What could be happening? I have to mention that there are two domains "www.piensapiensa.es" and "www.piensapiensa.com" addressing the same website and there's a redirection from piensapiensa.com to piensapiensa.com but it doesn't work properly. Thanks
On-Page Optimization | | juanmiguelcr0 -
How do i block an entire category/directory with robots.txt?
Anyone has any idea how to block an entire product category, including all the products in that category using the robots.txt file? I'm using woocommerce in wordpress and i'd like to prevent bots from crawling every single one of products urls for now. The confusing part right now is that i have several different url structures linking to every single one of my products for example www.mystore.com/all-products, www.mystore.com/product-category, etc etc. I'm not really sure how i'd type it into the robots.txt file, or where to place the file. any help would be appreciated thanks
On-Page Optimization | | bricerhodes0 -
On page "F" and I changed my tags long ago to match everything and still F?
My on-page analysis shows that my title tags are not supporting my keyword..........but they are! My title tag has my EXACT keyword phrase. What gives? Keyword is "San Diego Party Bus" Title is "San Diego Party Bus | xoxoxox | xoxoxoxo" F grade! ????????
On-Page Optimization | | DrewSpinoso0 -
How do I add a meta description to the individual pages on my cms driven site?
Hello, As you can tell from my question, I'm COMPLETELY new to SEO. SEOMOZ flagged some pages as lacking a meta description. How do I add a meta description to the site? The site has a CMS architecture so I can edit text on the individual pages, but I don't know how to add a meta description, or where on the page it should go. Thanks!
On-Page Optimization | | OTSEO0 -
Advice on why page isn't being indexed in google top 1000 for keyword
Hi, http://www.cgcomposites.com.au/composite-material.html This page isn't listed for keywords 'composite material' It has been live for a few weeks and gets grade A report for onpage optimisation? regards Michael
On-Page Optimization | | bluelilyseo0 -
Are duplicate titles an issue for pages I don't need ranking for?
A client has a load of duplicate page titles on their site. However, to cut a long story short, most of these pages are pointless and therefore we don't need ranking for them. As such, I'm not concerned whether any of the pages with duplicate content on them are ranked or not..... unless having duplicate page titles / content on these pages could mean that other pages on the site, like the homepage, don't rank as high because of this. Do I need to worry about duplicate titles on these pages, or can I ignore duplicate content on pages that I don't want to be ranked? Hope that makes sense!
On-Page Optimization | | RiceMedia0
