Description tag not showing in the SERPs because page is blocked by Robots, but the page isn't blocked. Any help?
-
While checking some SERP results for a few pages of a site this morning I noticed that some pages were returning this message instead of a description tag,
A description for this result is not avaliable because of this site's robot.s.txt
The odd thing is the page isn't blocked in the Robots.txt. The page is using Yoast SEO Plugin to populate meta data though.
Anyone else had this happen and have a fix?
-
I am having the same issue currently with one of my clients sites. No robots.txt is or was ever in place. I can not find any issues at all that would cause the problem. Was wondering if you every had any resolution to this? Client also uses Yoast.
-
Hi there! Has this issue been resolved? We'd love to have an update, thanks! (Christy)
-
Could the pages have been blocked in robotos.txt in the past? It can take an age for Google to re-index pages a proper after having been blocked previously.
-
Certainly doesn't appear to be any issues there. It's difficult to determine the issue without taking a look closer but its not the robots.txt file and if your site is fully indexed then I can't offer more without seeing the site.
Would you be willing to share the URL?
-
No, it's not on. The site is indexed, just certain description tags aren't.
-
It's pretty standard
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/ -
I've seen people with similar issues before and it's been a small but fatal error in the robots.txt file itself. Can you please share it here and I'll advise further if needed.
-
Can you check something for me please?
Log in to wp-admin
Go to Setting then Privacy
Check if “Ask search engines not to index this site.” is set to ON
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google crawler showing cache of another page
For the page http://www.thinkdigit.com/top-products/Laptops-and-PCs/top-10-laptops-124.php google is showing another page in cache (http://www.thinkdigit.com/top-products/Ultrabooks/top-10-ultrabooks-153.php). Please let me know how this happened and how to correct it.
On-Page Optimization | | 9dot90 -
Page Analysis - Helping Product Pages Outrank Search Results Pages
Hi! We have a lot of our search results pages that have been indexed and outrank our product pages and in some instance the actual product pages barely show up at all. Here is an example query that includes our brand name: http://goo.gl/cgB6W So, we have loads of actual product pages, video pages, etc that should be showing up here, but are not and this is just one example. Unfortunately, there are a LOT of these Search Results pages out there and utlimately we would love to de-index them altogether, but it's going to have to be carefully done. So, was wondering if anyone would want to check out one of our product pages and give any feedback as to what we could change to possibly improve rank or to make them more search friendly or hopefully to help them rise above these indexed search results pages? Here is an example product page: http://goo.gl/2R4IT Thanks!! Craig
On-Page Optimization | | TheCraig0 -
I have more pages in my site map being blocked by the robot file than I have being allowed to be crawled. Is Google going to hate me for this?
Using some rules to block all pages which start with "copy-of" on my website because people have a bad habit of duplicating new product listings to create our refurbished, surplus etc. listings for those products. To avoid Google seeing these as duplicate pages I've blocked them in the robot file, but of course they are still automatically generated in our sitemap. How bad is this?
On-Page Optimization | | absoauto0 -
On page link question, creating an additional 'county' layer between states and zips/cities
Question We have a large site that has a page for all 50 states. Each of these pages has unique content, but following the content has a MASSIVE amount of links for each zip AND city in that state. I am also in the process of creating unique content for each of these cities and zips HOWEVER, I was wondering would it make sense to create an additional 'county' layer between the states and the zips/cities. Would the additional 'depth' of the links bring down the overall rank of the long tail city and zip pages, or would the fact that the counties would knock the on page link count down from a thousand or so, to a management 50-100 substantially improve the overall quality and ranking of the site? To illustrate, currently I have State -> city and zip pages (1200+ links on each state page) what i want to do is do state -> county (5-300 counties on each state page) -> city + zip (maybe 50-100 links on each county page). What do you guys think? Am I incurring some kind of automatic penalty for having 1000+ links on a page?
On-Page Optimization | | ilyaelbert0 -
What's the best practice for handling duplicate content of product descriptions with a drop-shipper?
We write our own product descriptions for merchandise we sell on our website. However, we also work with drop-shippers, and some of them simply take our content and post it on their site (same photos, exact ad copy, etc...). I'm concerned that we'll loose the value of our content because Google will consider it duplicated. We don't want the value of our content undermined... What's the best practice for avoiding any problems with Google? Thanks, Adam
On-Page Optimization | | Adam-Perlman0 -
How to properly remove pages and a category from Google's index
I want to remove this category http://www.webdesign.org/web-design-news-all/ and all the pages in that category (e.g. http://www.webdesign.org/web-design-news-all/7386.html ) from Google's index. I used the following string in the "Reomval URS" section in Google Webmaster Tools: http://www.webdesign.org/web-design-news-all/* is that correct or I better use http://www.webdesign.org/web-design-news-all/ ? Thanks in advance.
On-Page Optimization | | VinceWicks0 -
To enter keyword meta tags or to not enter keyword meta tags?
I've been doing SEO for awhile, but new to SEOMoz. I'm surprised that SEOMoz does not recommend keyword meta tags. I didn't enter them for the longest time because I know Google doesn't care about them. However, I did read that other search engines DO use them. And therefore that is why you should have them. I teach my customers about SEO, and I know it would be much easier for them not to enter or worry about the keyword meta tags. However, I would love to hear opinions here. And to Bing/Yahoo put any weight into them or is it only really small search engines? Thanks! Hilary
On-Page Optimization | | endlessrange0 -
Can I have a strong brand category page and a strong product page?
It seems Google base and other Comparison Shopping Engines like to see the brand in the product name. But, on my category page for that brand, website optimizer tells me including the brand name with each product is cannabilizes links. For example; I have a page for jewelerABC with 20 pieces of jewelry listed as well as original content about jewelerABC. I do not currently name these products as xyz by jewelerABC. This page comes up nicely in the serps. But in Google base The top listings for jewelry by jewelerABC seem to have every product named xyz by jewelerABC or JewelerABC xyzs. What is the best way to optimize.for both? Stephen
On-Page Optimization | | stephenfishman0
