"Issue: Duplicate Page Content " in Crawl Diagnostics - but these pages are noindex
-
Saw an issue back in 2011 about this and I'm experiencing the same issue. http://moz.com/community/q/issue-duplicate-page-content-in-crawl-diagnostics-but-these-pages-are-noindex
We have pages that are meta-tagged as no-everything for bots but are being reported as duplicate. Any suggestions on how to exclude them from the Moz bot?
-
Technically that could be done in your robots.txt file but I wouldn't recommend that if you want Google to crawl them too. I'm not sure if Rogerbot can do that. Sorry I couldn't be more help.
If you don't get one of the staffers on here in the next few days, I would send a ticket to them for clarification.
If you decide to go with robots.txt here is a resource from Google on implementing and testing it. https://support.google.com/webmasters/answer/156449?hl=en
-
Thanks for the information on Rogerbot. I understand the difference between the bots from Google and Moz.
Some errors reported in Moz are not real. For example we use a responsive slider on the home page that generates the slides from specific pages. These pages are tagged to no-everything so as to be invisible to bots, yet they are generating errors in the reports.
Is there anyway to exclude some pages from the reports?
-
Don't forget that Rogerbot (moz's crawler) is a robot and not an index like Google. Google used robots to gather the data but the results we see is an index. Rogerbot will crawl the pages regardless of noindex or nofollow.
Here is more info on RogerBot http://moz.com/help/pro/rogerbot-crawler
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate page titles and hreflang tags
Moz is flagging a lot of pages on our site which have duplicate page titles. 99% of these are international pages which hreflang tags in the sitemap. Do I need to worry about this? I assumed that it wasn't an issue given the use of hreflang. And if that's the case, why is Moz flagging them as an issue? Thanks.
On-Page Optimization | | ahyde0 -
Does hreflang restrain my site from being penalized for duplicated content?
I am curently setting up a travel agency website. This site is going to be targeting both american and mexican costumers. I will be working with an /es subdirectory. Would hreflang, besides showing the matching language version in the SERP´s, restrain my site translated content (wich is pretty much the same) from being penalized fro duplicated content? Do I have to implement relcannonical? Thank ypu in advanced for any help you can provide.
On-Page Optimization | | kpi3600 -
Social Media Profile and Website content dupe content issues ?
Is there a dupe content issue if your social media profiles and website have same content/copy ? For example if you have written great copy for your website home page then literally copy & paste that into your G+ co page introduction (since sums everything up perfectly and don't want to change it) would that have a negative effect or be perfectly ok ? cheers dan
On-Page Optimization | | Dan-Lawrence1 -
"Heading 1" vs. "Title" Style for SEO
In Word, you can specify "Heading 1" text which Google presumably treats the same as an HTML tag. Is there any benefit in using the "Title" style? Is it the equivalent of a web page's title?
On-Page Optimization | | BlueLinkERP0 -
How to "on page" seo a small local service business - particularly headers
First off, let me apologize if this question is posted elsewhere, worded differently. I've looked around quite a bit and have been unable to find the answer. Basically, we are a small web design firm just getting our feet with with SEO. Most of our clients, especially initially, will be quite small, local, service businesses. For example: and electrician, a pet sitter, a retail printing and map store, a surgeon etc. Almost all of their sites will follow a basic "business card on the web" format... Home Page - About Us - Testimonials - Rates - FAQ - Contact Us - Etc So, from what I've read about on-page optimization, making sure my keywords are in the title, header, body, and meta description is one of the easiest and quickest things we can do for our clients. This is a straightforward concept for me when applied to the homepage. For example, take the local pet sitting business. Her keywords are: Pet sitting, Dog walking, and the city we live in, Anytown USA. So, I've used those keywords in all the appropriate places on the home page: title: Dog Walking and Pet Sitting Service in Anytown USA header: Dog Walking and Pet Sitting Service in Anytown USA first sentence of body: We are a professional Dog Walking and Pet Sitting Service in Anytown USA meta description: We are a professional Dog Walking and Pet Sitting Service in Anytown USA. At Business Name your furry friends become a part of our family. So, my question is: Do I also optimize the "about us" page? I've changed the title of all the pages to follow this format: Dog Walking and Pet Sitting in Anytown USA - Home Dog Walking and Pet Sitting in Anytown USA - About Us Dog Walking and Pet Sitting in Anytown USA - Rates Dog Walking and Pet Sitting in Anytown USA - FAQ Dog Walking and Pet Sitting in Anytown USA - Etc Easy enough so far. Also pretty easy for the meta description, and the body. However, how would I add keywords to the header without making it look ridiculous? We use wordpress with the genesis framework, and child themes from studiopress. The header is always prominently visible at the top of the page. Most people would expect to see the header be the same as the link they clicked on the nav bar: for example, on the "about us" page, people expect the header to be: "about us" Not: "dog walking and pet sitting in Anytown USA - About Us" Do I just not worry about the headers on the other pages? For that matter, I'd really like people to "land" on the home page, not any of the other pages, so should I not optimize them at all? Does optimizing the rest of the pages help the home page to show up higher in the SERPS? If I do end up optimizing the rest of the pages, should I use slightly different spellings of the keywords: like Dog walker instead of dog walking? Or pet sitter instead of pet sitting? I've repeatedly seen people talk about not using the same keywords on more than one page... but for most of these businesses there are really fairly few keywords. There just isn't that many different ways that someone is going to search for an electrician, or a plumber, or a pet sitter. By the second or third page that I optimize on one site, I imagine I'll start running out of different variations of the keywords. I recognize that a lot of what we'll do that will be most helpful to local clients has nothing to do with on page optimization (setting up google places, google+, yahoo + bing local, etc). I'd just like to make sure that I'm doing the on page stuff as perfectly as possible. Thanks for your time and responses! -Matt p.s. while I'm at it, let me ask another question about domain names as well. Right now the pet sitting client mentioned above is using: www.petcare_Anytown_.com After operating her business for the last year she realized she is much more interested in dog walking than pet sitting. We are in the processes of redesigning the site, and when finished, are considering moving it to: www.dogwalking_Anytown_.com My assumption is that as long as we use permanent redirects from the old site to the new one, we shouldn't lose much SEO value. Is this thinking correct? On a related note though: another article I read mentioned that using a brand name in the domain may actually be more useful than the keyword rich domains above. However, www._businessname._com happens to already be taken by a pet sitting business at the other end of the country. We could however use: www.businessnameAnytown.com Which one do you think would work better? The keyword/location domain, or the businessname/location domain? Thanks!
On-Page Optimization | | Webformix0 -
Issue: Duplicate Page Content
For duplicate page content, how different should pages be? For example, I have seven locations and on each location page, we offer a discount. The discounts are the same currently and open into a pop-up window. So it looks something like this: mysite.com/locationA/dicount mysite.com/locationB/discount mysite.com/locationX/discount The pages are identical. Should I change the verbiage on each page or let it be? I noticed that our organic search rankings have dropped since our site upgrade and this is one item that SEOMOZ has noted. Thanks! DHO
On-Page Optimization | | DougHoltOnline0 -
The "100 links/page recommendation" - Do Duplicate Links Count?
We have way too many links on our homepage. The PageRank Link Juice Calculator (www.ecreativeim.com/pagerank-link-juice-calculator.php) counts them to 300. But all of them are not unique, that is some links point to the same URL. So my question: does the "100 links/page recommendation" refer to all anchors on the page or only to unique link target URLs? I know "100" is just a standard recommendation.
On-Page Optimization | | TalkInThePark0 -
Which pages should use rel="canonical" links?
I have many pages showing up as multiple content. Most of the them belong to product pages for my store, login pages that show up everywhere on the site, etc. I know that I need to use the rel=canonical link in the header but after searching the forum I'm still unsure of what pages need it. Is it the pages that I don't want searched/crawled by Google or the other way around? Thanks! Crystal
On-Page Optimization | | COfashionista0