The pages that add robots as noindex will Crawl and marked as duplicate page content on seo moz ?
-
When we marked a page as noindex with robots like
{<meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noindex</a>" />}
will crawl and marked as duplicate page content(Its already a duplicate page content within the site. ie, Two links pointing to the same page).So we are mentioning both the links no need to index on SE.But after we made this and crawl reports have no change like it tooks the duplicate with noindex marked pages too.
Please help to solve this problem.
-
What you're seeing is normal. What you want to check is that the number of pages is the number you expect.
From the SEOMoz FAQ on the Crawl Report:
We are still seeing duplicate content on SEOmoz even though we have marked those pages as "noindex, follow." Any ideas why?
SEOMoz is not a search engine index, it uses a crawler. If those pages are not blocked by the robots.txt file, then SEOMoz will crawl them. They ignore the noindex tag because they don't index anything. Search engines will honor the noindex tag and not index a page if you specify with the robots meta tag. However, to remove pages from the crawl, disallow them in the robots.txt or metarobots.(http://www.seomoz.org/help/crawl-diagnostics)
Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
moz crawl is stopped?
moz stopped indexing the links due to some updates? can some one confirm me thanks
Moz Pro | | 42409300125323700 -
Abnormal crawl issues appearing in my Moz results
I have been asked to look at a site for a friend and was more than surprised to see 16,9k crawl issues appear in the dashboard... of this 6,238 are duplicate page content and 5878 are duplicated page titles. What on earth is going on? I have spoken to the web developer as it appears there is a dev site somewhere and this is his response [Can I stress that Google determines which site was in the index first and then removes other sites it sees as having duplicate content. Our dev sites appearing in the search index would not affect your ranking due to duplicate content as Google would see your site as the first site with the content] As I cannot make contact with him, I am scratching my head, surely a dev site should be no-indexed, it sounds as though he is saying that its ok because Google will take the main site as the first site with the content... Very confused! Help need MOZ community. Manythanks, Sarah
Moz Pro | | Mutatio_Digital0 -
New to Moz, need some probably basic answers about Keywords, Linking, Competitors and General SEO
Hi, So I have quite a lot of data colelcted about my site now, regarding keyword research, page crawling and competitor research ect. But I find myself second guessing myself about what I have done and what to do next. I have done basic research for as many relevant keywords I could think of to my site, including branded and non branded terms. If the main competitive keywords for my niche are very competitive, shall I start doing more research for long tail keywords and only try to rank for them? Does is matter how many keywords I am doing research for? Does is matter how many keywords I try to optimise for each webpage? Are the amount of branded keywords I am researching skewing my results? As they are all ranked #1, but nearly all of the non branded keywords are much further down the list... Once I have decided what keywords are worth trying to ranking for for each page, are the techniques to actually rank more highly for them - Title, H1 Tag, Description, Meta Data, Fresh Content and using the keywords on the page? Or are there more techniques I haven't heard of? Under Keyword Rankings - I noticded that some of my keywords are directing to specific pages, like "Cavity Waxes" is directing to the URL ending in .com/cavity-waxes - How do you assign the keywords im researching to specific URLs? - Or does Moz do it automatically? As most of my keywords seem to be unassigned to any URL, is that because they are not ranking highly enough? How do I best use the data collected through Moz? Good practices? Techniques? Tips and Tricks? What is the best practice for finding potential link partners and asking them for mutual linking? Techniques for finding partners that are likely to link with us, but still provides link juice. I must apologise for this long-winded set of questions, but these are troubling me! Any help would be greatly appreciated, Kind regards, Max Johnson
Moz Pro | | BiltHamber10 -
How long for authority to transfer form an old page to a new page via a 301 redirect? (& Moz PA score update?)
Hi How long aproximately does G take to pass authority via a 301 from an old page to its new replacement page ? Does Moz Page Authority reflect this in its score once G has passed it ? All Best
Moz Pro | | Dan-Lawrence
Dan3 -
Crawl Diagnostics - Crawling way more pages than my site has?
Hello all, I'm fairly new here, more of a paid search guy dabbling in SEO on the side. I have a client that I have in SEOMoz and the Crawl Diagnostics report is showing 10,000+ pages crawled and I think the site has at most 800 pages (e-commerce site using freewebstore.org as the platform). Any reasons this would be happening?
Moz Pro | | LodestoneGen0 -
New on SEO Moz and need help regarding panda recover.
Hello Dear Members, I am a new member to not only seomoz but to the SEO World as well. I was learning and implementing SEO from the last 5,6 months and got my website on Google.com page 1. I was quite happy with all the results. I recently got adsense approved on my site and was quite happy with the earnings as well. Unfortunately the most recent Panda update has effected most of my SEO efforts I was doing from the last 5,6 months. My rankings are effected a lot and same with my earnings from adsense. My rankings are effected in this manner. A couple of keywords that were in top 3 spots (1st page Google.com) are now at 7,8 (1st page Google.com) position. There are 3,4 pages that were previously on 5,6 spot (1st page Google.com) are now on 15, 16 spot (2nd page Google.com). Now I want to ask what tools from seomoz can help me to regain my lost positions in Google? Secondly what type of strategy (backlinks) can help me to regain the lost rankings? Best Regards Sam
Moz Pro | | sampaul5490 -
How to remove /index.html that causes duplicated content
Hi, How to remove /index.html that causes duplicated content?
Moz Pro | | whitelies
From my website navigation links, it does not shows the /index.html. However, when I run the seomoz crawl errors, it show duplicated content. Can anyone tell me how to do it?0 -
Is there a Tool to compare Duplicate content for non web Live content?
Is there a tool that can give me % of duplicate content when comparing two pieces of content that are not Live on the web? Like copyscape but for content that may not be indexed by copyscape or not live on the web? Does Word or any other program allow you do do this?
Moz Pro | | bozzie3110