Welcome to the Q&A Forum

Not sure about you all, but I’m loving the new Moz Site Crawler.  However, I was noticing that it is identifying a huge amount of pages as duplicate content.

There are about 30,000 pages in this website, with that said we’ve had to make many templates to make the site scalable.  Additionally a url rule was lost which caused a significant amount of duplicate pages to be created.  I am working through using the moz crawl tool to identify duplicate pages but noticing many pages under “Affected Pages,” are actually unique content pages with initial content that is duplicate.

I read that Moz flags any pages with 90% or more content overlapping content or code.  My theory for this is that some templates that are too similar, to the point that Moz reads them as duplicative.  Has this happened for anyone else?

In addition, if Moz is flagging these similar pages as duplicate content, do we surmise that Google bots are having the same issue? We have seen issues with rankings as it pertains to the actual duplicate pages but hadn't experienced issues across the unique pages, they are hyperlocal pages so we are able to see rankings quite easily.

We had an internal debate about the importance of having a sitemap.xml on your website. Basically, there is Google documentation that indicates a sitemap.xml is due diligence: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156184

And other authoritative forums, blogposts, etc. which indicate that sitemap creation and maintenance is a waste of your time, e.g.  http://webmasters.stackexchange.com/questions/4803/the-sitemap-paradox/

A bigger question is: Are there cases in which not having a sitemap.xml actually became detrimental or risky?

Thanks in advance!

Thanks to both of you, I appreciate your reply and that post was helpful. The website I'm talking about is very well optimized w/onpage content, conversion points, etc., and has a strong number of links compared to competition. Local SEO could really bring some strong results for them, so I might go that route.

I'm considering hiring an SEO vendor for link building, and assessing whether or not my website really needs link building, vs. another SEO priority (for instance, I KNOW there's an opportunity for local SEO). What factors should I look at to determine if link building is a high priority? Should I just compare the number links my site has with competitors, using a tool like Open Site Explorer?