641 Crawl Errors In My Moz Report - 190 are high priority Duplicate Content
-
Hi everyone,
There are high and medium level errors. I was surprised to see any especially since Google Analytics shows no errors whatsoever.190 errors - duplicate content.A lot of images are showing in the Moz Crawl Report as errors, and when I click on one of these links in the report, it directs to the image which displays on a blog post on the site unusually since I haven't started blogging yet.. So it looks like all those errors are because the images are appearing on their own post.So for example a picture of a mountain would be referred to with www.domain.com/mountains ; the image would be included in the content on a page but why give an image a page/post all of it's own when that was not my intention. Is there a way I can change this?# ----------------------------------------
These are things I first see at the top of the Moz Report:There are 2 similar home urls at the top of the report:
http status code is 200 for both (1) and (2)
Link Count for (1) is 71. Link count for (2) is 60.
No client or server errors
Rel Canonical Rel-Canonical Target
Yes http:// domain. co.uk/home
Yes http:// domain. co.uk/home/Does this mean that the home page is being seen as a duplicate by Google and the search engines?http status codes on every page is 200.Your help would be appreciated.Best Regards,
-
Hi ,
Many thanks for responding. Yes definitely a few issues for sure it's been quite frustrating so far and I have been looking at ways to get the site fully correct: Traffic lost, 3 Htaccess files, 2 robots.tx files. A new site was built on a subdomain and then migrated to the live domain.
I looked at the file structure and I saw lots of duplicate images on the subdomain where the new version of the website was created, and lots of duplicates on the current site. I will be running another Moz Crawl test to see if the removal of the duplicates makes an impact on the 190 errors. I will keep you updated so please leave this topic discussion. I think perhaps part of what has happened is that some of the images have previously uploaded but have not loaded into the right folder, but to the home directory (hence to domain.com/exampleimage instead of domain.com/wp-content/uploads/exampleimage. Also I think that an add was done during the migration which was done by a 3rd party, instead of doing a replacement of the destination files; but I can't confirm or guarantee this.
- I will check the link that you kindly left. It's 4am here in the UK so I will read it before I retire for the evening:
https://moz.com/community/q/cms-pages-multiple-urls
- The CMS is WordPress
Many thanks for your time and for your much needed reply.
Kind Regards.
-
Hi there,
Sounds like you are dealing with a bunch of issues here.
I suggest the first thing you address is the trailing slash issues
http:// domain. co.uk/home
http:// domain. co.uk/home/are different urls in Googles eyes despite being the same on your website.
To fix this take a loot at a previous post by myself figuring it out
https://moz.com/community/q/cms-pages-multiple-urls
This should hopefully cut down the number of errors appearing dramatically. Make sure your only ever using one version of link either with or without a trailing slash. Make the decision and always use that format.
As for your images appearing on a page of their own. We would need to know your CMS platform. A link to your site would also be helpful as you have so many errors its kinda a stab in the dark to figure it out for you without looking
-
I did find this on Moz dating back to 2014 and there was still no fix for the problem back then?
https://moz.com/community/q/duplicate-content-wordpress-image-attachement
Please can you help. Thank you
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I am confuse with google analytic custom and segment report
Hi All, In google analytic when I create custom report for my ecommerce site then figures go mad. I really not able to judge peformance of device, browser and it's version, conversion, ecommerce conversion rate etc. same way if I add secondary dimension in report then also figures are not accurate. Again when I create different segment like desktop, mobile, tablet then in tablet segment mobile devices comes and in mobile device tablet appear why segment also not accurate? Is it because I am using free version? Also do we have alternative of google analytic which give same report like google analytic either device, browser, os, segment, enhance ecommerce etc? Thanks!
Reporting & Analytics | | dhisman0 -
Google Webmaster indicates robots.text access error
Seems that Google has not been crawling due to an access issue with our robots.txt
Reporting & Analytics | | jmueller0823
Late 2013 we migrated to a new host, WPEngine, so things might have changed, however this issue appears to be recent. A quick test shows I can access the file. This is the Google Webmaster Tool message: http://www.growth trac dot com/: Googlebot can't access your site January 17, 2014 Over the last 24 hours, Googlebot encountered 62 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 8.8% Note the above message says 'over the last 24 hours', however the date is Jan-17 This is the response from our host:
Thanks for contacting WP Engine support! I looked into the suggestions listed below and it doesn't appear that these scenarios are the cause of the errors. I looked into the server logs and I was only able to find 200 server responses on the /robots.txt. Secondly I made sure that the server wasn't over loaded. The last suggestion doesn't apply to your setup on WP Engine. We do not have any leads as to why the errors occurred. If you have any other questions or concerns, please feel free to reach out to us. Google is crawling the site-- should I be concerned? If so, is there a way to remedy this? By the way, our robots file is very lean, only a few lines, not a big deal. Thanks!0 -
Moz Crawler suddenly reporting 1000s of duplicates (BE.net)
In the last 3-4 days we've had several thousand 'duplicate content' warnings appear in our crawl report, 99% of them related to our on-site blog. The blog is BlogEngine.Net, but the pages simply don't exist. The majority seem to be Roger trying quasi-random URLs like:
Reporting & Analytics | | Progauto
/?page=410 /?page=151 Etc. etc. The blog will present content for these requests, but it is of course the same empty page since there's only unique content for up to /?Page=10 or so. Two questions: 1. Did something change recently? These blogs have been up for months, and this problem has only come up this week. Did Roger change to become more aggressive lately? 2. Suggested remediation? On one of the blogs I've put no-index no-follow for any page that has a /?page querystring, and we'll see what effect that has come next crawl next week. However, I'm not sure this will work as per: http://moz.com/community/q/functionality-of-seomoz-crawl-page-reports Anyone else had dynamic blogs suddenly blossom into thousands of duplicate content warnings? Google (rightly) ignores these pages completely.0 -
How to get crawled pages indexed?
Hi, I've got over 1k pages crawled but approx 100 pages indexed. Although, i submit them on Google Fetch and the links are indexable,they are not indexed. What shall i do the get max pages indexed? Any input highly appreciated. Thanks!
Reporting & Analytics | | Rubix0 -
Duplicate content warnings
I have a ton of duplicate content warnings for my site poker-coaching.net, but I can't see where there are duplicate URLs. I cannot find any function where I could check the original URL vs a list of other URLs where the duplicate content is?
Reporting & Analytics | | CatfishTPA0 -
Duplicate content? Split URLs? I don't know what to call this but it's seriously messing up my Google Analytics reports
Hi Friends, This issue is crimping my analytics efforts and I really need some help. I just don't trust the analytics data at this point. I don't know if my problem should be called duplicate content or what, but the SEOmoz crawler shows the following URLS (below) on my nonprofit's website. These are all versions of our main landing pages, and all google analytics data is getting split between them. For instance, I'll get stats for the /camp page and different stats for the /camp/ page. In order to make my report I need to consolidate the 2 sets of stats and re-do all the calculations. My CMS is looking into the issue and has supposedly set up redirects to the pages w/out the trailing slash, but they said that setting up the "ref canonical" is not relevant to our situation. If anyone has insights or suggestions I would be grateful to hear them. I'm at my wit's end (and it was a short journey from my wit's beginning ...) Thanks. URL www.enf.org/camp www.enf.org/camp/ www.enf.org/foundation www.enf.org/foundation/ www.enf.org/Garden www.enf.org/garden www.enf.org/Hante_Adventures www.enf.org/hante_adventures www.enf.org/hante_adventures/ www.enf.org/oases www.enf.org/oases/ www.enf.org/outdoor_academy www.enf.org/outdoor_academy/
Reporting & Analytics | | DMoff0 -
Best practice SEO/SEM/Analaytics/Social reports
Hi All, does anyone have a best practice excel spreadsheet of a internal report we should be using.... ie what are the main factors we should be tracking? Unqiue views? time spent on site? Where they came from? seo/sem/network/direct to site? social media tracking? amount of +1/fb likes/tweets etc thanks
Reporting & Analytics | | Tradingpost0