My crawl diagnostic is showing 2 duplicate content and titles.
-
First of all Hi - My name is Jason and I've just joined - How you all doing?
My 1st question then:
When I view where these errors are occurring it says www mydomain co uk and www mydomain co uk/index.html
Isn't this the same page? I have looked into my root folder and only index.html exists.
-
Thanks Daniel!!!!
Looks like I'll be spending some time in the ol Q&A section
-
Hi Jason
That's perfect(and working) for redirecting all non www pages. You still need to decide on your original question regarding the index page!
Add the following to your htaccess file after the code you've already added :
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html\ HTTP/ RewriteRule ^index\.html$ http://www.keystonemortgages.co.uk/ [R=301,L]
-
This is what I have added does it look OK?
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{HTTP_HOST} ^keystonemortgages.co.uk
RewriteRule (.*) http://www.keystonemortgages.co.uk/$1 [R=301,L]
-
Thanks Alex I'll sit back and wait to see! fingers crossed it has a positive effect
-
It's not essential but still useful to have the canonical tag, it doesn't redirect but tells the crawlers the preferred address if you have more than one page/address containing the same content. It's particularly useful for duplicate content like print versions, or results pages based on query strings.
What you've actioned could improve the rankings to your homepage as before, the search engines will have seen the two addresses as separate pages, therefore competing against each other. When the redirect comes into effect the page authority and other metrics of both will be combined into one.
-
Hi quick update:
Just this mean I won't need a canonical tag in my header? Do they do similar things?
I've created the htaccess file and dropped it into my root folder at 1and1 so I guess now I sit back for a re crawl to see if it works?
One more thing. Is this likely to effect rankings?
-
Hey thanks Daniel cheers for the welcome
That sounds simple but I ain't got a clue how to do this so I'll start searching and let you know my results
Thanks for the lead,
Jason
-
Hi Jason and welcome to seoMoz!
If you do not have a rewrite in place(in your htaccess file) then both the www.mydomain.co.uk AND www.mydomain.co.uk/index.html will resolve in your browser. I suggest doing a 301 rewrite and while you're at it make sure you also rewrite all non www varients -> eg http://mydomain.co.uk 301 redirect that to http://www.mydomain.co.uk
You can then go set your preferred version in your Webmaster Tools.
This would be the code for apache(check with your host/developer if you're unsure) :
RewriteEngine On
RewriteCond %{HTTP_HOST} ^example.co.uk
RewriteRule (.*) http://www.example.co.uk/$1 [R=301,L]
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content/Missing Meta Description | Pages DO NOT EXISIT!
Hello all, For the last few months, Moz has been showing us that our site has roughly 2,000 duplicate content errors. Pages that were actually duplicate content, I took care of accordingly using best practice (301 redirects, canonicalization,etc.). Still remaining after these fixes were errors showing for pages that we have never created. Our homepage is www.primepay.com. An example of pages that are being shown as duplicate content is http://primepay.com/blog/%5BLink%20to%20-%20http:/www.primepay.com/en/payrollservices/payroll/payroll/payroll/online-payroll with a referring page of http://primepay.com/blog/%5BLink%20to%20-%20http:/www.primepay.com/en/payrollservices/payroll/payroll/online-payroll. Some of these are even now showing up as 403 and 404 errors. The only real page on our site within that URL strand is primepay.com/payroll or primepay.com/payroll/online-payroll. Therefore, I am not sure where Moz is getting these pages from. Another issue we are having in relation to duplicate content is that moz is showing old campaign url’s tacked on to our blog page i.e. http://primepay.com/blog?title=&page=2&utm_source=blog&utm_medium=blogCTA&utm_campaign=IRSblogpost&qt-blog_tabs=1. As of this morning, our duplicate content went from 2,000 to 18,000. I exported all of our crawl diagnostics data and looked to see what the referring pages were, and even they are not pages that we have created. When you click on these links, they take you to a random point in time from the homepage of our blog; some dating back to 2010. I checked our crawl stats in both Google and Bing’s Webmaster tool, and there are no duplicate content or 400 level errors being reporting from their crawl. My team is truly at a loss with trying to resolve this issue and any help with this matter would be greatly appreciated.
Moz Pro | | PrimePay0 -
Site Crawl Error
In moz crawling error this message is appears: MOST COMMON ISSUES 1Search Engine Blocked by robots.txt Error Code 612: Error response for robots.txt i asked help staff but they crawled again and nothing changed. there's only robots.XML (not TXT) in root of my webpage it contains: User-agent: *
Moz Pro | | nopsts
Allow: /
Allow: /sitemap.htm anyone please help me? thank you0 -
How to find page with the link that returns a 404 error indicated in my crawl diagnostics?
Hi Newbie here - I am trying to understand what to do, step by step, after getting my initial reports back from seomoz. The first is regarding the 404 errors shown as high priority to fix, in crawl diagnostics. I reviewed the support info help on the crawl diagnostics page referring to 404 errors, but still did not understand exactly what I am supposed to do...same with the Q&A section when I searched how to fix 404 errors. I just could not understand exactly what anyone was talking about in relation to my 404 issues. It seems I would want to find the page that had the bad link that sent a visitor to a page not found, and then correct the problem by removing the link, or correcting and re-uploading the page being linked to. I saw some suggestions that seemed to indicate that seomoz itself will not let me find the page where the bad link is and that I would need to use some external program to do this. I would think that if seomoz found the bad page, it would also tell me what page the link(s) to the bad page exists on. A number of suggestions were to use a 301 redirect somehow as the solution, but was not clear when to do this versus, just removing the bad link, or repairing the page the link was pointing to. I think therefore my question is how do I find the links that lead to 404 page not founds, and fix the problem. Thanks Galen
Moz Pro | | Tetruss0 -
Duplicate content analysis
Good morning everyone, I have just run a test from SEOmoz PRO tool and I got more than 2,000 double content errors. How might I see which are the 2 pages whose content is double? My website is just newly revamped and can't find these similarities on my own: for me there are not). Thanks for your help! Francesca
Moz Pro | | astojanov0 -
How can I correct this massive duplicate content problem?
I just updated a clients website which resulted in about 6000 duplicate page content errors. The way I set up my clients new website is I created a sub folder calles blog and installed wordpress on that folder. So when you go to suncoastlaw.com your taken to an html website, but if you click on the blog link in the nav, your taken to the to blog subfolder. The problem I'm having is that the url's seem to be repeating them selves. So for example, if you type in in http://suncoastlaw.com/blog/aboutus.htm/aboutus.htm/aboutus.htm/aboutus.htm/ that somehow is a legitimate url and is being considered duplicate content of of http://suncoastlaw.com/aboutus.htm/. This repeating url only seems to be a problem when the blog/ is in the url. Any ideas as to how I can fix this?
Moz Pro | | ScottMcPherson0 -
Open Site Explorer is showing "No Data" for my page titles under the "Top Pages" Tab
This is for my site: www.kibin.com/ Does this mean there is something wrong with how we're titling our pages? We're working on our on-page SEO, titles, meta descriptions this week and getting them up to snuff. I don't understand what is going on here or if it's something to be worried about. Thanks!
Moz Pro | | Kibin0 -
How to remove URLS from from crawl diagnostics blocked by robots.txt
I suddenly have a huge jump in the number of errors in crawl diagnostics and it all seems to be down to a load of URLs that should be blocked by robots.txt. These have never appeared before, how do I remove them or stop them appearing again?
Moz Pro | | SimonBond0 -
Duplicate content and what to say to my webmaster
Hi, SEOmoz is telling me I have a duplicate content issue between kansascityrealestate dot com and kansascityrealestate dot com/Real-Estate-Homes-Kansas-City.asp my webmaster says Google should figure out is the same page. Suggestions on what to do and how to explain it to the webmaster? Thank you.
Moz Pro | | Ken_Jansen0