Duplicate pages coming from links from the login page - what should we do about them?
-
This is a follow on to an earlier question which was well answered by Dirk Ceuppens regarding abnormal crawl issues. We are seeing that the issues relating to Duplicate Pages are coming from links from the login page which shows information about where the user was redirected from.
For example, if the visitor is not logged on and wishes to wish-list an item, they will be redirected to the login page, with the item code and intended action in the url; which can then continue on to the desired page once logged on.
The MOZ crawler is seeing these pages as having Duplicated Content whilst they are all the same apart from a piece of information in the URL. Should we be blocking these duplications? Are they a risk to us? What should we be doing?
Many thanks,
Sarah
-
Hi Sarah,
Somehow I answered this and I must have forgotten to post the answer! Arg, it was a long one, too. Let me try to summarize what I'd do:
-If possible, noindex any page that doesn't display content while not logged in. Wait for those pages to drop out of the index, and monitor for errors.
- If not possible, skip straight to blocking pages behind a login wall with robots.txt. For example, to block anything in the login folder:
Disallow: /login
Or to block anything with a login variable:
Disallow: /*?login
This should prevent bots from crawling those URLs where you don't have any content to show them. Make sure to use this carefully.
I do apologize for the delay. If you have additional questions please feel free to PM me. I'd be happy to do a quick consult online or over the phone, as I feel bad that I never actually answered, and I can give you more specific ideas if we look at the site. If this answers your question that's fine too.
Good luck!
-
Hi Sarah,
I missed this notification on this one somehow!
To be honest, I don't have an answer for you on this one. Perhaps it might be worth either getting in touch with the Moz team or posting another question specifically tagged as "Product Support". They seem to be pretty good at answering those queries too
-
Thanks for this Chris.
One other thing, how then do I block this from showing up in my MOZ crawl, which is giving me 16,9k crawl issues and also how do i then work out what the other crawl issues are that are mixed up in this huge report?
-
Honestly I wouldn't be real worried about it. It seems Google is smart enough these days to understand what's going on there though canonicalization would be wise - just point the canonical tag on the login page to itself.
By doing this, assuming your URLs look something like domain.com/login?product=-product-name, all variations will theoretically be seen as the /login page.
If you really wanted to, you could use Robots to block these as well but I honestly wouldn't bother.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content nightmare
Hey Moz Community I ran a crawl test, and there is a lot of duplicate content but I cannot work out why. It seems that when I publish a post secondary urls are being created depending on some tags and categories. Or at least, that is what it looks like. I don't know why this is happening, nor do I know if I need to do anything about it. Help? Please.
Moz Pro | | MobileDay0 -
SEOmoz Not Picking Up Links
I'm finding that SEOmoz often doesn't pick up on many of the links my client's websites are attaining. The links do appear in search engine and often have a high domain authority (often PR links, which I why I follow up to see if they've been picked up). I do tend to see many of the links appear in webmaster tools, so it's a little frustrating that they don't appear, even after months of pages being live. Thoughts?
Moz Pro | | SebastianDyer0 -
OK I'll try again.... Linking root domains and external links
I'm new to the crawl test tool, I think I know what is meant by external links and linking root domains but I need to check. If there are 500 external links and 50 linking root domains does that suggest the essentially those 500 links are coming from 50 root domains? If yes, then what would 1000 linking root domains and 100 links mean? Or am I completely wrong in my assumptions and if so, can somebody point me in the right direction? Thank you.
Moz Pro | | JemRobinson0 -
Only few pages (308 pages of 1000 something pages) have been crawled and diagnosed in 4 days, how many days till the entire website is crawled complete?
Setup campaign about 4-5 days ago and yesterday rogerbot said 308 pages were crawled and the diagnostics were provided. This website has over 1000+ pages and would like to know how long it would take for roger to crawl the entire website and provide diagnostics. Thanks!
Moz Pro | | TejaswiNaidu0 -
Link not showing up in OSE
I created a profile in April this year on CrunchBase for my company http://www.crunchbase.com/company/wallpapered but it is not appearing in the "inbound links" of Open Site Explorer. All the other companies I have checked in OSE have their CrunchBase profile in their inbound links (many share the same Page authority as mine). Any suggestions would be really helpful. Thanks
Moz Pro | | roberthseo0 -
Internal Links
I've searched high and low, but I could just be missing it... Is there a report on internal links? Maybe a count? Or better yet, a way to see internal anchor text?
Moz Pro | | MCIMaui0 -
How do i get rid of a duplicate page error when you can not access that page?
How do i get rid of a duplicate page error when you can not access that page? I am using yahoo store manager. And i do not know code. The only way i can get to this page is by copying the link that the error message gives me. This is the duplicate that i can not find in order to delete. http://outdoortrailcams.com/busebo.html
Moz Pro | | tom14cat140 -
Redirecting duplicate .asp pages??
Hi all, I have a bit of a problem with duplicate content on our website. The CMS has been creating identical duplicate pages depending on which menu route a user takes to get to a product (i.e. via the side menu button or the top menu bar). Anyway, the web design company we use are sorting it out going forward, and creating 301 redirects on the duplicate pages. My question is, some of the duplicates take two different forms. E.g. for the home page: www.<my domain="">.co.uk
Moz Pro | | gdavies09031977
www..<my domain="">.co.uk/index.html
www.<my domain="">.co.uk/index.asp</my></my></my> Now I understand the 'index.html' page should be redirected, but does the 'index.asp' need to be directed also? What makes this more confusing is when I run the SEOMoz diagnostics report (which brought my attention to the duplicate content issue in the first place - thanks SEOMoz), not all the .asp pages are identified as duplicates. For example, the above 'index.asp' page is identified as a duplicate, but 'contact-us.asp' is not highlighted as a duplicate to 'contact-us.html'? I'm a bit new to all this (I'm not a IT specialist), so any clarification anyone can give would be appreciated. Thanks, Gareth0