Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

How does the crawl find duplicate pages that don't exist on the site?

Intermediate & Advanced SEO

580

Deniz last edited by

It looks like I have a lot of duplicate pages which are essentially the same url with some extra ? parameters added eg:

http://www.merlin.org.uk/10-facts-about-malnutrition

http://www.merlin.org.uk/10-facts-about-malnutrition?page=1

http://www.merlin.org.uk/10-facts-about-malnutrition?page=2

These extra 2 pages (and there's loads of pages this happens to) are a mystery to me. Not sure why they exist as there's only 1 page.

Is this a massive issue? It's built on Drupal so I wonder if it auto generates these pages for some reason?

Any help MUCH appreciated. Thanks
1 Reply Last reply
Reply Quote 0
Deniz @BenRWoodard last edited by

Thanks Ben - much appreciated!
1 Reply Last reply
Reply Quote 0
BenRWoodard last edited by

This is being caused by your "Related Post" plugin/module. To correct this problem simply add rel="nofollow" to the links in that module.
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Why doesn't my website crawl by Google?

Hi mozzers and members, I am having issues, why my website: http://profilecosmeticsurgery.com/ crawl by Google? let me share more clearly when this starts happening. A month or around 45 days back our website is being indexed and crawled quite well without any issues with having .html extension pages with static built website.
We finally thought to change to .php version and make whole website and its pages to be treated dynamically.
Once we changed all changes, thereafter this issues started. It has been more than 45 days, our website isn't being crawled since then. I didn't know what are the things preventing this to? Please help. Thanks in Advance Capture1.PNG
Intermediate & Advanced SEO | | SEOOOOOoooooooo

0
Google crawling 200 page site thousands of times/day. Why?

Hello all, I'm looking at something a bit wonky for one of the websites I manage. It's similar enough to other websites I manage (built on a template) that I'm surprised to see this issue occurring. The xml sitemap submitted shows Google there are 229 pages on the site. Starting in the beginning of December Google really ramped up their intensity in crawling the site. At its high point Google crawled 13,359 pages in a single day. I mentioned I manage other similar sites - this is a very unusual spike. There are no resources like infinite scroll that auto generates content and would cause Google some grief. So follow up questions to my "why?" is "how is this affecting my SEO efforts?" and "what do I do about it?". I've never encountered this before, but I think limiting my crawl budget would be treating the symptom instead of finding the cure. Any advice is appreciated. Thanks! *edited for grammar.
Intermediate & Advanced SEO | | brettmandoes

0
Duplicate Page Content

We have different plans that you can signup for - how can we rectify the duplicate page content and title issue here? Thanks. | http://signup.directiq.com/?plan=100 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=104 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=116 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=117 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=102 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=119 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=101 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=103 | 0 | 1 | 32 | 1 | 200 |
| http://signup.directiq.com/?plan=5 |
Intermediate & Advanced SEO | | directiq

0
How To Detect Primary Site With Duplicate Domains?

I'm working with some backlink data, and I've run into different domains that host the same exact content on the same IP. They're not redirecting to each other, just looks like they're hosting the same content on differnet virtual hostnames. One example is: borealcanada.ca borealcanada.com borealcanada.org www.borealcanada.ca www.borealcanada.com www.borealcanada.org www.borealecanada.ca I'm trying to consolidate this data and choose which is the primary domain. In this example, it appears www.borealcanada.ca has a high number of indexed pages and also ranks first for "boreal canada". However, I'm trying to think of a metric I can use to definitively/systematically handle this (using SEO Tools or something like it). Anyone have ideas on which metric might help me determine this for a large number of sites?
Intermediate & Advanced SEO | | brettgus

0
How can I see all the pages google has indexed for my site?

Hi mozers, In WMT google says total indexed pages = 5080. If I do a site:domain.com commard it says 6080 results. But I've only got 2000 pages in my site that should be indexed. So I would like to see all the pages they have indexed so I can consider noindexing them or 404ing them. Many thanks, Julian.
Intermediate & Advanced SEO | | julianhearn

0
301 doesn't redirect a page that ends in %20, and others being appended with ?q=

I have a product page that ends /product-name**%20** that I'm trying to redirect in this way: Redirect 301 /products/product-name%20 http://www.site.com/products/product-name And it doesn't redirect at all. The others, those with %20, are being redirected to a url hybrid of old and new: http://www.site.com/products/product-name**?q=old-url** I'm using Drupal CMS, and it may be creating rules that counter my entries.
Intermediate & Advanced SEO | | Brocberry

0
Duplicating an existing website - new name and reskin

Would re-skinning, duplicating an exising ecommerce website with a new domain name cause any ranking issues? The plan would be that all product data, pricing info etc would be identical, the site would have a minor redesign to change colours, logos etc and all duplicate content would be rel=canonicaled to the original site. In case you are wondering the reason for this is a customer with an existing site wants to try out a new brand without incorporating a massive development costs. The majority of traffic would be driving through google shopping, a bit of PPC, social etc. Is this site duplication likely to harm the original site or will setting up rel=canonical to point to the original site going to be sufficient enough to prevent this happening? Is there anything else is should consider? Many thanks for your help
Intermediate & Advanced SEO | | JustinTaylor88

0
Redirecting Existing Domains to My Main Site

Hi I have a main property related website featuring different countries around the world. I also have many different seperate country websites 20+. All keyword rich domains with a good 9 years+ domain age and PR3's with decent links and moz rankings and unique content. Many of the sites are very low Alexa rank now and receive little traffic. I don't have the time now to spend on each of the individual domains and am wanting to consolidate them and their PR juice to the corresponding country page of my main website. My question is - is it possible - will google see this as me trying to manipulate them and is my main site likely to suffer from any penalties or downgrading? Thanks for your input.
Intermediate & Advanced SEO | | freecall

0