Welcome to the Q&A Forum

If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?

Hi there,

We have a client with a large eCommerce site with about 1500 duplicate URLs caused by the parameters in the URLs (such as the sort parameter where the list of products are then sorted by price, age etc.)

Example:

www.example.com/cars/toyota

First duplicate URL: www.example.com/cars/toyota?sort=price-ascending

Second duplicate URL: www.example.com/cars/toyota?sort=price-descending

Third duplicate URL: www.example.com/cars/toyota?sort=age-descending

Originally we had advised to add a robots.txt file to block search engines from crawling the URLs with parameters but this hasn't been done.

My question: If we add the robots.txt now and exclude all URLs with filters - how long will it take for Google to disregard the duplicate URLs?

We could ask the developers to add canonical tags to all the duplicates but these are about 1500...

Thanks in advance for any advice!

Unfortunately we can't control the content on the aggregator website (e.g. with rel="canonical" etc.)

Hi there,

No we can't control what is being put on the aggregator website (chrono24.com, a large website displaying watches from different dealers).

We won't be changing domain names, copying over all product content, just restyling and adding new content in the about us/services pages.

So I assume the only option is to have Google index our content first. Thanks for the video!

Hi there,

Our client has an ecommerce website, their products are also showing on an aggregator website (aka on a comparison website where multiple vendors are showing their products). On the aggregator website the same photos, titles and product descriptions are showing.

Now with building their new website, how can we avoid such duplicate content? Or does Google even care in this case? I have read that we could show more product information on their ecommerce website and less details on the aggregator's website. But is there another or better solution?

Many thanks in advance for any input!

In this Moz guide regarding Google webmaster recommendations, it says you should still set the paginated page parameter in Google's Webmaster Tools:

https://moz.com/ugc/seo-guide-to-google-webmaster-recommendations-for-pagination (search for the part "Coding Instruction for the View-All Option")

Hope this helps!

Have you checked the site's backlinks in Moz Open Site Explorer? To me it seems there are some pretty spammy backlinks, could this be the reason?

You could try disavow them here https://www.google.com/webmasters/tools/disavow-links-main

Hi Andy,

We're not sure yet about the amount of scrolling pages, however the resources you've given here have helped us a lot!

Thanks very much!

Hi all,

We are implementing an eCommerce site where the results pages of the products will be visibile on one page (always loading new products when you scroll down the page).

Now, I have read that the Google spiders cannot "load" new products scrolling down the page, hence the spider only sees the first few products of the results page.

Our developer wants to implement a system where a users sees the first products on

example.com/products

Then scrolling down, he will see new products with the URL changing to example.com/page/2 and so on.

Is it enough that we add a pagination link that goes from example.com/products to example.com/page/2

Then another link that goes from example.com/page/2 to example.com/page/3 and so on, so the Google spider can make his way through all the pages? Or is that too much deep linking and the spider wouldn't even crawl all the results pages?

Any recommendations how to go about this?

Many thanks in advance!

Exactly that was the case, I've checked and they have a 302 redirect.

Thanks a lot Tom!

Hi there,

I have a quick question, when looking for our client's domain name in Google (Google.it, Google.co.uk and Google.com), we search for example.com, but the first (and only) search result showing is example.co.uk (which redirects then to example.com)

Why is the co.uk domain showing instead of the com domain where we are redirecting to? I don't assume that this is any form of penalisation?

Thanks!

Hi there,

we have a website with a more than 60% bounce rate. I'm trying to find out where these visitors (the ones that are bouncing off) are coming from, to what percentage are they coming from the PPC campaign, from organic search results or referrals.

I suspect that the keywords from the PPC campaign are not relevant to the website content (and hence visitors are bouncing off).

Does anyone know how to check this in Google Analytics?

Thanks in advance!

Thanks a lot Anthony. Unfortunately the problem cannot be fixed at programming level so I'll try the "solution" with the canonical tags.

Cheers!

We have a website with 4 product categories (1. ice cream parlors, 2. frozen yogurt shops etc.).

A few sub-categories (e.g. toppings, smoothies etc.) and the products contained in those are available in more than one product category (e.g. the smoothies are available in the "ice cream parlors" category, but also in the "frozen yogurt shops" category).

My question:

Unfortunately the website has been designed in a way that if a subcategory (e.g. smoothies) is available in more than 1 category, then itself (the subcategory page) + all its product pages will be automatically visible under various different urls.

So now I have several urls for one and the same product:

www.example.com/strawberry-smoothie|SMOOTHIES|FROZEN-YOGURT-SHOPS-391-2-5

and

http://www.example.com/strawberry-smoothie|SMOOTHIES|ICE-CREAM-PARLORS-391-1-5

And also several ones for one and the same sub-category (they all include exactly the same set of products):

http://www.example.com/SMOOTHIES-1-12-0-4 (the smoothies contained in the ice cream parlors category)

http://www.example.com/SMOOTHIES-2-12-0-4 (the same smoothies, contained in the frozen yogurt shops category)

This is happening with around 100 pages.

I would add canonical tags to the duplicates, but I'm afraid that by doing so, the category (frozen yogurt shops) that contains several non-canonical sub-categories (smoothies, toppings etc.) , might not show up anymore in search results or become irrelevant for Google when searching for example for "products for frozen yoghurt shops". Do you know if this would be actually the case?

I hope I explained it well..