How to stop crawls for product review pages? Volusion site

[Jerrion]

Hi guys, I have a new Volusion website. the template we are using has its own product review page for EVERY product i sell (1500+) When a customer purchases a product a week later they receive a link back to review the product. This link sends them to my site, but its own individual page strictly for reviewing the product. (As oppose to a page like amazon, where you review the product on the same page as the actual listing.)

**This is creating countless "duplicate content" and missing "title" errors. What is the most effective way to block a bot from crawling all these pages? Via robots txt.? a meta tag? **

Here's the catch, i do not have access to every individual review page, so i think it will need to be blocked by a robot txt file? What code will i need to implement? i need to do this on my admin side for the site? Do i also have to do something on the Google analytics side to tell google about the crawl block?

Note: the individual URLs for these pages end with: *****.com/ReviewNew.asp?ProductCode=458VB

Can i create a block for all url's that end with /ReviewNew.asp etc. etc.?

Thanks! Pardon my ignorance. Learning slowly, loving MOZ community

1354bdae458d2cfe44e0a705c4ec38dd

[MonicaOConnor]

No you should be fine

[Jerrion]

thanks. you say "update on 4/21" you're talking about googles update requiring more mobile friendly sites? My volusion template has its own mobile version. It is not a responsive template. So i should not be affected correct?

[MonicaOConnor]

Parameters are good for pages that are a result of a search or sort. I guess it isn't necessary really, I am just a little ocd about that kind of stuff. The parameters in WMT basically tell google that these things might appear in the URL, and then you can the bot to ignore it or let GoogleBot decide how to read the URL.

The mobile site is not the same as a responsive design and on of the main reasons I left Volusion. The mobile site will get you through the update on 4/21, but if possible you should ask them for a responsive site. Just call the support number, or your account manager and ask.

[Jerrion]

ive had the following in my robots.txt file. Do i need to add the astrisk like you have posted above?

Currently in my robot.txt:

User-agent:*
Disallow: /reviews.asp/

User-agent:*
Disallow: /reviewnew.asp/

[Jerrion]

Thanks monica, can you elaborate a bit more on the webmaster tools parameter? what specifically does adding a parameter like that do? You did that as a backup in case the robot txt file was not working? we do have a mobile version enabled which came with our template. Ill keep an eye out for the 404's. where do i check for a responsive template? Ours is one of their premium templates so its possible we are already on a responsive one? Can you clarify what responsive template means?

thanks.

[MonicaOConnor]

I did that in my Volusion store. I also added ReviewNew.asp?ProductCode= as a parameter in Google Webmaster Tools. Do you have an enabled mobile site as well? If you do there are several 404 errors that you will start to see from there. Make sure you are adding parameters accordingly. I am not sure if Volusion has started offering their responsive templates yet, but if they have I would see if you can implement that over the mobile site.

[Alick300]

Hi ,

Yes you can block such URL by using below code in robots.txt file.

User-agent: *
Disallow: /*ReviewNew.asp

Thanks