Penalization for Duplicate URLs with %29 or "/"

[sfecommerce]

Hi there -

Some of our dynamically generated product URLs somehow are showing up in SEOmoz as two different URLs even though they are the same page- one with a %28 and one with a

e.g.,

http://www.company.com/ProductX-(-etc/

http://www.company.com/ProductX-(-etc/

Also, some of the URLs are duplicated with a "/" at the end of them.

Does Google penalize us for these duplicate URLs? Should we add canonical tags to all of them?

Finally, our development team is claiming that they are not generating these pages, and that they are being generated from facebook/pinterest/etc. which doesn't make a whole lot of sense to me. Is that right?

Thanks!

[JaredMumford]

Canonical tags should drastically help with this. The % is being generated because the URL is being encoded and has a "(" in it. Have your product page each contain their own canonical with the URL you want indexed.  Not sure which URL to use? Check your internal links and see how your site is linking to your product pages. Presumably its:

http://www.company.com/ProductX-(-etc/

or

http://www.company.com/ProductX-(-etc

Add this URL as your canonical and the SE's will understand what page is the 'real' page. This will solve both problems from an SEO standpoint. If you want to actually stop the site from doing this, you can remove trailing slashes and encoding using HTACCESS.

[jenmcardle]

In short, your site software should completely control all links generated on your site. If you hand code a site using .NET, Cold Fusion, HTML/CSS/PHP, etc. you are in complete control over your links. If you use a CMS or other software such as WordPress, Magento, etc. then the software creates the URLs for you. In either case a skilled developer should be able to offer you options.

In brief, I recommend using a standard format for your URLs. We like to have all categories end with a trailing slash, and all web pages end without one. For example: www.mysite.com/cars/ or www.mysite.com/cars/2010-ford-mustang.

Whatever choice you make, enforce it throughout your site. You can also use the canonical tag to help control issues where a page may be offered under multiple URLs, but the best choice would be solving the root issue.

**Our development team is claiming that they are not generating these pages, and that they are being generated from facebook/pinterest/etc. which doesn't make a whole lot of sense to me. **

Without looking at your website and an example URL of this issue, it is not possible to offer a definitive answer. I have never encountered this issue.