Penalization for Duplicate URLs with %29 or "/"
-
Hi there -
Some of our dynamically generated product URLs somehow are showing up in SEOmoz as two different URLs even though they are the same page- one with a %28 and one with a
e.g.,
http://www.company.com/ProductX-(-etc/
http://www.company.com/ProductX-(-etc/
Also, some of the URLs are duplicated with a "/" at the end of them.
Does Google penalize us for these duplicate URLs? Should we add canonical tags to all of them?
Finally, our development team is claiming that they are not generating these pages, and that they are being generated from facebook/pinterest/etc. which doesn't make a whole lot of sense to me. Is that right?
Thanks!
-
Canonical tags should drastically help with this. The % is being generated because the URL is being encoded and has a "(" in it. Have your product page each contain their own canonical with the URL you want indexed. Not sure which URL to use? Check your internal links and see how your site is linking to your product pages. Presumably its:
http://www.company.com/ProductX-(-etc/
or
http://www.company.com/ProductX-(-etc
Add this URL as your canonical and the SE's will understand what page is the 'real' page. This will solve both problems from an SEO standpoint. If you want to actually stop the site from doing this, you can remove trailing slashes and encoding using HTACCESS.
-
In short, your site software should completely control all links generated on your site. If you hand code a site using .NET, Cold Fusion, HTML/CSS/PHP, etc. you are in complete control over your links. If you use a CMS or other software such as WordPress, Magento, etc. then the software creates the URLs for you. In either case a skilled developer should be able to offer you options.
In brief, I recommend using a standard format for your URLs. We like to have all categories end with a trailing slash, and all web pages end without one. For example: www.mysite.com/cars/ or www.mysite.com/cars/2010-ford-mustang.
Whatever choice you make, enforce it throughout your site. You can also use the canonical tag to help control issues where a page may be offered under multiple URLs, but the best choice would be solving the root issue.
**Our development team is claiming that they are not generating these pages, and that they are being generated from facebook/pinterest/etc. which doesn't make a whole lot of sense to me. **
Without looking at your website and an example URL of this issue, it is not possible to offer a definitive answer. I have never encountered this issue.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Creating a help hub, not sure the best name to use, " keyword help " or " help hub "?
I've been creating new content for our site, lots of help related content, so I created a help hub section. Now the more I go through it, and look at url structure and breadcrumbs, I can't help but think I should be using a keyword in there, but also don't want to over do it, since the keyword we are shooting for is also a subsection of our site, complete with url keyword and breadcrumb. So I just don't want to have too many over redundant titles like keyword this and keyword that, so I came here to get some advice from the awesome community of folks. Keep help hub so it's: Url: site.com/help-hub/helppage1 Breadcrumb: Home > Help-Hub > Help Page 1 or Url: site.com/keyword/help/helppage1 Breadcrumb: Home > Keyword > Help > Help Page 1
Technical SEO | | Deacyde0 -
"non-WWW" vs "WWW" in Google SERPS and Lost Back Link Connection
A Screaming Frog report indicates that Google is indexing a client's site for both: www and non-www URLs. To me this means that Google is seeing both URLs as different even though the page content is identical. The client has not set up a preferred URL in GWMTs. Google says to do a 301 redirect from the non-preferred domain to the preferred version but I believe there is a way to do this in HTTP Access and an easier solution than canonical.
Technical SEO | | RosemaryB
https://support.google.com/webmasters/answer/44231?hl=en GWMTs also shows that over the past few months this client has lost more than half of their backlinks. (But there are no penalties and the client swears they haven't done anything to be blacklisted in this regard. I'm curious as to whether Google figured out that the entire site was in their index under both "www" and "non-www" and therefore discounted half of the links. Has anyone seen evidence of Google discounting links (both external and internal) due to duplicate content? Thanks for your feedback. Rosemary0 -
Http:// to https:// 301 or 302 redirect
I've read over the Q & A in the Community, but am wondering the reasoning behind this issue. I know - 301's are permanent and pass links, and 302s are temporary (due to cache) and don't pass links. But, I've run across two sites now that 302 redirect http:// to https://. Is there a valid reason behind this? From my POV and research, the redirect should 301 if it's permanent, but is there a larger issue I am missing?
Technical SEO | | FOTF_DigitalMarketing1 -
Is "commented out" text still read by the SEs?
A site I reviewed was showing up in Google rankings for key phrases specific to a city, however the page that was showing up had the 'city' key phrases commented out. Does Google still read and utilized commented out text? Or is it more likely that the page in question got indexed before the key phrases were commented out and it's just still appearing for the related search queries?
Technical SEO | | MLTGroup1 -
20 000 duplicates in Moz crawl due to Joomla URL parameters. How to fix?
We have a problem of massive duplicate content in Joomla. Here is an example of the "base" URL: http://www.binary-options.biz/index.php/Web-Pages/binary-options-platforms.html For some reason Joomla creates many versions of this URL, for example: http://www.binary-options.biz/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html or http://www.binary-options.biz/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html So it lists the URL parameter ?q= and then repeats part of the beforegoing URL. This leads to tens of thousands duplicate pages in our content heavy site. Any ideas how to fix this? Thanks so much!
Technical SEO | | Xmanic0 -
Build links to / or without ./ on root
Hi there for linkbuilding what is better, building with / or without / www.domain.com www.domain.com/ I have been told it depends what your webserver is doing, also i was told they are the same? what does Google see? I know i need to stay consistent, just wondering which practice is better? Any help as always much appreciated.
Technical SEO | | pauledwards0 -
Is it possible to be penalized as duplicate content for one keyword but not another?
I help develop an online shopping cart and after a request from management about some products not showing up in the SERP's I was able to pinpoint it down to mostly a duplicate content issue. It's a no brainer as some times new products are inserted in with copied text from the manufacturers website. I recently though stumbled across a odd problem. When we partially re-wrote the content to seem unique enough it seemed to remedy the issue for some keywords and not others. A) If you search the company name our category listing shows as #1 ahead of the manufacturers website. We always did rank for this term. B) If you search the product name our product page is listed #3 behind two other listings which belong to the manufacturer. C) If you search the keywords together as "company product" we are still being filtered out as duplicate content. When I allow the filtered results to show we are ranking #4 It's been a full month since the changes were indexed. Before I rewrite the content even further I thought I would ask to see if any one has any insight as to what could be happening.
Technical SEO | | moondog6040 -
URL Duplicate Content Issues (Website Transition)
Hey guys, I just transitioned my website and I have a question. I have built up all the link juice around my old url styles. To give you some clarity: My old CMS rendered links like this: www.example.com/sweatbands My new CMS renders links like this: www.example.com/sweatbands/ My new CMS's auto-sitemap also generates them with the slash on the end. Also throughout the website the CMS links to them with the slash at the end and i link to them without the slash (because it's what i am used to). I have the canonical without the slash. Should I just 301 to the version with the slash before google crawls again? I'm worried that i'll lose all the trust and ranking i built up to the one without the slash. I rank very high for certain keywords and some pages house a large portion of our traffic. What a mess! Help! 🙂
Technical SEO | | Hyrule0