Duplicate content http:// something .com and http:// something .com/
-
Hi,
I've just got a crawl report for a new wordpress blog with suffusion theme and yoast wordpress seo module and there is duplicate content for:
http:// something .com
and
http:// something .com/
I just can't figure out how to handle this. Can I add a redirect for .com/ to .com in htaccess?
Any help is appreciated!
By the way, the tag value for rel canonical is **http:// something .com/ **for both.
-
All so rember the canonicalization SEO advice: url canonicalization by MATT CUTTS on JANUARY 4, 2006 in GOOGLE/SEO (I got my power back!) Before I start collecting feedback on the Bigdaddy data center, I want to talk a little bit about canonicalization, www vs. non-www, redirects, duplicate urls, 302 “hijacking,” etc. so that we’re all on the same page. Q: What is a canonical url? Do you have to use such a weird word, anyway? A: Sorry that it’s a strange word; that’s what we call it around Google. Canonicalization is the process of picking the best url when there are several choices, and it usually refers to home pages. For example, most people would consider these the same urls: www.example.com example.com/ www.example.com/index.html example.com/home.asp But technically all of these urls are different. A web server could return completely different content for all the urls above. When Google “canonicalizes” a url, we try to pick the url that seems like the best representative from that set. Q: So how do I make sure that Google picks the url that I want? A: One thing that helps is to pick the url that you want and use that url consistently across your entire site. For example, don’t make half of your links go to http://example.com/ and the other half go to http://www.example.com/ . Instead, pick the url you prefer and always use that format for your internal links. Q: Is there anything else I can do? A: Yes. Suppose you want your default url to be http://www.example.com/ . You can make your webserver so that if someone requests http://example.com/, it does a 301 (permanent) redirect to http://www.example.com/ . That helps Google know which url you prefer to be canonical. Adding a 301 redirect can be an especially good idea if your site changes often (e.g. dynamic content, a blog, etc.). Q: If I want to get rid of domain.com but keep www.domain.com, should I use the url removal tool to remove domain.com? A: No, definitely don’t do this. If you remove one of the www vs. non-www hostnames, it can end up removing your whole domain for six months. Definitely don’t do this. If you did use the url removal tool to remove your entire domain when you actually only wanted to remove the www or non-www version of your domain, do a reinclusion request and mention that you removed your entire domain by accident using the url removal tool and that you’d like it reincluded. Q: I noticed that you don’t do a 301 redirect on your site from the non-www to the www version, Matt. Why not? Are you stupid in the head? A: Actually, it’s on purpose. I noticed that several months ago but decided not to change it on my end or ask anyone at Google to fix it. I may add a 301 eventually, but for now it’s a helpful test case. Q: So when you say www vs. non-www, you’re talking about a type of canonicalization. Are there other ways that urls get canonicalized? A: Yes, there can be a lot, but most people never notice (or need to notice) them. Search engines can do things like keeping or removing trailing slashes, trying to convert urls with upper case to lower case, or removing session IDs from bulletin board or other software (many bulletin board software packages will work fine if you omit the session ID). Q: Let’s talk about the inurl: operator. Why does everyone think that if inurl:mydomain.com shows results that aren’t from mydomain.com, it must be hijacked? A: Many months ago, if you saw someresult.com/search2.php?url=mydomain.com, that would sometimes have content from mydomain. That could happen when the someresult.com url was a 302 redirect to mydomain.com and we decided to show a result from someresult.com. Since then, we’ve changed our heuristics to make showing the source url for 302 redirects much more rare. We are moving to a framework for handling redirects in which we will almost always show the destination url. Yahoo handles 302 redirects by usually showing the destination url, and we are in the middle of transitioning to a similar set of heuristics. Note that Yahoo reserves the right to have exceptions on redirect handling, and Google does too. Based on our analysis, we will show the source url for a 302 redirect less than half a percent of the time (basically, when we have strong reason to think the source url is correct). Q: Okay, how about supplemental results. Do supplemental results cause a penalty in Google? A: Nope. Q: I have some pages in the supplemental results that are old now. What should I do? A: I wouldn’t spend much effort on them. If the pages have moved, I would make sure that there’s a 301 redirect to the new location of pages. If the pages are truly gone, I’d make sure that you serve a 404 on those pages. After that, I wouldn’t put any more effort in. When Google eventually recrawls those pages, it will pick up the changes, but because it can take longer for us to crawl supplemental results, you might not see that update for a while. That’s about all I can think of for now. I’ll try to talk about some examples of 302′s and inurl: soon, to help make some of this more concrete. http://www.ragepank.com/articles/3/preventing-duplicate-content/ Hope I was of help, Thomas Von Zickell
-
thanks!
Can some body please also clarify exactly what should be in the second line:
As eyepaq wrote: RewriteRule ^(.+)/$ [%{HTTP_HOST}...] [R=301,L]
Should I insert something in/after "[%{HTTP_HOST}...]"?
-
After RewriteEngine if i'm not wrong
-
Should I keep the existing wordpress rewrite? If I keep it, should I then place your code before or after?
BEGIN WordPress
RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
END WordPress
-
Hi,
Google is pretty good in understanding that the trailing slash version is the same with the non-trailing slash version so you are safe on that side.
Even if the crawler said this is an issue it's not something you should focus on.
However, if you want to play by the book, you can httaccess it so it will 301 redirect to oen or another.
Bellow is a sample code:
#get rid of trailing slashes
RewriteCond %{HTTP_HOST} ^(www.)?example.com$ [NC]
RewriteRule ^(.+)/$ [%{HTTP_HOST}...] [R=301,L]Hope it helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content and Other Issues from Blog Tags and Categories
I have recently taken over the maintenance/redesign of our website and after setting up Moz I see many errors:
On-Page Optimization | | jgoethert
Duplicate content
Missing descriptions
Duplicate titles
etc. All are related to blog categories and tags. My questions are: are these errors hurting us? Should I simply remove tags/categories from the sitemaps or bite the bullet and create content for every single category page? Our site is https://financiallysimple.com/ and we are using Yoast plugin in Wordpress (if that helps)2 -
How to organise subpages for good SEO content without duplicate text?
We are working on many subpages for our services. We have original content for each page however there are few text which we need to always duplicate like: Contact sales window, why to choose us window, supported files etc. What's the best way to do this so it's not consider as duplicated text. Should we redirected it or add it as a picture and always change name of the picture? Thank you Lukas
On-Page Optimization | | Lukas-ST0 -
How do I fix duplicate page issue on Shopify with duplicate products because of collections.
I'm working with a new client with a site built on Shopify. Most of their products appear in four collections. This is creating a duplicate content challenge for us. Can anyone suggest specific code to add to resolve this problem. I'm also interested in other ideas solutions, such as "don't use collections" if that's the best approach. I appreciate your insights. Thank you!
On-Page Optimization | | quiltedkoala0 -
Duplicate content from pagination and categories found in multiple locations
Hey Moz community, Really need help resolving duplicate content issues for an eCommerce utilizing Magento. We have duplicate content issues with category pagination and categories found in multiple locations. here's an example: "www.website.com/style/sequin-dresses" is also found at "www.website.com/features/sequin-dresses" *to resolve this issue do we just need to place a canonical tag on "www.website.com/features/sequin-dresses" pointing to "www.website.com/style/sequin-dresses"? In addition, the category "Sequin Dresses" also has pagination. to resolve duplicate content issues with pagination do we need to implement a rel=next/prev tag? (we do not have a view-all due to the amount of products featured) If anyone has experience with this or any insights on how to resolve these issues please let me know. Thanks!
On-Page Optimization | | LeapOfBelief0 -
Duplicate Mega tags
we have a e-commerce site, we have products that are the exact same but different sizes each has a page, we use the same mega tag would it be better to use no mega tag
On-Page Optimization | | DFC0 -
The "100 links/page recommendation" - Do Duplicate Links Count?
We have way too many links on our homepage. The PageRank Link Juice Calculator (www.ecreativeim.com/pagerank-link-juice-calculator.php) counts them to 300. But all of them are not unique, that is some links point to the same URL. So my question: does the "100 links/page recommendation" refer to all anchors on the page or only to unique link target URLs? I know "100" is just a standard recommendation.
On-Page Optimization | | TalkInThePark0 -
What is the best way to manage industry required duplicate Important Safety Information (ISI) content on every page of a site?
Hello SEOmozzer! I have recently joined a large pharmaceutical marketing company as our head SEO guru, and I've encountered a duplicate content related issue here that I'd like some help on. Because there is so much red tape in the pharmaceutical industry, there are A LOT of limitations on website content, medication and drug claims, etc. Because of this, it is required to have Important Safety Information (ISI) clearly stated on every page of the client's website (including the homepage). The information is generally pretty lengthy, and in some cases is longer than the non-ISI content on each page. Here is an example: http://www.xifaxan.com/ All content under the ISI header is required on each page. My questions are: How will this duplicated content on each page affect our on-page optimization scores in the eyes of search engines? Is Google seeing this simply as duplicated content on every page, or are they "smart" enough to understand that because it is a drug website, this is industry standard (and required)? Aside from creating more meaty, non-ISI content for the site, are there any other suggestions you have for handling this potentially harmful SEO situation? And in case you were going to suggest it, we cannot simply have an image of the content, as it may not be visible by all internet users. We've already looked into that 😉 Thanks in advance! Dylan
On-Page Optimization | | MedThinkCommunications0 -
User experience regarding dulpicate content and managing this content with google.
Hi long title i know! We are moving on to magento and have chosen to allocate a specific colour to each category using corresponding tabbed navigation for user experience.All products within each of the coloured tabs then inherit the repective colour, giving the products a category identiy within the store. This layout has had a positive feedback from our "testers" As a lot of our products are seasonal and can be represented in different categories there is a significant amount of duplicate content. ATM i see our options as being: Alter the site structure so that the category is not shown in the url, therefore eliminating our duplicate products. The downside of this is that the colour co-ordination of the categories would not work at product level as its the category path that assigns the colour. create canonical links for every duplicate, can this be damaging? keep the duplicates and do nothing let google decide the most important version of a product. any guidance would be appreciated!
On-Page Optimization | | LadyApollo0