Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content warning: Same page but different urls???
Hi guys i have a friend of mine who has a site i noticed once tested with moz that there are 80 duplicate content warnings, for instance Page 1 is http://yourdigitalfile.com/signing-documents.html the warning page is http://www.yourdigitalfile.com/signing-documents.html another example Page 1 http://www.yourdigitalfile.com/ same second page http://yourdigitalfile.com i noticed that the whole website is like the nealry every page has another version in a different url?, any ideas why they dev would do this, also the pages that have received the warnings are not redirected to the newer pages you can go to either one??? thanks very much
White Hat / Black Hat SEO | | ydf0 -
How Do You Know or Find Out if You've been hit by a Google Penalty?
Hi Moz Community, How do you find out if you have been hit with a Google Penalty? Thanks, Gary
White Hat / Black Hat SEO | | gdavey0 -
On the use of Disavow tool / Have I done it correctly, or what's wrong with my perception?
On a site I used GSA search engine ranker. Now, I got good links out of it. But, also got 4900 links from one domain. And, I thought according to ahrefs. One link from the one domain is equal to 4900 links from one domain. So, I downloaded links those 4900 and added 4899 links to disavow tool. To disavow, to keep my site stable at rankings and safe from any future penalty. Is that a correct way to try disavow tool? The site rankings are as it is.
White Hat / Black Hat SEO | | AMTrends0 -
Are multiple domains spammy if they're similar but different
A client currently has a domain of johnsmith.com (not actual site name, of course). I’m considering splitting this site into multiple domains, which will include brand name plus keyword, such as: Johnsmithlandclearing.com Johnsmithdirtwork.com Johnsmithdemolition.com Johnsmithtimercompany.com Johnsmithhydroseeding.com johnsmithtreeservice.com Each business is unique enough and will cross-link to the other. My questions are: 1) will Google consider cross-linking spammy? 2) what happens to johnsmith.com? Should it redirect to new site with the largest market share, or should it become an umbrella for all? 3) Any pitfalls foreseen? I've done a fair amount of due diligence and feel these separate domains are legit, but am paranoid that Google will not see it that way, or may change direction in the future.
White Hat / Black Hat SEO | | SteveMauldin0 -
Redirecting old domains for SEO ranking?
It's been a while since I read anything seriously out of the box on SEO but I thought I would see what others thought of the bold assertions made in this article. Most of it revolves around buying expired domains and using a 301 to point them, and their juice, to new sites. This guy makes a living doing this so he has to know a bit more than the average Joe but I'm wondering where the other shoe is and when it drops.
White Hat / Black Hat SEO | | Highland0 -
Owning multiple domains across similar fields
We have a printing website specializing in a single field but had added other products to it along the way, we are looking at adding a second site (redesigned from the ground up) that with cover everything (so will inevitably cover some of the same keywords) is this ok. We are considering some of the other things such as putting both sites on different IP's and also wondered about the registration details (would it be ok to have them under the same company?). Just to make it clear they will not be used for cross link or any other such things so just wanting to know if they could end up on page one for some of the same keywords and Google would be ok with this. Thank you for your time.
White Hat / Black Hat SEO | | BobAnderson0 -
Geo-targeted Organic Search Traffic to a sub-domain
For a client of ours, we are likely to create a sub-domain that is to be targeted at a specific country. Most of the content on this sub-domain will be from the main site, although with some specific differentiation to suit that geographic market. We intend to tell Google through Webmaster Centre that the sub-domain is targeted at a specific country. Some questions: a) Any idea how long it could take before google gives precedence to the content in this sub-domain for queries originating from that particular country? b) What is the likely impact of content duplication ? What extent of differentiation is necessary from a search engine perspective? Thanks.
White Hat / Black Hat SEO | | ontarget-media1
