Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Following urls should be add in disavow file or not
Hey Moz Friends, Should I include following spam link urls in disavow file or not? OR Will Google handle automatically? These type I have thousands urls. =>>>web-seek.org/the_worlds_most_visited_web_pages_42.html<<<=
White Hat / Black Hat SEO | | Rajesh.Prajapati
=>>>web-seek.net/the_worlds_most_visited_web_pages_42.html<<<=
=>>>websearching.net/the_worlds_most_visited_web_pages_42/
=>>>websearch.pl/the_worlds_most_visited_web_pages_42.html<<<=
=>>>web-search.net/the_worlds_most_visited_web_pages_42/
=>>>web-pages.org/the_worlds_most_visited_web_pages_42/
=>>>web-page.org/the_worlds_most_visited_web_pages_42.html<<<=
=>>>the-world.net/the_worlds_most_visited_web_pages_42.html<<<=
=>>>the-internet.tv/the_worlds_most_visited_web_pages_42.html<<<=
=>>>the-internet.in/the_worlds_most_visited_web_pages_42.html<<<=
=>>>the-globe.tv/the_worlds_most_visited_web_pages_42/
=>>>theglobe.sk/the_worlds_most_visited_web_pages_42.html<<<=
=>>>theglobe.ru/the_worlds_most_visited_web_pages_42.html<<<=
=>>>theglobe.pl/the_worlds_most_visited_web_pages_42.html<<<= Hope you will give any solution. Waiting for your positive response.0 -
Keyword + Location domains
Hi All, Just wanted to get everyones opinions on this, I see it more and more now where businesses own multiple domains for [keyword] + [location], they have multiple domains for different locations and setup individual sites on them. I see these types of domains rank very easily for medium competition keywords, as long as the on page is good and there are a handful of back links, they rank. just to clarify, for example - iphonerepairmanchester.co.uk (purely an example not sure how this site ranks!!) What are Googles views on this? I've always insisted its better to build a strong brand with the "real" business rather than creating extra websites named by keywords. But I've recently had a client want to pursue this and it seems it currently works, but is there a danger down the line Google will penalise it? The short term traffic increase is undeniable but like anything in the world of Google at the moment, I'd rather persuade clients not to go this route if it will protect future interests.
White Hat / Black Hat SEO | | gamnaking10 -
Will implementing 301's on an existing domain impact massively on rankings?
Hi Guys,I have a new SEO client who only has the non-www domain setup for GWT and I am wondering if implementing a 301 for www will have a massive negative impact on rankings. I know a percentage of link juice and PageRank will be affected. So my question is: If I implement the 301 should I brace myself for a fall in rankings. Should I use a 301 instead to maintain link juice and PageRank? Is it good practice to forward to www? Or could I leave the non www in place and have the www redirect to it to maintain the data? Dave
White Hat / Black Hat SEO | | icanseeu0 -
Is there any value in "starting from scratch" on a new domain?
Hi, Our ecommerce store - we have had some duplicate content issues and they have been corrected, but of course, Google takes time to pick up on these. Our link profile is very poor, so we wont lose a lot by going to a new domain in that sense. My question is, in what instances is it worthwhile starting under a new domain? And in which not? Presumably you can also 301 the whole site - when is it worth doing this or not? Thanks, Ben
White Hat / Black Hat SEO | | bjs20100 -
Webmaster Tools Showing Bad Links Removed Over 60 Days Ago
Hello, One of my clients received the notorious message from Google about unnatural links late last March. We've removed several hundred (if not thousands) of links, and resubmitted several times for reconsideration, only to continue with responses that state that we still have unnatural links. Looking through the "links to your site" in google webmaster tools, there are several hundred sites / pages listed, from which we removed our link over 60 days ago. If you click each link to view the site / page, they contain nothing, viewable or hidden, regarding our website / address. I was wondering if this (outdated / inaccurate) list is the same as the one their employees use to analyze the current status of bad links, and if so how long it will take to reflect up-to-date information. In other words, even though we've removed the bad links, how long do we need to wait until we can expect a clean resubmission for reconsideration. Any help / advice would be greatly appreciated -
White Hat / Black Hat SEO | | Bromtec0 -
Best use of domains with keywords
I own a domain with just the company name in it (no keywords) that I use as main domain. I also own some other domain with keywords inside that right now I redirect all to the main domain with a 301 redirect. What is the best use for these domains? Should I use them when I do link building or is better to use just the main domain? Can they be useful to increase the main domain link juice/page rank? If yes, how? Thanks
White Hat / Black Hat SEO | | darkanweb0 -
Can you block backlinks from another domain
Wondering if this is somehow possible. A site got hacked and created a /data folder with hundreds of .php files that are web pages selling all sorts of stuff. We deleted the /data folder and blocked Google from indexing it. Just noticed in Webmaster Tools that the site has 35,000 backlinks from other sites that got hacked with the same way. Is there a way to block these sites? I am assuming there isn't, but wanted to see if anyone ran into the same problem. It is a wordpress site is that helps.
White Hat / Black Hat SEO | | phatride0 -
Can't figure out how my competitor has so many links
I suspect something possibly black-hat is going on with the amount of inbound links for www.pacificlifestylehomes.com ( http://www.opensiteexplorer.org/links?site=www.pacificlifestylehomes.com ) mainly because they have such a large volume of links (for my industry) with their exact targeted keyword. Can anyone help clear this up for me?
White Hat / Black Hat SEO | | theChris0
