Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL masking
Hi all, Within my organization we maintain a bunch of websites, say child1.com, child2.com and child3.com. It was recently suggested that we merge all three into a single one, say mother.com. The marketing community within my organization strongly argued against this, mainly due to risks related to brand awareness, rank and traffic loss for each of these websites. But we resigned to the idea that we must do it, so we started considering the next proper steps. Now, my understanding is that setting up redirects is crucial in order to cushion the fall and mitigate the losses. However, some people are suggesting url masking. To me personally, url masking doesn't sound like a white hat practice, maybe it's borderline grey, but the bottom line is I need some advice on this topic. Could someone kindly address the following: 1. How is url masking different from url redirect? 2. Is url masking different from url cloaking? 3. Would google penalize us for implementing url masking? 4. Would that have any impact on our PPC campaigns? 5. Are there any documented cases of successful and google-sanctioned websites that are actively using url masking? 6. Are there any pitfalls to using this strategy? Thank you
White Hat / Black Hat SEO | | SimonaCretu0 -
I would like opinions on Brian Dean's training courses and his advice -- is it useful?
I would like opinions on Brian Dean's training courses and his advice -- has anyone used it successfully? Is it worth the cost? And useful?
White Hat / Black Hat SEO | | marketingdepartment.ch1 -
Exact match domain - should i use one
i have the domain "region"familyholidays.co.uk for an upcoming site. i was pleased as its memorable and tells the user what its about. i am targetting keywords such as: region family holidays region family hotels region famliy cottages region family campsites is it something i should avoid because of potential penalties. i will be adding plenty of good content and doing all the offsite things but dont want to start with a handicap with an emd? thanks neil
White Hat / Black Hat SEO | | neilhenderson0 -
Subdomain and root domain effects on SEO
I have a domain lets say it's mydomain.com, which has my web app already hosted on this domain. I wanted to create a sub-product from my company, the concept is a bit different than my original web app that is on mydomain.com and I am planning to host this on mynewapp.mydomain.com. I am having doubts that using a sub-domain will have an impact on my existing or new web app. Can anyone give me any pointers on this? As much as I wanted to use a directory mydomain.com/mynewapp, this is not possible because it will just confuse existing users of the new product/web app. I've heard that subdomains are essentially treated as a new site, is this true? If it is then I am fine with this, but is it also true that subdomains are harder to reach the top rank rather than a root domain?
White Hat / Black Hat SEO | | herlamba0 -
Site architecture change - +30,000 404's in GWT
So recently we decided to change the URL structure of our online e-commerce catalogue - to make it easier to maintain in the future. But since the change, we have (partially expected) +30K 404's in GWT - when we did the change, I was doing 301 redirects from our Apache server logs but it's just escalated. Should I be concerned of "plugging" these 404's, by either removing them via URL removal tool or carry on doing 301 redirections? It's quite labour intensive - no incoming links to most of these URL's, so is there any point? Thanks, Ben
White Hat / Black Hat SEO | | bjs20100 -
Is bad English detected by Google
Hi, I am based in the UK and in a very competitive market - van leasing - and I am thinking about using an Indian SEO company for my ongoing SEO. They have sent me some sample artilces that they have written for link building and the English is not good. Do you think that google can tell the difference between a well written article and a poorly written article? Will the fact that articles are poorly writtem mean we will lose potential value from the link? Any input would be much appreciated. Regards John J
White Hat / Black Hat SEO | | Johnnyh0 -
Redirect n domain to one
What happen when I redirect301 10 domain to one? I have 10 domain with ave Page Authority=45 and Domain Authority 60 and want to increase my new domain by redirect them. is it right or wrong?
White Hat / Black Hat SEO | | vahidafshari450 -
Are expired domains for Godaddy backlinks already reset by googel ?
When does Google actual reset backlinks to a domain. If i am buying expired domains from godaddy action for linking purpose am i wasting my time.Also if that the case whats the point of buying expired domains with many links pointing to them. If the backlinks to the expired domain still show up under my Google webmaster tool window does that mean Google counts that link?
White Hat / Black Hat SEO | | Ryguy870