Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Massive Spam attack against my domain - automate disvow of tld?
We've been getting hundreds of new links from unique domains every day - all the domains follow a pattern like this: www.someword-1f4163e1.space/wiki/Someterm Hundreds... every day. What techniques exist to deal with a prolonged negative seo attack of this type. By the time we can detect and disvow, the damage is done.
White Hat / Black Hat SEO | | sonar0 -
Meta tags for international domains
Hi Mozers I have 3 top level domains co.nz com.au and com each meta tag for the home page is unique for each country and I have tried to figure this out for the last 3 months, but unfortunately I can't seem to pin point why all 3 meta tags are showing up exactly the same. It seems all meta tags are showing up for the co.nz domain. In the attachments you can see all urls are showing up correctly for each country specific domain, however the meta tag description defaults to the NZ Any help around this would be much appreciated! Thanks all
White Hat / Black Hat SEO | | edward-may0 -
IT want to do a name server redirect
Hi, I am in a little bit of a pickle, and hope that you clever people can help me... A little background: In April this year we relaunched one of our brands as a standalone business. I set up page to page 301 redirects from the old website to the new branded domain. From an SEO perspective this relaunch went amazingly smoothly - we only lost around 10% of traffic and that was just for a couple of months. We now get more traffic than ever before. Basically it's all going swimmingly. I noticed yesterday that the SSL certificate on the old domain has expired, so I asked IT to repurchase one for us to maintain the 301 redirects. IT are saying that they would prefer to do a name server redirect instead, which would remove all the page to page 301s. They are saying that this would maintain the SEO. As far as I am aware this wouldn't. Please can someone help me put together a polite but firm response to basically say no? Thanks, I really welcome and appreciate your help on this! Amelia
White Hat / Black Hat SEO | | CommT0 -
Should You Link Back from Client's Website?
We had a discussion in the office today, about if it can help or hurt you to link back to your site from one that you optimize, host, or manage. A few ideas that were mentioned: HURT:
White Hat / Black Hat SEO | | David-Kley
1. The website is not directly related to your niche, therefore Google will treat it as a link exchange or spammy link.
2. Links back to you are often not surrounded by related text about your services, and looks out of place to users and Search Engines. HELP:
1. On good (higher PR, reputable domain) domains, a link back can add authority, even if the site is not directly related to your services.
2. Allows high ranking sites to show users who the provider is, potentially creating a new client, and a followed incoming link on anchor text you can choose. So, what do you think? Test results would be appreciated, as we are trying to get real data. Benefits and cons if you have an opinion.2 -
Ask Bloggers/Users To Link To Website
I have a web service that help bloggers to do certain tasks and find different partners. We have a couple of thousand bloggers using the service and ofcourse this is a great resource for us to build links from. The bloggers are all from different platforms and domains. Currently when a blogger login to the service we tell the blogger that if they write a blog post about us with their own words, and tell their readers what they think of our service. We will then give them a certain benifit within the service. This is clearly encouraging a dofollow-link from the bloggers, and therefore it's not natural link building. The strategy is however working quite good with about 150 new blog posts about our service per month, which both gives us a lot of new visitors and users, but also give us link power to increase our rankings within the SERP. Now to my questions: This is not a natural way of building links, but what is your opinion of this? Is this total black hat and should we be scared of a severe punishment from Google? We are not leaving any footprints more than we are asking the users for a link, and all blogposts are created with their own unique words and honest opinions. Since this viral marketing method is working great, we have no plans of changing our strategy. But what should we avoid and what steps should we take to ensure that we won't get in any trouble in the future for encouraging our users to linking back to us in this manner?
White Hat / Black Hat SEO | | marcuslind0 -
Does the Traffic boost SEO/SERP ranks?
Hello, I know a guy that sells Organic traffic, bought 10k from him, will this help me to bost google seo ranks? Attached a screenshoot thank you!
White Hat / Black Hat SEO | | 7liberty0 -
Here's some more proof white hat SEO works
I guess this is the most logical place to share this with you. I do SEO for many sites. I've recently been focusing on two in particular for the same client. We used Netfirms SEO services to get links--he insisted--which basically consists of writing articles in broken English and placing them all over blog networks with our desired anchor text. On the other site, I simply refused to employ those services. This was the client's main site, and was way too important to mess around with. I built links myself, the legit way. Long story short, for months I watched the shady, black hat site climb and climb in the SERPs, while the white hat one kept falling. This morning, I checked my SEOmoz campaigns and my white hat site went from #8 to #2 and my black hat site went from page 2 to no longer being in the top 50. Just another example of what's been happening with Google lately and how great it is. Interestingly, the black hat site never got a warning in GWT about buying links. Now I just have to figure out a way to break the news to my boss and tell him I told him so without actually using those words.
White Hat / Black Hat SEO | | UnderRugSwept5 -
What happened with Hayneedle's rankings?
Hayneedle is an e-commerce company that operates 200 niche sites selling indoor and outdoor home products. They were ranking at the top of the first page for most terms related to their sites (fire pits, fountains, benches, etc.), but all of a sudden at the end of April they lost their rankings, getting dropped to page 4 or lower for tons of their sites (barstools.com, patiofurnitureusa.com, adirondackchairs.com, benches.com, etc.). Does anybody know what caused this? Other than one thread on an SEO forum, we haven't been able to find any discussion about it online. It seems like cross-linking between the sites could have been a problem here, but we'd love to hear thoughts from the experts here on this. Our company is using the same business model of one brand with niche sites and we want to avoid anything like this happening to us.
White Hat / Black Hat SEO | | outdoorliving0