Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Technical : Duplicate content and domain name change
Hi guys, So, this is a tricky one. My server team just made quite a big mistake :We are a big We are a big magento ecommerce website, selling well, with about 6000 products. And we are about to change our domaine name for administrative reasons. Let's call the current site : current.com and the future one : future.com Right, here is the issue Connecting to the search console, I saw future.com sending 11.000 links to current.com. At the same time DA was hit by 7 points. I realized future.com was uncorrectly redirected and showed a duplicated site or current.com. We corrected this, and future.com now shows a landing page until we make the domain name change. I was wondering what is the best way to avoid the penalty now and what can be the consequences when changing domain name. Should I set an alias on search console or something ? Thanks
White Hat / Black Hat SEO | | Kepass0 -
On the use of Disavow tool / Have I done it correctly, or what's wrong with my perception?
On a site I used GSA search engine ranker. Now, I got good links out of it. But, also got 4900 links from one domain. And, I thought according to ahrefs. One link from the one domain is equal to 4900 links from one domain. So, I downloaded links those 4900 and added 4899 links to disavow tool. To disavow, to keep my site stable at rankings and safe from any future penalty. Is that a correct way to try disavow tool? The site rankings are as it is.
White Hat / Black Hat SEO | | AMTrends0 -
Duplicate keywords in URL?
Is there such a thing as keyword stuffing URLs? Such as a domain name of turtlesforsale.com having a directory called turtles-for-sale that houses all the pages on the site. Every page would start out with turtlesforsale.com/turtles-for-sale/. Good or bad idea? The owner is hoping to capitalize on the keywords of turtles for sale being in the URL twice and ranking better for that reason.
White Hat / Black Hat SEO | | CFSSEO0 -
Changing url to recover from algorythmic penalty
Hello, If I think that a website was hit algorithmically, I would like to buy a new domain name and publish all the content from the first website there. I will take the first site down and this one would be the only one this content. Will Google see that it's the same content than a penalized website posted before and will penalize the new domain name even though it has 0 links pointing to it? Regards.
White Hat / Black Hat SEO | | EndeR-0 -
Do you choose PA/DA over PR when purchasing expiring domains?
Hey guys, So a lot has been said about private blog network. I have but only 1 question: Do you choose PA/DA over PR when purchasing expiring domains or PR is most critical? Thanks a lot!
White Hat / Black Hat SEO | | nicenike0 -
Webmaster Tools Showing Bad Links Removed Over 60 Days Ago
Hello, One of my clients received the notorious message from Google about unnatural links late last March. We've removed several hundred (if not thousands) of links, and resubmitted several times for reconsideration, only to continue with responses that state that we still have unnatural links. Looking through the "links to your site" in google webmaster tools, there are several hundred sites / pages listed, from which we removed our link over 60 days ago. If you click each link to view the site / page, they contain nothing, viewable or hidden, regarding our website / address. I was wondering if this (outdated / inaccurate) list is the same as the one their employees use to analyze the current status of bad links, and if so how long it will take to reflect up-to-date information. In other words, even though we've removed the bad links, how long do we need to wait until we can expect a clean resubmission for reconsideration. Any help / advice would be greatly appreciated -
White Hat / Black Hat SEO | | Bromtec0 -
Check For Bad Directory Backlinks For Free
I used http://deletebacklinks.com/ yesterday to search 7 of the directories they have access to for searching bad links. I found one of my sites had links on these directories and I was able to remove them for fairly reasonable price. Thought this is a good tool to do a free quick check for any bad linkbacks on deindexed directories. I know this may be a small portion but every little bit helps.
White Hat / Black Hat SEO | | TheSEODR0 -
Which of these elements are good / bad link building practices?
Hi, I need some help. I recently got some help with an seo project from a contractor. He did 50 directory submissions and 50 article submissions. I got good results, going up about 20 places (still a long way to the first page!) on google.co.uk on a tough key word Since this project I learned article marketing is not cool. So I am wondering about what I should do next. The contractor has proposed a new bigger project consisting of the elements listed below. I don’t know which of these elements are ok and which aren’t. If they are not ok are they: 1) a waste of time or 2) something I could get penalized for? Let me know what you think?? Thanks, Andrew 100 ARTICLE SUBMISSIONS [APPROVED ARTICLES] -> 1 article submitted to 100 article directories 50 PRESS RELEASE SUBMISSIONS [APPROVED & SCREENSHOTS]-> 1 PR writing & submissions to top 50 PR distribution sites each 150 PRIVATE BLOGS SUBMISSION [APPROVED ARTICLES] -> 1 article submitted to 150 private blogs submission 100 WEBSITE DIRECTORY SUBMISSION -> 1 url (home page) submitted to 100 top free web directories 50 SOCIAL BOOKMARKING [CONFIRMED LINKS] -> 1 url of site submitted to top 50 social bookmarking websites 40 PROFILE BACK-LINKS [CONFIRMED LINKS] -> 1-3 url's of site submitted and create 40 profile websites 50 SEARCH ENGINES -> submission to all the major search engines 20 NEWS WEBSITES -> Ping all links from reports to news websites
White Hat / Black Hat SEO | | fleurya0
