Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Embedded links/badges
Hi there Just picking up on something Rand said in his blog analysing his predictions for 2014. Rand predicted that Google will publicly acknowledge algorithmic updates targeting...embeddable infographics/badges as manipulative linking practices While this hasn't exactly materialised yet, it has got me thinking. We have a fair few partners linking to us through an embedded badge. This was done to build the brand, but the positives here wouldn't be worth being penalised in search. Does anyone have any further evidence of websites penalised for doing this, or any views on whether removing those badges should be a priority for us? Many thanks
White Hat / Black Hat SEO | | HireSpace0 -
Whether to use new domain or old ecommerce site domain that has been incomplete for a long time.
Hello, We are starting a second store in our niche. Which of the following should I choose: A. We have a site from a year and a half ago that we put content on but never actually added products. The category and article content needs to be completely rewritten. We will completely rewrite the content to be much better and up to date. We're planning on adding products and rewriting the manufacturer descriptions. B. We could use a new domain that is closer to exact match for our main keyword. We'd just buy one for $15 I don't know whether A or B would be the fastest way to get the site going. I'm concerned that leaving a site half done for a year could cause an issue, but I really don't know. If you've got experience with this, please advise. Thank you.
White Hat / Black Hat SEO | | BobGW0 -
It's not link buying, but...
Which of these strategies, if any, cross the line from relationship building to link buying? Assume all links are do-follow. You're a local business. You give the local Boys & Girls club a few hundreds buck a year. In return, you get a very nice link on their Sponsorship page for 12 months. You send a sample of your product to influential bloggers, for the purpose of a review and hopefully a link back to your website. One of your clients is a college bar. You invite 50 college kids over for a slow evening and stuff them full of chicken wings. Then, you ask them to please review and link to the bar on their college wiki. You give a client a free service, in exchange for that client linking to your business on its blog roll. You take a blogger out to lunch, and pick up the tab. Later that day, the blogger writes up an amusing little story for the blog, and links back to your desired website. In your email newsletter, you put out a request to your customer base, "Please link to my website, and I'll provide you a special 20% off coupon."
White Hat / Black Hat SEO | | ExploreConsulting1 -
Adding Domain Names
Hi all, I am pretty new to SEO but i wanted to check out an idea with you guys, I have a business that is called Totaldriveways and the URL is www.totaldrivewaysne.co.uk Although Driveways is a big part of what we do, we are also heavily involved in other services to do with landscaping. The idea ive got is to brand each service with the "Total" for eample Total Driveways is a reconisgened name in my area and i would like to expand that into Total Patios Total Artificial Grass Total Driveway cleaning so each page and service come under a similar brand, my question is is it worth buying these domains also and directing them at the specific pages that they are relevant to, these domains are obviously keyword rich and am guessing will create extra traffic, but will i be penalised at all by google etc. I know this is probably simple stuff to alot of you's but i have no idea Thanks John
White Hat / Black Hat SEO | | totaldriveways0 -
Anybody have useful advice to fix a very bad link profile?
Hello fellow mozzers. I am interested in getting the communities opinion on how to fix an extremely bad link profile, or whether it would be easier to start over on a new domain. This is for an e-commerce site that sells wedding rings. Prior to coming to our agency, the client had been using a different service that was doing some serious black hat linkbuilding on a truly staggering scale. Of the roughly 53,000 links that show up in OSE, 16,500 of them have the anchor text "wedding rings", 1,300 "wedding ring sets", etc. For contrast, there are only two "visit website", and just one domain name anchor text. So it is about the farthest from natural you can get. Anyway, the site traffic was doing great until the end of February, when it took a massive hit and lost over half the day to day traffic volume, and steadily declined until April 24th (Penguin), when it took another huge hit and lost almost 70% of traffic from Google. Note that the traffic from Yahoo/Bing stayed the same. So the question is, is it worth trying to clean up this mess of a backlink profile or would it be smarter to start fresh with a new domain?
White Hat / Black Hat SEO | | CustomCreatives0 -
So what's up with UpDowner.com?
I've noticed these guys in link profiles for several sites I manage. They'll usually show up around 1,000-10,000 times in the backlink profile. From what I can tell they index websites, build up keyword relationships, and then when you search for something on their site (e.g. poker) they'll present a list of related sites with stats about them. The stats seem to be yanked straight from Alexa. Where the backlink comes from is that every time 'your' site shows up for a search result they'll put a little iframe that contains your site. This means if your site's name/keywords are pretty broad, you could be showing up thousands and tens of thousands of times as being linked from these guys on their pages that Google indexes. And Google indexes, boy do they ever. At the height, they had over 53 million pages indexed. That has apparently shrunk now to around 25 million. I believe their strategy is to generate a crap-load of automated content in the hopes they can cash in on obscure long tails. So my questions for you guys are: Are you seeing them in your backlinks too? Should I block their spider/referrers? What is their deal man?
White Hat / Black Hat SEO | | icecarats0 -
Unique businesses - unique domain names?
A client of mine owns a studio space where he teaches yoga and martial arts. It's a new business and we're deciding how to create the website(s) and which domain(s) to buy. The idea right now is to have 3 websites for each side of the business, and I'm looking for validation of this idea. I haven't been able to find an answer in the Q&A forum that quite applies to our situation. Website 1: for the studio itself. The audience is other yoga teachers, martial arts teachers, or personal trainers. He will rent out the studio space to them and they bring in their own clients. Content and keywords will relate to this. Website 2: yoga classes. The audience is members of the public who want to take yoga classes. Content and keywords will relate to this. Website 3: martial arts. The audience is members of the public who want to take martial arts classes. Content and keywords will relate to this. We will make certain there's no duplicate content on the sites, but it makes sense for them to link to each other because they're similar in nature (personal health and fitness at the studio), and the latter 2 services are offered at the studio, of course. Question 1: (a) is it a good idea to get a separate domain for each site? for example: www.city-studio.com, www.unique-name-yoga.com, www.unique-name-martial-arts.com (b) Or would it be better to keep it all under city-studio.com and use subdomains like yoga.city-studio.com and martial-arts.city-studio.com? In either case, the keywords "yoga" and "martial arts" would be in the domain name, which has benefit. Does that still apply for subdomains? (c) Or would these services even be considered similar enough that I just use www.city-studio.com/yoga.php and www.city-studio.com/martial-arts..php There will of course be several pages on yoga and several on martial arts. Question 2: if registering multiple domains, they will interlink as much as possible. (a) what do we consider when buying the domains? (b) use a different address for WHOIS of each domain? (c) can technical contact be the same address (mine, the consultant)? (d) use a different credit card for each? (e) ok if the name on the credit card is the same? (f) can we register them all the same day? (h) same domain registrar? (i) same host? we don't want to appear black hat by having multiple sites, but I think it's very legitimate to have the business split into 3 sites like this just because they're separate sides of the business with different audiences, content, keywords. Question 3: when the domains come up for renewal in 1 year (or more), would it be safe to switch them all over to one credit card then, for convenience to the owner? Question 4: is there anything important I haven't mentioned here? I appreciate any input and discussion.
White Hat / Black Hat SEO | | Kenoshi0 -
Are paid reviews gray/black hat?
Are sites like ReviewMe or PayPerPost white hat? Are follow links allowed within the post? Should I use those aforementioned services, or cold contact high authority sites within my niche?
White Hat / Black Hat SEO | | 10JQKAs0