External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will Redirection of Unnatural Links to the New Domain will work?
Hi All, If a website www.mywebsite.com has received 100 unnatural links and because of that rankings are dropped. I want to follow the below strategy to protect my website from these 100 unnatural links. Please let me know if it will work: Buy a new domain www.newdomain.com with new whois record (Not related to www.mywebsite.com ) Do 301 redirection of all 100 unnatural links (Those are pointing to www.mywebsite.com) to www.newdomain (New domain) Do 302 redirection from www.newdomain (New domain) to www.mywebsite.com (Old Domain) After applying the above strategy, will google still consider those 100 unnatural links for Old domain??
Technical SEO | | RuchiPardal0 -
Domain Registrar
Looking for opinions on some good domain registrars since I have decided to transfer from Godaddy. A question while I'm here discussing the topic. Does the reputation of a Domain Registrar affect SEO in any form?
Technical SEO | | greenfoxone0 -
Registering expired domains
Hi there, I've found a good domain that is available for a new project. It has been expired for about 4 months or so. It has a couple of links, with the domain name as an anchor, nothing horrible. Will buying a domain like this be safe from an seo perspective? I'm guessing it would be treated the same as buying a new domain that has never been registered before, Would I be right? Peter
Technical SEO | | PeterM220 -
Links from the same server has value or not
Hi Guys, Sometime ago one of the SEO experts said to me if I get links from the same IP address, Google doesn't count them as with much value. For an example, I am a web devleoper and I host all my clients websites on one server and link them back to me. Im wondering whether those links have any value when it comes to seo or should I consider getting different hosting providers? Regards Uds
Technical SEO | | Uds0 -
Is it better for our Blog to be blog.domain.tld or domain.tld/blog ?
I'd dread the answer being the latter rather than the former as we've spent two years building it blog.domain... However I noticed SEOmoz are domian.tld/blog and it got me thinking.... Cheers. R.
Technical SEO | | RobertChapman0 -
Can I do a redirect to a new domain name only a couple of weeks after having redirected to another domain?
I have a client with two website with very similar content. Both had a lot of inbound links and performed fairly well in SERPS. We recently combined both sites and have redirected one of the domains to the other. The traffic dipped slightly initially, but is recovering nicely. Now the client registered a new domain name he would like to use for the site. Should I wait a few weeks for everything to settle down after the first redirect/consolidation of sites before doing a new redirect to a new domain name, or should I not worry about having any issues with doing it right away?
Technical SEO | | Drewco0 -
Links from Youtube Channel
I stumbled across this blog post: http://garyreid.com/youtube-removes-nofollow/ and also this one : http://www.kevin-barnes.com/youtube-secret-authority-loophole/ which talks about no-follow links from your Youtube Channel Page. We've setup a Youtube channel, and have begun updating it regularly, however the link appears to be a redirect-type link -presumably this means no link juice is passed? The code of the link on our Youtube channel: http://www.pretavoir.co.uk The second blog mentions building PA on your Youtube channel by commenting on other videos which then links back to your channel page - if that juice can't go to your site, then I assume the technique is of limited use? Apart from boosting your Youtube Channel's rankings of course, which I guess can't hurt.
Technical SEO | | seanmccauley0 -
What should I set my domain setting to?
In Google Wemnaster tools, I have the option to set it to either have as default the "www" or without it. What are the pros and cons of one way or the other . . . or is this a way more complicated question/can of worms I have opened?
Technical SEO | | damon12120