External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Via this intermediate link issue - for (multiple domains) same brand
Hi, I have sudden increase in links pointing to my .com website from my .ca website, i have recently launched a new ,.ca website to target Canada and i don't know why i see 1000's of links from .ca is point to my .com websites has backlinks and i am afraid it could hurt my seo for .com, because ,ca is having no domain authority or no ranking currently. however .com has good rankings currently with decent link profile Can any one help me on how i can get rid of this backlinks from .ca to com should i just add no follow links to all my links which are pointing from .ca to .com? Please help Regards Anoop
Technical SEO | | Vitarockstore0 -
What's the best way to pass link juice to a page on another domain?
I'm working with a non-profit, and their donation form software forces them to host their donation pages on a different domain. I want to attempt to get their donation page to appear in their sitelinks in Google (under the main website's entry), but it seems like the organization's donation forms are at a disadvantage because they're not actually hosted on that site. I know that no matter what I do, there's no way to "force" a sitelink to appear the way I want it, but... I was trying to think if there's a way I can work around this. Do you think 1) creating a url like orgname.org/donate and having that be a 301 redirect to the donation form, and 2) using the /donate redirect all over the site (instead of linking directly to the form) would help? Are there alternatives other folks recommend?
Technical SEO | | clefevre0 -
Disavowing links, Is it effective?
Looking for your experiences with disavowing back-links? We've been flooded with new clients who need spammy link removal services and wanted to hear more about your experience with the disavow tool. For sites that have been penalized, how long did it take for them to come back using the disavow tool? Did you see sites come back after the next algo update? Here's the basics of our services for link deletion: 1. Find all the spammy links
Technical SEO | | Keith-Eneix
2. Contact webmasters to delete them
3. Disavow all spammy links that are part of an obvious network
4. Implement a content plan for new quality links to get the site healthy again.
5. Report on all links removed and new links attained Just want to make sure our processes are in line with what everyone else is doing?0 -
Link building to ROOT domain OR to WWW.?
Hello, Here I come with one more 'sensitive' question, hoping that you SEO gurus could give some input on. My title explains pretty much what I'm wondering about, but let me give you some short data. I have from .htaccess file set that all traffic goes to WWW.mydomain.com. I know that it is 'better' for search engines not to have duplicate destinations as that can give decreased page rank because of 'double content'. As for search engines http://domain.com and http://www.domain.com is totally different domains. Now wondering one thing: If I build a several thousands of backlinks at various sources, blogs, directories, web sites etc etc. - shall I link to domain ROOT or shall I include WWW prefix? When looking at Moz Keyword Analysis for my domains, I can see a block about 'Linking Root Domains' and 'Page Linking Root Domains'. But no 'www' variable (sub-domain) there. As I have already set canonical part so everything shows with WWW on my website - what logic shall I use when building backlinks? How will search engine translate the link juice in regards I wrote above? Thanks in advance, great forum!
Technical SEO | | SEOisSEO0 -
How to prevent duplicat content issue and indexing sub domain [ CDN sub domain]?
Hello! I wish to use CDN server to optimize my page loading time ( MaxCDN). I have to use a custom CDN sub domain to use these services. If I added a sub domain, then my blog has two URL (http://www.example.com and http://cdn.example.com) for the same content. I have more than 450 blog posts. I think it will cause duplicate content issues. In this situation, what is the best method (rel=canonical or no-indexing) to prevent duplicate content issue and prevent indexing sub domain? And take the optimum service of the CDN. Thanks!
Technical SEO | | Godad0 -
Showing My domain twice
I have a question, I am showing my domain twic, how can i have page rank 32 and page rank 44 for the same page? Regards | The UK's Leading Probate Brokers - Lowest Probate Quote guarantee! Probate experts. http://finalduties.co.uk/ 83 (over by 13) 32 1 The UK's Leading Probate Brokers - Lowest Probate Quote guarantee! Probate experts. http://www.finalduties.co.uk/ | 83 (over by 13) | 44 | 112 |
Technical SEO | | Chris__Chris0 -
Domains and Hosting Question
I bought hosting for unlimited domains on Godaddy. It's not a dedicated server. It was just $85 a year. I have unlimited latency but a limited amount of "space." I don't know a lot about hosting servers etc... My question is relatively simple. When I go in GoDaddy to my hosting. There is a site that shows up as hosted, and all of the other sites show up under that site in it's directory. If you type the name of the site I bought the hosted package on, then type a forward slash and the name of one of the other sites on the hosting package, you will actually go to the other website. What is this relationship? Is it normal? Does that make all of my websites subdomains of the main site (that I bought the hosting package on)? I don't fully comprehend how this effects everything...
Technical SEO | | JML11790 -
Campaigns Domain and Subdomain... ?
I made two separate campaigns before I understood the meaning of "subdomain". I make one campaign for my www.com and another for my .com. I now realize I should have made the .com the domain and the www. the subdomain in the same campaign. Is there a way to edit this? Thanks!
Technical SEO | | musicforkids0