External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hosting images externally
In these days of CDNs does it matter for SEO whether images (and PDFs etc.) are hosted off-site? Does it make a difference if images hosted on Flickr, photobucket etc. Thanks
Technical SEO | | bjalc20110 -
Sub Domain Redirect
Hey Everyone, Here is the situation : Currently, a website's sub domain is being redirected to the main website home page. We're having issues getting the sub domain pages indexed. Just want to confirm that it is because of the redirect on the sub domain URL. Should we kill the sub domain redirect and set it up as it's own page? Will that solve the indexing issue for the sub domain pages. More explanation below: subdomain.domain.com currently redirects to domain.com We're having issues indexing pages belonging to the sub domain ( subdomain.url.com/page1 or subdomain.url.com/page2) Appreciate your input in advance. Cheers,
Technical SEO | | SEO5Team0 -
Link profile
Hi All, I am doing a link profile audit I have few questions 1. Should i stop worrying about backlinks that i once had and now the websites is down or page is 404 2. The link is nofollow Also i have 60% of my site links few root link and many articles/blogs links pasted in sites without any anchor text, should i worry about them? Thanks
Technical SEO | | mtthompsons0 -
How to optimize for new subdomain when root domain has all link juice and built up authority?
We recently took control of a root domain for a business that was not doing e-commerce. They just had a single page business card website at the root domain. However, it had been around long enough to have built up some amount of domain authority and link juice. When we took over to enable the site with e-commerce, we redirected the root domain to point to a www subdomain where the store is now located. Now, in my seomoz campaign, i see that all the link juice and authority stats are in the root domain metrics, and the subdomain we are tracking has nothing. What is the best way for me to take advantage of all the built up authority for the root domain to help with the newly enabled ecommerce site at the subdomain? or am I basically starting from scratch since i have been reading that link juice does not flow as well from root domains to subdomains. thank you and happy new year to all!
Technical SEO | | devinjy0 -
Country Specific Domains
Is there any type of "best practice" for country level domains? I run a TLD .com, and have a few country specific domains (.co.uk, .eu, ...). Right now, I'm not doing anything with them. Previously, I had them redirected to the main .com, but didn't want to anger the Google gods with any type of duplicate content, redirects, or anything of that nature. Any suggestions on how to best utalize these domains?
Technical SEO | | ShippingContainer0 -
Domain Relocation
My client is running a online news website, which is running for 4 years. He's now looking to change the site into a new domain. I would like to know what are the factors to look out for when changing the site into new domain (In SEO point of view)
Technical SEO | | augmoz110 -
Onpage linking
On my homepage, I currently link to about 40 internal pages. I'm considering altering the internal linking structure to have 50-100 links on the 2nd level pages. If I was to do this, I'd only need 8 homepage links. Do you think the 8 pages linked from the homepage would go up in the SERPs as the pagerank would be less diluted? I've heard so many mixed views on this. Be interested to see what people here think. Thanks, Pete
Technical SEO | | PeterM220