External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL Structure On Site - Currently it's domain/product-name NOT domain/category/product name is this bad?
I have a eCommerce site and the site structure is domain/product-name rather than domain/product-category/product-name Do you think this will have a negative impact SEO Wise? I have seen that some of my individual product pages do get better rankings than my categories.
Technical SEO | | the-gate-films0 -
Remove page with PA of 69 and 300 root domain links?
Hi We have a few pages within our website which were at one time a focus for us, but due to developing the other areas of the website, they are now defunct (better content elsewhere) and in some ways slightly duplicate so we're merging two areas into one. We have removed the links to the main hub page from our navigation, and were going to 301 this main page to the main hub page of the section which replaces it. However I've just noticed the page due to be removed has a PA of 69 and 15,000 incoming links from 300 root domains. So not bad! It's actually stronger than the page we are 301'ing it to (but not really an option to swap as the URL structure will look messy) With this in mind, is the strategy to redirect still the best or should we keep the page and turn it into a landing page, with links off to the other section? It just feels as though we would be doing this just for the sake of google, im not sure how much decent content we could put on it as we've already done that on the destination page. The incoming links to that page will still be relevant to the new section (they are both v similar hence the merging) Any suggestions welcome, thanks
Technical SEO | | benseb0 -
Keyword in Domain Name
Hello!My website is www.enchantingquotes.com. I also own the domain www.enchantingwallquotes.com,which forwards to my site. About 90% of my business comes from the keyword "wall quotes". Should I consider changing switching to the enchantingwallquotes.com domain and redirecting? And if I do, do I need to recreate the entire website or is there an easier way that I am overlooking? Thank you for any advise/insight!
Technical SEO | | eqgirl0 -
Linking domains on the same C Block together
Hey, I have an online store selling dj equipment, sound & light products such as speakers, lasers, decks, pa systems, karaoke systems etc. I just bought a new domain but I registered it under a different name and address (my personal details). And I plan on hosting the website on a seperate server so it has no connection with my eCommerce store. The main purpose of the website will be to review the products I sell, write detailed how to guides for DJ's, party planners, mobile DJ's etc. There will be links on the current ecommerce website (which currently gets around anything from 500 to 1000 unique hits a day) going to the new blog website. But would I be better off keeping it on the same C Block even though they are going to be two very different websites and the blog may not always necessarily be about the products on my ecommerce website and may be products on say eBay, Amazon, etc. (In otherwords, it's going to be it's own website with an unbiased opinion, but the ecommerce site will be linking to it on certain products that are reviewed on there). Any help is appreciated 🙂
Technical SEO | | tomhall900 -
Home link tags slash or full domain name
Recently I was asked the question by a client who has some SEO knowledge..... We built their site and during testing we simply set the home page links to be: home rather than: home This was mainly to help while in test and dev as they were on different servers to the live site and if we had put the full domain in it kept taking us from dev/test to live. Is it bad to have just a slash for SEO purposes or is it REALLY that important on home links to have the full domain as a slash takes you back to the full domain? Thanks
Technical SEO | | spiralsites0 -
Sub Domain vs. New Root Domain for New Brand
Would you recommend a new brand be placed as a subdomain to the existing parent company or create a separate root domain for this new brand?
Technical SEO | | ScratchMM0 -
Why would you remove a canonical link?
Currently, my client's blog makes a duplicate page every time someone comments on a post. The previous SEO consultant told the developer to not put a canonical link directing it to the main blog post. Did taking out the canonical link result in these duplicate pages? My question is why would she recommend this action? Is it best to now add in the canonical link in or should we implement a 301 redirect or insert a index: no follow? Would adding a canonical link keep duplicate pages from happening in the future?
Technical SEO | | Scratch_MM0 -
Is this seen as a Link Exchange
If i give a self serve banner ad to someone on my blog or a image with a link and they give me a text link ad is that in googles eyes a link exchange or a one way link.
Technical SEO | | DavidKonigsberg0