Welcome to the Q&A Forum

One of our clients has suddenly experienced a sudden increase in crawl errors for smart phones overnight for pages which no longer exist and there are no links to these pages according to Google.

There is no evidence as to why Google would suddenly start to crawl these pages as they have not existed for over 5 years, but it does come after a new site design has been put live. Pages do not appear to be in the index when a site search is used.

There was a similar increase in crawl errors on desktop initially after the new site went live, but these quickly returned to normal. Mobile crawl errors only became apparent after this.

There are some URL's showing which have no linking page detected so we don't know where these URL's are being found. WMT states "Googlebot couldn't crawl this URL because it points to a non-existent page". Those that do have a linking page are showing an internal page which also doesn't exist so it can't possibly link to any page.

Any insight is appreciated.

Andy and Mark at Click Consult.

Hi guys,

I'm working with the API at the moment.  I have most of the information I want, but I cannot find the uipl in any of the returned data using several methods.

The methods I have tried are:

http://lsapi.seomoz.com/linkscape/links/www.google.co.uk?Scope=page_to_domain&AccessID=xxx&Expires=xxx&Signature=xxx

http://lsapi.seomoz.com/linkscape/url-metrics/www.google.co.uk?AccessID=xxx&Expires=xxx&Signature=xxx

I can't seem to find how to get the uipl.  Any ideas?

Thanks in advance!

Hi Mark,

It's a bit pointless having the same links in the menu at the top and then again within the footer. It's unlikely that users will click the links within the footer and as an SEO, you want to minimize the amount of links on a page as much as possible.

In reply to your question for whether Google disregards the second set of links, it depends how you interpret "disregard". Google will still crawl, follow (and the links will still count when it comes to pagerank calculation). Google will however disregard the anchor text - have you heard of the first link counts rule?

http://www.seomoz.org/blog/results-of-google-experimentation-only-the-first-anchor-text-counts

There are however 3 methods to avoid the 1st link counts rule:

http://www.seomoz.org/ugc/3-ways-to-avoid-the-first-link-counts-rule

Good luck!

Should you continue link building? Yes.

Should you continue the same strategy as you previously had adapted? No.

Look at building quality, natural links, it's more important that you build these links now than ever to be honest.

Good luck!

Yes, the latter would be preferable from both an SEO and user point of view. You're more likely have a higher % of type in traffic with just the .com rather than using a "domian hack" too.

You should take into consideration the country of which you're looking to target when registering a "domain hack". For example, let's say you registered doma.in. You'd only be able to target an Indian audience because you're using the .in ccTLD.

There are a few ccTLD's that Google treats as TLD's, review the list here:

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1347922 (see Generic Country Code Top Level Domains (gccTLDs)

Good luck!

Hi Patrick,

That section is simply there to advice on any URLs that Google feels are wrongly excluded within the robots.txt

If the URLs are not wrongly excluded, don't worry about it showing in WMT's - it's there just as an advisory.

Good luck!

Hi Rob,

I personally use a tool called Xenu to identify broken links. Give it a whirl yourself, it's free:

http://home.snafu.de/tilman/xenulink.html

Good luck!

Hi Mike,

I'd personally say that you're fine, depending how you impliment it.

I personally believe that as long as the text that you add reads what the image contains, you're fine.

SEOmoz even use image replacement!

Good luck!

Perhaps you could add the London based address to all of the pages which Google is showing the map beneath the listing in the SERPs and mark the London address with rich snippets.

It's best to have the keywords that you're targeting as close to domain as possible although it wouldn't matter when you have just two short folder names such as /letters/abc.

Good luck!

Glad we're not the only ones freezing here, it's -3 here in the north west too!

I've just taken a quick look at barrettsteel.com and whilst there aren't any duplicate content issues listed within SERPs currently, there's nothing stopping canonical issues arising in the future. You can currently view the homepage via the following URL's:

http://www.barrettsteel.com/default.aspx

http://www.barrettsteel.com/Default.aspx (you can change the case for any of the letters, as-long as it says default.aspx (EG DEFAULT.aspx also works).

http://www.barrettsteel.com/

If someone links to /default.aspx for example, there's a chance that /default.aspx could get indexed. Therefore it's a good idea to add a canonical tag now before you do run into any issues. It doesn't take much to add it anyway, right?

I also like to implement a canonical tag so that if someone completely rips a page, the benefit will hopefully be given back to the site that the page was ripped from due to the canonical tag.

Also unrelated to your question but I found this within the SERPs: http://barrettsteel.netconstruct-preview.co.uk/ Whilst it now has no content, I'd recommend that you 301 redirect all pages from this site to the relevant page on the main site.

I'd advise you firstly look at the purpose of the nofollow attribute to answer this one. When you nofollow, you're telling Google that you don't want to pass your page's reputation to the pages linked to.

There's therefore no reason why you would want to nofollow the social profiles.

You want to give reputation to these pages, you want them to rank for the brand etc.

It wouldn't cause "over-optimization" signals but it's pointless nofollow'ing such links.

Hi Tyler,

From an SEO point of view, this is useless and you'd be better spending the time elsewhere.

Don't panic, but it does look like you've been hacked. Using Yahoo! Site Explorer you can see where that page is linked from:

http://siteexplorer.search.yahoo.com/uk/search?p=www.reflexologyonline.co.uk/reflexologyonline.php?Action=Webring&bwm=i&bwmo=d

^ 34 links to your page, all from other pages that have "Action=Webring" in the url. If you've never heard of a webring before then here's a primer: http://en.wikipedia.org/wiki/Webring - but basically it's a shady SEO tactic (from around 2000), the idea being that if everyone in the Webring links to each other on their page then that will give SEO benefit to each page in the ring.

If you hadn't said you knew nothing of it then I would have assumed you'd signed up to such a scheme but if not then someone has planted that page there

Solution:

1. Log into your FTP and delete reflexologyonline.php

2. I can see from this link that you're using Wordpress: http://www.reflexologyonline.co.uk/wp-admin/ so upgrade to the latest version.

3. If you notice this problem comes back then it is likely one of Wordpress plugins you have installed that has a security issue. In that case you're best disabling everything not-essental, and updating the essential ones to the latest version.

Of course your SEO agency/web developer will be able to do this for you if you're not part of the tech squad - at least I know we'd do it for our client.

Any more questions, give me a reply.

• Jon

Hi Mark,

It's a bit pointless having the same links in the menu at the top and then again within the footer. It's unlikely that users will click the links within the footer and as an SEO, you want to minimize the amount of links on a page as much as possible.

In reply to your question for whether Google disregards the second set of links, it depends how you interpret "disregard". Google will still crawl, follow (and the links will still count when it comes to pagerank calculation). Google will however disregard the anchor text - have you heard of the first link counts rule?

http://www.seomoz.org/blog/results-of-google-experimentation-only-the-first-anchor-text-counts

There are however 3 methods to avoid the 1st link counts rule:

http://www.seomoz.org/ugc/3-ways-to-avoid-the-first-link-counts-rule

Good luck!

Hi Anthematic,

Google does recognize these anchors, see this blog post from the Google blog detailing a search snippet for in-page anchors, you may have seen them in the SERPs before:

http://googleblog.blogspot.com/2009/09/jump-to-information-you-want-right-from.html

In reply to your second question, yes, they will be counted.

Both; 301 all pages from xyz.qc.ca to the corresponding page on xyz.ca and then set a change of domain within WMT's too.

You might see a dip in rankings initially however you can expect the rankings to return as previous over upcoming weeks.

Good luck!

Hi Naghirniac,

When you pay for a directory listing, you're paying for the review not the actual link. That being said, unless the directory is quality, don't go for it - money can be better spent elsewhere.

Good luck!

<code>This should work fine:</code>

<code>User-agent: *
Disallow: /cgi-bin/
Disallow: /development/</code>

<code>User-agent: rogerbot 
Allow: /development/</code>
```

Hi TT,

This should 301 redirect all pages without a trailing slash, to a version with the trailing slash:

RewriteCond %{REQUEST_URI} /+[^\.]+$``RewriteRule ^(.+[^/])$ %{REQUEST_URI}/ [R=301,L]