Webmaster Tools finding phantom 404s?
-
We recently (three months now!) switched over a site from .co.uk to .com and all old urls are re-directing to the new site.
However, Google Webmaster tools is flagging up hundreds of 404s from the old site and yet doesn't report where the links were found, i.e. in the 'Linked From' tab there is no data and the old links are not in the sitemap.
SEOmoz crawls do not report any 404s.
Any ideas?
-
Okay, thanks, and you are quite right, a whole morning is more than enough energy... gotta love SEO work!!!!
-
OK, well if it truly doesn't make sense (does sound odd, and it does seem like you've done the redirects fine) and three months is more than long enough for GWT to have caught up I'd take the above approach and periodically download the 404 list and seeing if there are any additions, as well as seeing if maybe Bing Webmaster Tools agrees with GWT.
If everything is redirecting fine, then I'd be inclined to just disregard it for the time being and focus my energies elsewhere.
Good luck with it! -
Maybe temporarily stop the 301 on the old site. Re-run your crawl reports and see if there were any 404s in existence on the old site that you hadn't previously thought. Plug the links and then reinstate the 301?
Either that or, if you're sure there's no problem, download the phantom 404s to CSV and then only take note of additions to that list in future?
-
That's the strange thing, all urls on the .co.uk are 301'ing to the new site.
-
If you can't tell where the links are coming from then the next bet is as Ben said is to identify the URLs which are being linked to and 301 them to a page which is closely related (or the home page otherwise), that way you don't lose any potential visitors to your site via those links.
-
Webmaster is picking up 494 pages returning a 404 and yet Roger Bot is finding none, so I can't even identify where the links are coming from. This is the part that is confusing me.
If I try and access any of the pages reported in the list, I am simply re-directed to the new site, as we are re-directing all .co.uk pages to the .com
-
I've had similar experiences after migrating sites over to a new CMS. The links are being picked up from somewhere so the best thing to do is create a 301 redirect if there is a valuable similar page to the 404. The only other option is to lose possible link juice from where ever the link is coming from. It's always better to have a pure link (no 301) but if you can't contact the original link then there is nothing you can do.
I'm curious as to where the links come from as well. Maybe someone will be able to speak to that.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Webmaster Tools is saying "Sitemap contains urls which are blocked by robots.txt" after Https move...
Hi Everyone, I really don't see anything wrong with our robots.txt file after our https move that just happened, but Google says all URLs are blocked. The only change I know we need to make is changing the sitemap url to https. Anything you all see wrong with this robots.txt file? robots.txt This file is to prevent the crawling and indexing of certain parts of your site by web crawlers and spiders run by sites like Yahoo! and Google. By telling these "robots" where not to go on your site, you save bandwidth and server resources. This file will be ignored unless it is at the root of your host: Used: http://example.com/robots.txt Ignored: http://example.com/site/robots.txt For more information about the robots.txt standard, see: http://www.robotstxt.org/wc/robots.html For syntax checking, see: http://www.sxw.org.uk/computing/robots/check.html Website Sitemap Sitemap: http://www.bestpricenutrition.com/sitemap.xml Crawlers Setup User-agent: * Allowable Index Allow: /*?p=
Technical SEO | | vetofunk
Allow: /index.php/blog/
Allow: /catalog/seo_sitemap/category/ Directories Disallow: /404/
Disallow: /app/
Disallow: /cgi-bin/
Disallow: /downloader/
Disallow: /includes/
Disallow: /lib/
Disallow: /magento/
Disallow: /pkginfo/
Disallow: /report/
Disallow: /stats/
Disallow: /var/ Paths (clean URLs) Disallow: /index.php/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /catalogsearch/
Disallow: /checkout/
Disallow: /control/
Disallow: /contacts/
Disallow: /customer/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/
Disallow: /aitmanufacturers/index/view/
Disallow: /blog/tag/
Disallow: /advancedreviews/abuse/reportajax/
Disallow: /advancedreviews/ajaxproduct/
Disallow: /advancedreviews/proscons/checkbyproscons/
Disallow: /catalog/product/gallery/
Disallow: /productquestions/index/ajaxform/ Files Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /STATUS.txt Paths (no clean URLs) Disallow: /.php$
Disallow: /?SID=
disallow: /?cat=
disallow: /?price=
disallow: /?flavor=
disallow: /?dir=
disallow: /?mode=
disallow: /?list=
disallow: /?limit=5
disallow: /?limit=10
disallow: /?limit=15
disallow: /?limit=20
disallow: /*?limit=250 -
Is there anyway to automatically find complete urls for 301 redirects?
Hello all, I'm working on a large site that is being prepared for a migration. After scraping on screaming frog, WMT, and deepcrawl, and deduping I have a list of 404 pages that need redirected. Many of these URLs are just incomplete URLs. For example the incomplete URL would be: "https://moz.com/beginners-guide-to" when it should be "https://moz.com/beginners-guide-to-seo" I have been finding the correct page manually searching google. With hundreds more to go, I am looking for a way the would be a little less laborious. Thanks! https://moz.com/beginners-guide-to-seo
Technical SEO | | AdamBohr0 -
Find all 404 links in my site that are indexed
Hi All, Find all 404 links in my site that are indexed. We deleted a lot of URl's from site but now i dont have the track of all we deleted. Any site/Tool can scan the index and give me the exact URL's so I can use https://www.google.com/webmasters/tools/removals?hl=en&rlf=all Regards Martin
Technical SEO | | mtthompsons0 -
Is there a tool to find a strange link
hi, i have used screaming frog seo and it has brought up a link to another site we run but this is a strange link the link is http://www.in2town.co.uk/www.clairehegarty.co.uk/virtual-gastric-band-with-hypnotherapy i have gone through the site on the home page as this is where it is and i cannot find, so i was just wondering if there was a tool that i can use to pin point where it is. any help would be great
Technical SEO | | ClaireH-1848860 -
Fix or Block Webmaster Tools URL Errors Not Found Linked from a certain domain?
RE: Webmaster Tool "Not Found" URL Errors are strange links from webstatsdomain.com Should I continue to fix 404 errors for strange links from a website called webstatsdomain.com or is there a way to ask Google Webmaster Tools to ignore them? Most of Webmaster Tools "URL Not Found errors" I find for our website are from this domain. They refer to pages that never existed. For example, one was to www.mydomain.com/virtual. Thanks for your help.
Technical SEO | | zharriet0 -
Backlink density & disavow tool
I am cleaning up my backlink profile for www.devoted2vintage.co.uk but before I start removing links I wanted some advice on the following: I currently have over 2000 backlinks from about 200 domains. Is this a healthy ratio or should I prune this? Is there a recommended max number of backlings per domain? Should I delete links to all or some of the spun PR articles (some of the article web pages have over 40 articles with links back to us)
Technical SEO | | devoted2vintage0 -
Help with Webmaster Tools "Not Followed" Errors
I have been doing a bunch of 301 redirects on my site to address 404 pages and in each case I check the redirect to make sure it works. I have also been using tools like Xenu to make sure that I'm not linking to 404 or 301 content from my site. However on Friday I started getting "Not Followed" errors in GWT. When I check the URL that they tell me provided the error it seems to redirect correctly. One example is this... http://www.mybinding.com/.sc/ms/dd/ee/48738/Astrobrights-Pulsar-Pink-10-x-13-65lb-Cover-50pk I tried a redirect tracer and it reports the redirect correctly. Fetch as googlebot returns the correct page. Fetch as bing bot in the new bing webmaster tools shows that it redirects to the correct page but there is a small note that says "Status: Redirection limit reached". I see this on all of the redirects that I check in the bing webmaster portal. Do I have something misconfigured. Can anyone give me a hint on how to troubleshoot this type of issue. Thanks, Jeff
Technical SEO | | mybinding10 -
Tool for extracting search queries
Hello, Does anyone know of or have a tool that takes referrer URLs coming from Google which extracts the search query from the URL string? Thank you
Technical SEO | | soeren.hofmayer0