Why are these URL's suddenly appearing in WMT?
-
One of our clients has suddenly experienced a sudden increase in crawl errors for smart phones overnight for pages which no longer exist and there are no links to these pages according to Google.
There is no evidence as to why Google would suddenly start to crawl these pages as they have not existed for over 5 years, but it does come after a new site design has been put live. Pages do not appear to be in the index when a site search is used.
There was a similar increase in crawl errors on desktop initially after the new site went live, but these quickly returned to normal. Mobile crawl errors only became apparent after this.
There are some URL's showing which have no linking page detected so we don't know where these URL's are being found. WMT states "Googlebot couldn't crawl this URL because it points to a non-existent page". Those that do have a linking page are showing an internal page which also doesn't exist so it can't possibly link to any page.
Any insight is appreciated.
Andy and Mark at Click Consult.
-
Hav you crawled your redesigned site with a tool like Xenu or ScreamingFrog? That will help ferret out any bad links / 404 pages. Also, did you submit an updated sitemap with the redesign? Use 301s? Etc.
Edit: Looks like Dirk above hit on the most likely issue regarding 301s. You should be set with that.
-
If you recently did a site redesign - you probably deleted old redirect rules as well. Google seems to keep all the url's it has indexed over time stored somewhere - and the moment the old redirects disappear - you'll see these very old url's popping up in webmastertools. I had the same experience after a recent migration, and if you search on this forum, you'll find plenty of other people who have encountered the same thing.
Nothing to worry about - if these errors in WMT are annoying you could put the old redirect rules in place again, or if they are using a seperate folder structure, you could delete these folders via WMT. These errors will disappear over time, even if you don't do anything.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Clean URL vs. Parameter URL and Using Canonical URL...That's a Mouthfull!
Hi Everyone, I a currently migrating a Magento site over to Shopify Plus and have a question about best practices for using the canonical URL. There is a competitor that I believe is not doing it the correct way, so I want to make sure my way is the better choice. With 'Vendor Pages' in Shopify, they show up looking like: https://www.campusprotein.com/collections/vendors?q=Cellucor. Not as clean. Problem is that Shopify also creates https://www.campusprotein.com/collections/cellucor. Same products, same page, just a different more clean URL. I am seeing both indexed in Google. What I want to do is basically create a canonical URL from the URL with the parameter that points to the clean URL. The two pages are very similar. The only difference is that the clean URL page has some additional content at the top of the page. I would say the two pages are 90% the same. Do you see any issue with that?
Technical SEO | | vetofunk0 -
What's best practice for cart pages?
i don't mean e-commerce site in general, but the actual cart page itself. What's best practice for the links that customers click to add products to the cart, and the cart page itself? Also, I use vanity URLs for my cart links which redirect to the actual cart page with the parameters applied. Should I use use 301 or 302 redirects for the links? Do I make the cart page's canonical tag point back to the store home page so that I'm not accruing link juice to a page that customers don't actually want to land on from search? I'm kinda surprised at the dearth of information out there on this, or maybe I'm not looking in the right places?
Technical SEO | | VM-Oz0 -
Community Discussion - What's been your experience with accessibility?
When Laura Lippay came to me with the idea to write a series of posts on the Moz blog about SEO and accessibility, it really got my gears turning. As the blog manager, I realized I'd been thinking about all sorts of ways to make the blog the best it can be, but accessibility was one place I had yet to explore in-depth. While I have my own goals and projects around this topic churning along in the background, I'd love to hear what the community's done to be inclusive to all users of the Internet. What've you struggled with in terms of making sites you've worked on accessible -- both technically and as an initiative in general? What's often missing that you've become passionate about including? Do you have any big wins you're especially proud of and want to share? Looking forward to reading your thoughts and stories, folks! 🙂
Technical SEO | | FeliciaCrawford1 -
Using "Div's" to place content at top of HTML
Is it still a good practice to use "div's" to place content at the top of the HTML code, if your content is at the bottom of the web page?
Technical SEO | | tdawson090 -
Changes to 'links to your site' in WebMaster Tools?
We're writing more out of curiosity... Clicking on "Download latest links" within 'Links to your site' in Google's WebMaster Tools would usually bring back links discovered recently. However, the last few times (for numerous accounts) it has brought back a lot of legacy links - some from 2011 - and includes nothing recent. We would usually expect to see a dozen at least each month. ...Has anyone else noticed this? Or, do you have any advice? Thanks in advance, Ant!
Technical SEO | | AbsoluteDesign0 -
Using a Feedburner RSS link in your blog's header tag
It was suggested in Quick Sprout's Advanced SEO guide that it's good form to place your Feedburner RSS link into the header tag of your blog. Anyone know if this needs to be done for every page header of the blog, or just the home/main/index page? Thanks
Technical SEO | | Martin_S0 -
How to remove entire directory off Google's Cache
The old version of Webmaster tools used to allow you to select whether to remove a single page from index or an entire directory.
Technical SEO | | vpahwa
http://www.canig.com/pageimages/submitremovalrequest.jpg How can I do this with the new Webmaster Tools? I can't find the option to remove an entire directory.0 -
Best practice: unique meta descriptions on blog 'tag' pages
Hi everyone, I'm curious, are there best practices for introducing unique meta descriptions on blog tag pages (I'm using wordpress)? For instance, using platinum seo, on an original post, the meta description is either the excerpt or a specified custom sentence. It doesn't appear that platinum seo allows for custom descriptions on tag pages. Love to hear your thoughts. Thanks! Peter
Technical SEO | | peterdbaron1