URL Errors Help - 350K Page Not Founds in 22 days
-
Got a good one for you all this time...
For our site, Google Search Console is reporting 436,758 "Page Not Found" errors within the Crawl Error report.
This is an increase of 350,000 errors in just 22 days (on Sept 21 we had 87,000 errors which was essentially consistently at that number for the previous 4 months or more). Then on August 22nd the errors jumped to 140,000, then climbed steadily from the 26th until the 31st reaching 326,000 errors, and then climbed again slowly from Sept 2nd until today's 436K.
Unfortunately I can only see the top 1,000 erroneous URLs in the console, of which they seem to be custom Google tracking URLs my team uses to track our pages.
A few questions:
1. Is there anyway to see the full list of 400K URLs Google is reporting they cannot find?
2. Should we be concerned at all about these?
3. Any other advice?thanks in advance!
C
-
No problem! Please let us know if you need any help once you have your results.
-
thank you all for the feedback. A comprehensive deep crawl is being conducted on the site now to help find out more. I truly appreciate all your guidance.
best
CC
-
I'm guessing this is for a news or ecommerce site? That is a lot of URLs.
Screaming Frog is a good resource, but I would look at the format of the URLs and how your platform creates URLs. I remember years ago many people were having issues with Wordpress, Joomla and other CMS's creating alternate URLs without the publisher knowing about them. Most likely its a setting in your system. Take a look at the URL settings and also the URLs that the tracking software is stating that it cannot find. Look for patterns across URLs and categories. You may find what you are looking for.
-
Not sure if this is related but myself and someone else have seen something similar around the same time happen, see here: https://moz.com/community/q/strange-increase-of-pages-not-found-gwt
-
Hi Usnseomoz,
1. Maybe perform a deeper search via a bit of kit like ScreamingFrog. This should help to further highlight any missing pages / errors etc
2. I would always be concerned with any problem until you have either been able to resolve it or discount it. Sounds like it could be the URL tracking parameters which are causing you some issues, especially if you are tracking users for multiple sources / sales affiliates. If they are solely used for tracking and no other purpose I would consider adding these parameter vairables to the crawl filters.
Side menu >> Crawl >> Url Parameters
https://support.google.com/webmasters/answer/6080548?rd=1Hope this is of some use
Cheers
Tim
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to handle blank, auto generated system pages/urls
Hi Guys Our backend system has been creating listing pages based on out of date and irrelevant data meaning we have hundreds of thousands of pages that are blank but currently indexable and active. They're almost impossible to access from the front end and have 0 traffic pointing at them but you can access these pages if you have the URL and i'm pretty sure due to the site architecture, google is crawling them regardless. For the most part, I think its likely best to 301 these pages to the most closely related page on the site but I'm concerned we're wasting crawl budget here. We don't want these pages to be crawled or found. Would a sound solution be to make them inactive, no-index and create a custom 404 in the event anyone (or the crawler) managed to get to them? Would this enormous increase in 404 pages cause us issues? Many thanks
Intermediate & Advanced SEO | | Jon.Kennett0 -
What is the Redirect Rule for corresponding https urls to new domain with the same https urls?
2 sites have the same urls but the owner wants just the 1 site. So I will be doing a 301 redirect with .htaccess from https://www.example.co.uk/sportsbook/SOCCER/today/ redirecting to https://www.example.com//sportsbook/SOCCER/today/ There are a lot of urls that are the same, so I was wondering what the rule is to put in the file please that will change them all to the corresponding urls? Would this be correct?... RewriteEngine on
Intermediate & Advanced SEO | | WSIDW
RewriteCond %{HTTPS_HOST} ^example.co.uk [NC,OR]
RewriteCond %{HTTPS_HOST} ^www.example.co.uk [NC]
RewriteRule ^(.*)$ https://example.com$1 [L,R=301,NC] Or would a simple rule like this work... redirect 301 / http://www.new domain.com/ If not correct could you please give me the correct rule, thanks! Then of course doing a change of address of address in webmaster tools after. Also... do I still need to do the forwarding from the https://www.example.co.uk/ domain provider after as well? Many thanks for your help in advance.0 -
If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?
If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?
Intermediate & Advanced SEO | | Gabriele_Layoutweb0 -
What are partial urls and why this is causing a sitemap error?
Hi mozzers, I have a client that recorded 7 errors when generating Xml sitemap. One of the errors appear to be coming from partial urls and apparently I would need to exclude them from sitemap. What are they exactly and why would they cause an error in the sitemap. Thanks!
Intermediate & Advanced SEO | | Ideas-Money-Art0 -
Redirect help
I work for a company that has a domain that is something like www.neat-stuff.com but most people just use www.neatstuff.com without the dash. The redirect for the homepage works fine. We recently launched a new site and if you use www.neat-stuff.com/category it redirects from the old site to the new site just fine. However if you use www.neatstuff.com/category it does not properly redirect to www.neat-stuff.com/category. How do I fix that?
Intermediate & Advanced SEO | | EcommerceSite0 -
SEO on page content links help
I run a website at the bottom we have scroller box which the old SEO guy used to contain all of the crap content so we can rank for keywords not on the page and put all of the links in to spread the link juice into the other inner category pages (some of these pages are only listed on our innerpages otherwise). We are trying to remove this content and add decent content above the fold with relevant long tail keywords in (it is currently decent but could do with expanding if we are removing this large chunk of text in theSEO box and some long tail keywords will be missing if we just remove it) we can add a couple of links into this new content but will struggle to list the category pages not on the left hand navigation. If we were to list all of the pages in the left hand nav would we dilute the power going to the main pages currently or would we be in the same position we are now? For example at the minute I would say the power is mainly going to the left hand nav links and then a small amount of power to the links in the SEO content if we put these into the nav will it not dilute the power to the main pages. Thank you for your time and hopefully your help.
Intermediate & Advanced SEO | | BobAnderson0 -
Does having a trailing slash make a url different than the same url without the trailing slash?
Does having a trailing slash make a url different than the same url without the trailing slash? www.example.com/services Or www.example.com/services**/** Does Google consider these to be the same link or does Google treat them as different links?
Intermediate & Advanced SEO | | webestate0 -
Rewriting URL
I'm doing a major URL rewriting on our site to make the URL more SEO friendly as well as more comfortable and intuitive for our users. Our site has a lot of indexed pages, over 250k. So it will take Google a while to reindex everything. I was thinking that when Google Bot encounters the new URLs, it will probably figure out it's duplicate content with the old URL. At least until it recrawls the old URL and get a 301 directing them to the new URL. This will probably lower the ranking of every page being crawled. Am I right to assume this is what will happen? Or is it fine as long as the old URLs get 301 redirect? If it is indeed a problem, what's the best solution? rel="canonical" on every single page maybe? Another approach? Thank you.
Intermediate & Advanced SEO | | corwin0