How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long does Google take to reduce the index size?
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
On-Page Optimization | | cplastiras840 -
Dynamic URL Parameters + Woocommerce create 404 errors
Hi Guys,
On-Page Optimization | | jeeyer
Our latest Moz crawl shows a lot of 404-errors for pages that create dynamical links for users? (I guess it are dynamic links, not sure). Situation: On a page which shows products from brand X users can use the pagination icons on the bottom, or click on: View: / 24/48/All.
When a user clicks 48 the end of the link will be /?show_products=48
I think there were some pages that could show 48 products but do not exist anymore (because products are sold out for example), and that's why they show 404's and Moz reports them. How do I deal with these 404-errors? I can't set a 301-redirect because it depends on how many products are shown (it changes every time).
Should I just ignore these kind of 404-errors? Or what is the best way to handle this situation?0 -
Google Mobile usability issues
Hi,
On-Page Optimization | | MProenca
I received a mail from Google to "Fix mobile usability issues found on http://miguel-proença.com/ ". I've changed to a responsive design and all site pages are now mobile-friendly. However, on webmaster tools the usability issues still appear. Does anybody have a solution to this ? Thanks0 -
Google showing my content on the serps in a different domain
Hi all, Recently a partner of ours discovered that Google is showing a meta description on the serps for his homepage that is not his but ours. On his site, he sells add-ons for our software, so the name of our software appears many times and as well there are many links pointing to our site. He claims he hasn´t copied this text from us, and I have used some tools to verify this. I don´t understand how Google can get confused and show our text as the meta desctiption on the serps for his homepage. Any idea on why this happened?
On-Page Optimization | | Paessler0 -
Canonical URL Tag
Hi, I have two pages that are identical on my site: http://www.absolutepower.nl/creatine-monohydraat and http://www.absolutepower.nl/CREATINE/creatine-monohydraat Should I use the canonical URL tag in this case? Thanks, Jasper
On-Page Optimization | | Japking0 -
URL Extensions (with or without??!!)
Hello, SEOers~ Today I have a question about URL extensions. Which one is more search engine friendly between URL with extensions and without extensions? e.g. URL with extension : www.example.com/tv/lcd.jsp URL without extension : www.example.com/tv/lcd I heard that URL without extensions is in trend considering user experience. User experience is also important but I would like to know from SEO perspective. Please people~ Help me out with this~! Thanks.
On-Page Optimization | | Artience0 -
Finding 404 links
My first campaign is nearly complete and I have like 40 - 404's. How can I find where these links are on my site? We recently moved over to WP from html and that's what happened.
On-Page Optimization | | azguy0