How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Fetch as Google is showing this, help!
Our Fetch as Google in Google Webmaster Tools is showing this. What is this?? Thanks! https://imgur.com/k6KOQZz
On-Page Optimization | | bluejay78780 -
URL Structure on Category Pages
Hi, Currently, we having the following URL Structure o our product pages: All Products Pages: www.viatrading.com/wholesale/283/All_Products.html Category Page: www.viatrading.com/wholesale/4/Clothing.html Product Page: www.viatrading.com/wholesale/product/LOAD-HE-WOM/Assorted-High-End-Women-Clothing-Lots.html?cid=4 Since we are going to use another frontend system, we are thinking about re-working on this URL Structure, using something like this: All Products Pages: www.viatrading.com/wholesale-products/ Category Page: www.viatrading.com/wholesale-products/category/ Product Page: www.viatrading.com/wholesale-products/category/product-title/ I understand this is better for SEO and user experience. However, we already have good traffic on the current URL Structure. Should we use same left-side filters on Category Pages as in All Products Page? Since we are using Faceted Navigation, when users filter the Category (e.g. Clothing) they will see same page as Clothing Category Page. Is that an issue for Duplicate Content? Since we are a wholesale company - I understand is using "/wholesale/products/" in URL for all product pages a good idea? If so, should we avoid word "wholesale" in product-title to avoid repeated word in URL? For us, SKU in URL helps the company employees and maybe some clients identify the link. However, what do you think of using the SEO-friendly product-title, and 301 redirect it to www.viatrading.com/BRTA-LN-DISHRACKS/, so 1st link is only used by company members and Canonicalized 2nd is the only one seen by general public? Thank you,
On-Page Optimization | | viatrading10 -
Google Rich Snippet
So i have been implementing rich snippets for work and all has been good until now, As you can see below the meta description has all of a sudden included the review date. The review date is the only date on the page. Any ideas what could be causing this? Thanks wqLKKl9
On-Page Optimization | | David-McGawn0 -
The value of changing URL structure
Dear Moz members, There have been many questions on this forum on this topic but I cannot find one that completely answers my question. We launched our new website about 7 months ago and the website contains around 3.000 product pages. The average page authority of the product pages is quite still quite low (12). The URL structure is built like this: www.website.com/catalog/ID/productname/ (with right keywords in the product name). So e.g the current URL is Our competitors rank higher on certain keywords while page authority (and DA) are significantly lower. Their URL structure is set up like this: www.website.com/productname/. Our most import keyword is "grafmonumenten" and the link we would like to rank on is: https://www.denhollandsche.nl/grafmonumenten/ My question now is:
On-Page Optimization | | stepsstones
1: how important is the length/depth of the URL structure?
2: is it beneficial for us to change the structure (www.website.com/productname/) use permanent redirects? My expierence is that changing the 'page url' can cause a short term drop in the serps, but can have positive effects on the longer term. Thanks for helping me out!0 -
URL contain special character
Hello, I am using URLs which contain special character such as ', ". I found in the Google Webmaster Tool report errors related URL contain ' character. Google have indexed partial URL from beginning to ' character and cut off the rest of URL. For example: I submitted URL www.example.com/vietnam-visa-corp'-test-page.html, then google report error NOT FOUND for URL www.example.com/vietnam-visa-corp I don't know why and how to fix it? Please help! Thanks,
On-Page Optimization | | JohnHuynh1 -
Long tail traffic - what is the best way to go back and add focus to repetitive long tail keywords?
Hey everybody, So, our niche doesn't have a million and a half searches per month, which makes a handle full of visitors look mighty enticing to a CMO Our price point is very high too, so to the question, is it worth taking the time to put a whole new content strategy in line for a few new visitors, the answer is yes. Now's the hard part. How on earth do I make 1,000 pages for similar topics? Is making new pages the best way to go about this? (probably so right? It's the only thing that I can see that would certainly increase likelihood of being more relevant, plus if I don't I will be missing out on the benefits of beefing up our site, AND the opportunity to more specifically answer a users query.) With phrases like "keyword" and "aftermarket keyword," the searcher is asking for two totally separate collections of results. I'm always reading about the importance of being there throughout the buyers complete purchasing /research process, which makes me think that considering doing anything other than creating unique pages is simply missing out.. Suggestions? Massive Content Strategy Help? Anybody? Thanks, TA
On-Page Optimization | | TylerAbernethy0 -
URL Strucutre
Hi there, Need some advice please on URL structure. I have been doing SEO for quite sometime now, however one thing that always get me is URL structure. I have a decision to make, its either: URL 1 /conditions/allergies/food/ URL 2 /conditions/allergies-food/ Lets say i am optimizing for the key-phase "Food Allergies" what do you think is best practice? I know that this is not a major factor in gaining high SERPs & maybe i'm thinking about it too much, however your input would be really helpful. Kind Regards,
On-Page Optimization | | Paul780 -
How do I fix a 404 with a 301
I understand the need for fixing 404's but I have yet to have a serious walkthrough of how to set up a 301. From all the talk on the forums and such I'm pretty sure this is easy but I've just never done it before and I could really use a walk through. Thanks
On-Page Optimization | | BenRWoodard1