How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site not showing up in Google search since move
Hi, hoping someone might help me with some answer(s) as to why our site no longer shows up in Google search results. Even when we type the full name and city into Google, the site is absent. Our Facebook page, LinkedIn and some backlinks show, but our site is missing. I can no longer find it in Google places. I'm sure I've done something wrong since moving from a static (flash-based) site to Wordpress. But the robots.txt file looks okay to me and the sitemap.xml file is present. Anyway, this is what happens when you ask a network technician handle website design... We know just enough to be dangerous! Here is the site in question: www.newfrontiertechnologies.com located in Shoreline WA. Any advice is much appreciated.
On-Page Optimization | | NFTECH0 -
Modify URL, how to re-index
hello, I have just modified URL, do I need to re-submit sitemap or something else to search engines?
On-Page Optimization | | JohnHuynh0 -
Why will google not index my Images
Hi I've added to index images in our sitemap although they are showing as being submitted Google hasn't indexed a single one. This has been the case for about 3 months. Is there any reason why Google would not index them? Thanks
On-Page Optimization | | tidybooks0 -
How to handle long dynamic meta tags?
Hi All, I have a site that has upwards of 40 000 pages and I'm redeveloping it so really want to get some SEO elements spot on for the new development. Hoe do I go about handling the following: The user creates a title for their advert which I use as the meta title. The problem is titles are quite often longer that the accepted lengths. How should I handle this? String manipulation down to the desired size, leave it as is or is there another solution? The meta descriptionn is pulled from a summary they created as part of their profile. Is this the right way to do it? Any advice would be appreciated. Ross
On-Page Optimization | | Mulith0 -
Canonical URL tags help I am not sure what this is
I am trying to get an A grade on my webpage and this is one of the critical steps canonical URL tags I cant find much information as to what this even is never mind fixing it. Thanks I am a total newbe at this any advice is appreciated
On-Page Optimization | | gemfirez0 -
How long can rich snippets be?
How long (how many words) can a description tag of a rich snippets code?
On-Page Optimization | | seoflorida0 -
Hierarchy and consistency in ecommerce URLs
One of the first things I remember reading about SEO and URLs, a long time ago, is that keywords are important, and hierarchy is important, for search engines and for users. Hierarchy in URLs would give the search engines an idea of the structure of the site, and users would be able to edit the URLs to continue navigating. I'm wondering about URLs, hierarchy and usability lately, since I've seen that ASOS uses a new URL structure on their site. At first glance, I thought it was brilliant, so I would like to get all of your opinions as well. For those of you that haven't seen the URLs: for categories, ASOS uses a structure as you would expect it, but for products they don't insert the category in the URL. Instead they insert the brand name as the first part of the URL, followed by the product title. Some examples: Category:
On-Page Optimization | | DocdataCommerce
www.asos.com/women/dresses/... Product:
www.asos.com/french-connection/french-connection-tie-waist-pocket-stripe-dress/... I can see the importance of brand name for a site like ASOS, and like how they stressed this by inserting not the category but the brand for products. I don't know how much ASOS still relies on organic non-ASOS related keyword traffic, but still. Now, for hierarchy, I guess a good internal linking structure will tell the search engines about the hierarchy of a site as well, right? So perhaps hierarchy in the URL isn't that important? Perhaps something like this would be just as good as anything, given a good internal link structure? www.onlinestore.com/category/
www.onlinestore.com/subcategory/
www.onlinestore.com/brand/product-title/ Now, I understand that if you use this structure, you wouldn't be able to have men/shirts and women/shirts, but let's say that you don't have subcategories that use the same names. In this case, how important is hierarchy? And, what do you think about this URL structure for an ecommerce site for which brands are important?0 -
Long meta description
Hello, I apologize in advance because the question must have already been posted, but 90% of my searches in "Search for a Question" drive me to a "no questions found" (i would be nice to improve this aspect of the Pro Q&A Forum 😉 So, a friend of mine asked to me what he should put for the meta description of automatically generated pages for his project. He has no tool to set a customized meta description for every page (and does not want to find one !) but he asked me the following : what is the less worse between : put the first words of the content (150 characters) put nothing and let google find what will be the better parts of the content for the user request put the whole content (600 characters) in order to avoid having just the begining of the content, which is not always useful in that case Did anyone try these options, what would be the less unproductive ? Thanks Loïc
On-Page Optimization | | mandinga0