How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Pages Not Showing Up In Search After Being Indexed....
Hello, I'm trying optimize my google search for local business in Fort Myers, FL area. I've created specific keywords that people are searching and have made my page title and on-page experience reflect this relevency. However, the page isn't appearing anywhere on google search even though search console has stated that the page has been indexed. My question is, how do I trouble shoot this so that I can appear in local search for these search terms?
On-Page Optimization | | scottgray06200 -
Ranking for a word when your image is showing up as #2?
I'm trying to rank for several specific words in my industry. I've noticed that many of the pages I'm trying to rank for the specific words, the main image of the page is being ranked. So my question is should I still try to rank my page even though the image is ranking as the first image displayed in the google results. Will this count against me by Google if I do? Or should I try to make the image being ranked more enticing for google searchers to try to increase traffic? Right now the images are not generating many clicks.
On-Page Optimization | | mr_w0 -
If Google still shows my old title in the search results then there is no way my title change could have changed my ranking yet right?
If Google still shows my old title in the search results then there is no way my title change could have changed my ranking yet right? In other words if I changed from TItleA to TitleB but search results and cache still show TitleA then any SERP drops or increases would still be entirely based on titleA right? Thanks.
On-Page Optimization | | Rezo0 -
DMCA Complaint to Google
I have several sites copying the content from my pages. I filed numerous complaints with Google and checked under the DMCA dashboard and it's showing my requests are approved; however, still several days later the infringing pages still remain. I'm a bit confused by this as I thought Google was supposed to remove the page. I guess I'm not understand why it was "approved" when I don't see changes. Am I supposed to contact the site's owner to get the pages down now? I don't want to show duplicate content to my site. Please help!
On-Page Optimization | | tutugirl0 -
Can't canonical, but need pages to show in Google News
We are a news media site in which much of our content is third-party, and already published by several other sources. Our current version of our CMS doesn't expose head tags, so I can't canonical to the original and avoid a duplicate content penalty. Is it ok for news sites NOT to use canonical, or do I have to NOINDEX until our CMS is fixed?
On-Page Optimization | | Aggie0 -
Hierarchy and consistency in ecommerce URLs
One of the first things I remember reading about SEO and URLs, a long time ago, is that keywords are important, and hierarchy is important, for search engines and for users. Hierarchy in URLs would give the search engines an idea of the structure of the site, and users would be able to edit the URLs to continue navigating. I'm wondering about URLs, hierarchy and usability lately, since I've seen that ASOS uses a new URL structure on their site. At first glance, I thought it was brilliant, so I would like to get all of your opinions as well. For those of you that haven't seen the URLs: for categories, ASOS uses a structure as you would expect it, but for products they don't insert the category in the URL. Instead they insert the brand name as the first part of the URL, followed by the product title. Some examples: Category:
On-Page Optimization | | DocdataCommerce
www.asos.com/women/dresses/... Product:
www.asos.com/french-connection/french-connection-tie-waist-pocket-stripe-dress/... I can see the importance of brand name for a site like ASOS, and like how they stressed this by inserting not the category but the brand for products. I don't know how much ASOS still relies on organic non-ASOS related keyword traffic, but still. Now, for hierarchy, I guess a good internal linking structure will tell the search engines about the hierarchy of a site as well, right? So perhaps hierarchy in the URL isn't that important? Perhaps something like this would be just as good as anything, given a good internal link structure? www.onlinestore.com/category/
www.onlinestore.com/subcategory/
www.onlinestore.com/brand/product-title/ Now, I understand that if you use this structure, you wouldn't be able to have men/shirts and women/shirts, but let's say that you don't have subcategories that use the same names. In this case, how important is hierarchy? And, what do you think about this URL structure for an ecommerce site for which brands are important?0 -
No Data Available for this URL
Hi,
On-Page Optimization | | ostiguyj
I really don't understand why I have this message "No data available for this URL"
in my SEOMOZ campain. (www.bienchezsoi.ca) When I look at my page rank, I get a score of 0 I have no idea of to fix it. Please help. Thanks0 -
Absolute URLs
Hi, this is a very basic question but I want to confirm, as I remembered it was consider a good practice to use the absolute version of your links when linking to other pages of your site, not for any issue related to passing authority or PageRank, but because if someone scraps your content then they would take the links as well (as if they didn't remove them). Have the practices for internal linking with absolute or realtive URLs changed in any way? Which is the best way? absolute or relative? is there any harm for using the relative version? Relative: Absolute: [](<strong><em>http://www.cheapdomain.com/myfolder/mypage.html)[](<strong><em>http://www.cheapdomain.com/myfolder/mypage.html) [Thanks!](<strong><em>http://www.cheapdomain.com/myfolder/mypage.html)
On-Page Optimization | | andresgmontero0