How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site not showing up in Google search since move
Hi, hoping someone might help me with some answer(s) as to why our site no longer shows up in Google search results. Even when we type the full name and city into Google, the site is absent. Our Facebook page, LinkedIn and some backlinks show, but our site is missing. I can no longer find it in Google places. I'm sure I've done something wrong since moving from a static (flash-based) site to Wordpress. But the robots.txt file looks okay to me and the sitemap.xml file is present. Anyway, this is what happens when you ask a network technician handle website design... We know just enough to be dangerous! Here is the site in question: www.newfrontiertechnologies.com located in Shoreline WA. Any advice is much appreciated.
On-Page Optimization | | NFTECH0 -
Google webmaster markup validation error
Type: Schema product Property: Image Error: Missing required field "name (fn)". Google Webmaster is showing this error when I try to validate markup on webmaster. This is my domain - www(dot)wishpicker(dot)com Would be great if someone could please help with this. Thanks
On-Page Optimization | | bansheeviv
Prakul0 -
When You Add a Robots.txt file to a website to block certain URLs, do they disappear from Google's index?
I have seen several websites recently that have have far too many webpages indexed by Google, because for each blog post they publish, Google might index the following: www.mywebsite.com/blog/title-of-post www.mywebsite.com/blog/tag/tag1 www.mywebsite.com/blog/tag/tag2 www.mywebsite.com/blog/category/categoryA etc My question is: if you add a robots.txt file that tells Google NOT to index pages in the "tag" and "category" folder, does that mean that the previously indexed pages will eventually disappear from Google's index? Or does it just mean that newly created pages won't get added to the index? Or does it mean nothing at all? thanks for any insight!
On-Page Optimization | | williammarlow0 -
Articles URL
Hello, Currently, I am parsing article base on article ID on URL request. For example:
On-Page Optimization | | JohnHuynh
To go to the content of article **What is the visa on arrival? How to get it? **I am using URL like that http://www.vietnamvisacorp.com/news/what-is-the-visa-on-arrival--how-to-get-it--245.html and base on ID 245 to get this page content.
But, now I want to optimize this URL to http://www.vietnamvisacorp.com/news/what-is-the-visa-on-arrival--how-to-get-it.html. Then I got a problem: How to get an article without ID? Thanks,0 -
Long Url but makes no sense
Hi Just joined. Crawl states that I am getting a lot of errors, looks like the spider is getting confused and looping back on itself ? Is there a way to see where the crawl was formulated (ie where from) ? It is generating urls like: http://www.wickman.net.au/wineauction/wine_auction_alert.aspx/auction/auction/auction/auction/auction/auction/Default.aspx from http://www.wickman.net.au/wineauction/wine_auction_alert.aspx
On-Page Optimization | | blinkybill0 -
Keyword in URL
Hi everyone, I've heard many times that keyword prominence in url have a good impact for on-page optimization, even in SEOmoz it is one of the on-page factors. But what if i put keyword in URL then some of the page weight will be targeted to the page in the URL. Which in my vision makes only a negative impact. For ex. Targeted page <a>with keyword "buy car in NY" has a link with anchor text "buy car in NY" pointing on page **, so some weight from A will be transferred to B. Also I think this subject cover a cross linking, so I would like to know, what is the right way of doing cross linking and does it still brings any impact on keyword rankings in SERP.**</a> <a>**Good answer will be appreciated. Cheers, Russel**</a>
On-Page Optimization | | smokin_ace0 -
Why Does SEOMOZ Crawl show that i have 5,769 pages with Duplicate Content
Hello... I'm trying to do some analysis on my site (http://goo.gl/JgK1e) and SEOMOZ Crawl Diagnostics is telling me that I have 5,769 pages with duplicate content. Can someone, anyone, please help me understand: how does SEOMOZ determine if i have duplicate content Is it correct ? Are there really that many pages of duplicate content How do i fix this, if true <---- ** Most important ** Thanks in advance for any help!!
On-Page Optimization | | Prime850 -
Should I let Google index tags?
Should I let Google index tags? Positive? Negative Right now Google index every page, including tags... looks like I am risking to get duplicate content errors? If thats true should I just block /tag in robots.txt Also is it better to have as many pages indexed by google or it's should be as lees as possible and specific to the content as much as possible. Cheers
On-Page Optimization | | DiamondJewelryEmpire0