How long will Google take to stop crawling an old URL once it has been 301 redirected
-
I need to do a clean-up old urls that have been redirected in sitemap and was wondering about this.
-
I agree with Ryan. As long as the old URLs are out there in the web, Google will follow them.
At the same time, redirected URL list will be very lengthy if you are maintaining a big site and maintaining them will consume your resources.
Check link value of the old URLs as a first step. If they have no value then you will be fine to retire such old URLs.
-
Google's crawler will always attempt to crawl any links it finds. If somewhere on the internet there is a link to an old URL and Google sees that link, it will never stop trying to explore that link. Each time it attempts to crawl the link, it will see the 301 and follow the redirect, but if you ever remove that redirect then the original URL will be seen.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to Choose destination page for a 301 redirect?
I am doing some SEO for a wedding chapel in Vegas. There are some old packages that no longer exist and the bounce rate for the page is high so I am planning to 301 the page. How to best determine the best 301 destination? I have a few options. As an example the page was optimized for garden weddings. The page itself does not place well in the SERPS for garden weddings in Las Vegas, but our outdoor wedding packages in Las Vegas page places in the top 10. So that page is in an option. However, there is a different location that has a garden setting. Is that a better choice? Some content might match better than others, but any page I choose would be relevant content. Thank you so much 🙂
Technical SEO | | leslieevarts0 -
Delete old blog posts after 301 redirects to new pages?
Hi Moz Community, I've recently created several new pages on my site using much of the same copy from blog posts on the same topics (we did this for design flexibility and a few other reasons). The blogs and pages aren't exactly identical, as the new pages have much more content, but I don't think there's a point to having both and I don't want to have duplicate content, so we've used 301 redirects from the old blog posts to the new pages of the same topic. My question is: can I go ahead and delete the old blog posts? (Or would there be any reasons I shouldn't delete them?) I'm guessing with the 301 redirects, all will be well in the world and I can just delete the old posts, but I wanted to triple check to make sure. Thanks so much for your feedback, I really appreciate it!
Technical SEO | | TaraLP1 -
Backlink management: 301 redirect unsuccessful.
I am managing my company's spammy backlinks using Open Site Explorer. Our company owns a few URLs that are related to our company or are iterations of our main URL. All of these additional URLs have 301 redirects to our main domain. Open Site Explorer has identified one of these URLs as having a spam score of 8 indicating a 56% chance of Google crawler penalization. Obviously, this is a red flag. Instead of being redirected to our main domain upon visiting the URL, I was directed to what seems to be an automatically generated, generic webpage with links that seem to have been generated by keywords from our main domain. I have seen this type of webpage before when incorrectly typing in URLs from other pages. They tend to look the same. They have a black background with the URL written in grey at the top and a rectangular related links bar. Is anyone familiar with my problem and could you offer any advice? Thanks, Ben
Technical SEO | | SOLVISTA0 -
Will blocking the Wayback Machine (archive.org) have any impact on Google crawl and indexing/SEO?
Will blocking the Wayback Machine (archive.org) by adding the code they give have any impact on Google crawl and indexing/SEO? Anyone know? Thanks! ~Brett
Technical SEO | | BBuck0 -
How long does it take for Google for deindexing pages?
Hi mozzers, We just launched a mobile website(parallel) and realized that it created many duplicate content with desktop URLs. I decided to add name="robots" content="No index, No follow" /> to the entire mobile site. My only concern is that I am still seeing the mobile site indexed when it's been almost a week I added these tags. Does anyone know how long it takes google to deindex your content? Thanks
Technical SEO | | Ideas-Money-Art0 -
301 Redirect From Dynamic Page To Static
I want to 301 redirect all "id" and "type" numbers from my page dynamic.php page (I have thousands of IDs and thousands of Types) all to a single URL. So for example the following.... www.mysite.com/dynamic.php?id=1&type=5 www.mysite.com/dynamic.php?id=2&type=5 www.mysite.com/dynamic.php?id=3&type=5 www.mysite.com/dynamic.php?id=1&type=6 www.mysite.com/dynamic.php?id=2&type=6 www.mysite.com/dynamic.php?id=3&type=6 ...would all be sent to: www.mysite.com/page.html How can this be done without doing a redirect for each ID/Type?
Technical SEO | | TheDude1 -
Setting up a 301 redirect from expired webpages
Hi Guys, We have recently created a new website for one of our clients and replaced their old website on the same domain. One problem that we are having is that all of the old pages are indexed within Google (1000s) and are just getting sent to our custom 404 page. We are finding that there is an large bounce rate from this and also, I am worried from an SEO point of view that the site could lose rank positioning through the number of crawl errors that Google is getting. Want I want is to set up a 301 redirect from these pages to go to the 'our brands' page. The reason for this is that the majority of the old URLs linked to individual product pages, and one thing to note is that they are all .asp pages. Is there a way of setting up a rule in the htaccess file (or another way) to say that all webpages that end with the suffix of .asp will be 301 redirected to the our brands' page? (there is no .asp pages on the new site as it is all done in php). If so, I would love it if someone could post the code snippet. Thanks in advance guys and if you have any other ideas then be my guest to suggest 🙂 Matt.
Technical SEO | | MatthewBarby0 -
301 Redirects Change?
Hi, Mozzers- I've noticed pages that are getting 301 redirected are staying out there longer. It used to be that you would implement a 301 redirect and then after a couple of months the old page would disappear out of Google's index. Over the last couple of months I've noticed pages lingering, popping up.... What gives? Thanks in advance! LHC
Technical SEO | | lhc670
