Pages with 301 redirects showing as 200 when crawled using RogerBot
-
Hi guys,
I recently did an audit for a client and ran a crawl on the site using RogerBot. We quickly noticed that all but one page was showing as status code 200, but we knew that there were a lot of 301 redirects in place. When our developers checked it, they saw the pages as 301s, as did the Moz toolbar.
If page A redirected to page B, our developers and the Moz toolbar saw page A as 301 and page B as 200. However the crawl showed both page A and page B as 200.
Does anyone have any idea why the crawl may have been showing the status codes as 200? We've checked and the redirect is definitely in place for the user, but our worry is that there could be an issue with duplicate content if a crawler isn't picking up on the 301 redirect.
Thanks!
-
I have this situation on my sites too. This might not be right, but to double check that my 301s aren't being seen as duplicates of the 200s that they are redirecting to I'll filter my crawl test tool report from Moz to only show the records where the HTTP status code doesn't equal 200 and duplicate page content is yes.
I have a couple different crawlers that I use and they each work a little differently. I think some might look at your situation and return page A as a 200 just because when it resolves to page B it is a 200, where other crawlers are more sensitive and see page A as a 301.
I hope that helps!
-
Hi Gareth!
This will be hard to troubleshoot without knowing the full URLs. Could you send us the exact URLs you are working with to help@moz.com so we can take a look?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is the best way to show content in Listing Pages?
If it is e-commerce site and a product listing page there is always a conflict how to show the content? As per my understanding we can show content in two different ways. 1. To show little content and use **Read more. (**In this case there is a direct message to the google: Here is the content visible and rest content is hidden but available for visitors to read more 2. Can use** Scroll bar**. So here is the message to Google and visitors that my full content is available here. So just scroll down to read further. So I want to know that which method of showing content is best and it's impact of SEO where there is UI constraint or both the method is ok without any SEO impact. Please share your suggestions. DCdRJpH
Technical SEO | | kathiravan0 -
Increase in pages crawled per day
What does it mean when GWT abruptly jump from 15k to 30k pages crawled per day? I am used to see spikes, like 10k average and a couple of time per month 50k pages crawled. But in this case 10 days ago moved from 15k to 30k per day and it's staying there. I know it's a good sign, the crawler is crawling more pages per day, so it's picking up changes more often, but I have no idea of why is doing it, what good signals usually drive google crawler to choose to increase the number of pages crawled per day? Anyone knows?
Technical SEO | | max.favilli1 -
My Alexa ranking dropped after a 301 redirect is that bad?
I had all of my non www pages redirect to the www versions. My alexa ranking dropped and keeps dropping after I did this. I'm guessing its because its tracking the non www version. Does anyone know if this is correct and should I worry?
Technical SEO | | CandleCam0 -
Does any one have experience with SEO and .NET using 301 redirects?
A while ago I altered some of the URL's of my website. Google now thinks that I have two duplicate pages (duplicate content), I have asked my third party web developers (Who use .NET and a custom built CMS system) to simply 301 redirect the old URL to the other. However, my web developers say the following: "Solving the problems by 301 permanent re directs are out of the question as this would create infinite loops. Likely to bring down our server." They also wont do a canonical, as they say there is only one page (but two URLs) Firstly, has any one heard of this before and do they think this is true? Also, does anyone have an alternative method of getting rid of the old URL? Any thoughts would be much appreciated.
Technical SEO | | CoGri0 -
Will a timed 301 redirect work for Googlebot?
Our client is changing brand names and domain names. We know we need to 301 redirect the old domain, but for marketing reasons we want people to see a short message saying that the brand has changed and that they will be redirected. Example: | | Our concern is how, or if, Googlebot will intepret the redirect. Will this accomplish our SEO objective of moving the value of the page to the new domain, or do we need to do just a plain old fashioned 301 redirect and not even let the page load? Thanks for your help.
Technical SEO | | GOODSIR0 -
If a permanent redirect is supposed to transfer SEO from the old page to the new page, why has my domain authority been impacted?
For example, we redirected our old domain to a new one (leaving no duplicate content on the old domain) and saw a 40% decrease in domain authority. Isn't a permanent redirect supposed to transfer link authority to the place it is redirecting to? Did I do something wrong?
Technical SEO | | BlueLinkERP0 -
Looking for some help adding a 301 redirect for my Site
Hi there, I am trying to eliminate the 'www' using a 301 redirect script as I have duplicate page titles for both versions (with and without the 'www') I checked the page authority and found the pages without the 'www' to be ranked higher. For this reason I believe it would be wise to go for this option. I have an .htaccess file, all I need is the code and I should be ok 🙂 Thanks!
Technical SEO | | debeenus0 -
Too many 301 redirects - good or bad?
Hi, Currently, page A is redirecting to page B. I am in the process of developing new site for the same domain and this time page B will be redirected to page C. This is gonna happen on many pages. Is it correct or should i adopt some other strategy? Will it have adverse effect on the speed of my site? Page A -----> Page B ------> Page C Regards, Shailendra
Technical SEO | | IM_Learner0