Pages with 301 redirects showing as 200 when crawled using RogerBot
-
Hi guys,
I recently did an audit for a client and ran a crawl on the site using RogerBot. We quickly noticed that all but one page was showing as status code 200, but we knew that there were a lot of 301 redirects in place. When our developers checked it, they saw the pages as 301s, as did the Moz toolbar.
If page A redirected to page B, our developers and the Moz toolbar saw page A as 301 and page B as 200. However the crawl showed both page A and page B as 200.
Does anyone have any idea why the crawl may have been showing the status codes as 200? We've checked and the redirect is definitely in place for the user, but our worry is that there could be an issue with duplicate content if a crawler isn't picking up on the 301 redirect.
Thanks!
-
I have this situation on my sites too. This might not be right, but to double check that my 301s aren't being seen as duplicates of the 200s that they are redirecting to I'll filter my crawl test tool report from Moz to only show the records where the HTTP status code doesn't equal 200 and duplicate page content is yes.
I have a couple different crawlers that I use and they each work a little differently. I think some might look at your situation and return page A as a 200 just because when it resolves to page B it is a 200, where other crawlers are more sensitive and see page A as a 301.
I hope that helps!
-
Hi Gareth!
This will be hard to troubleshoot without knowing the full URLs. Could you send us the exact URLs you are working with to help@moz.com so we can take a look?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moving Blog and 301 Redirect Advice
Hello Moz Community, We recently moved our blog from its own domain to a directory on our website. We do not plan on moving over all the old blog posts because a majority most of them are based on events or time-sensitive information that has passed. We need advice on what to do with all of the old blog URL's? Should we just 301 all of them to the new blog directory on our website (www.domain.com/blog)? Should we take the time to move over all the old blog content and put the appropriate 301's in place? Any and all advice would be greatly appreciated. Thank you in advance.
Technical SEO | | All-Star-Vacation-Homes
Best,
Rich0 -
301 Redirects on Large Real Estate Website
Hi guys,We are about to move over to a new website and need advice on handling the 301 redirects.We have a large real estate website with around 12,000 pages, a lot of these are properties (about 10,000)On our old website, the url structure for each property is as follows -domainname.com/property/view?property=14863on our new site, the url structure isdomainname.com/properties/view/6137The property ID number is always different from old site to new. The way we see it, we have two options. a.) a manual redirect of each and every property url. A very very long jobb.) a folder level redirect, so redirect the 'property' folder on the old site into the 'properties' folder on new. The con with this one is we are not sure if this is the best route to take, if it is how we would go about it?Some advice would be really appreciated guys. I know there are some hyper intelligent SEO's in here and we need to make sure we handle this right!Many thanks in advance.Mark
Technical SEO | | Nextman0 -
301 redirects and seo..
I bought a domain and it has nice traffic. It only has about 5 main pages in php When i got the site i switched to html because php was overkill. I did the 301 and google deleted the php files and replaced with html version when i check site:domain.com It has been about 7 days. I DID NOT use 301 for each of the 5 pages to go php to html instead is used this code RewriteEngine On
Technical SEO | | samerk
RewriteCond %{HTTP_HOST} ^mydomain.com
RewriteRule (.) http://www.mydomain.com/$1 [R=301,L]
RedirectMatch 301 (.).php$ http://www.mydomain.com$1.html So basically if you load php it will load the html version. dog.php > dog.html Is this OKAY? or should it be done differently.... worried! Thanks !0 -
301 redirects
At the moment it's possible to access the home page of my website via two different urls, with and without www. and you've told me that this can be resolved with Canonicalization and a 301 redirect. Do I do this with my web hosting package or in my html pages? If I can't do it with my web host (1&1) then is there an idiot's guide of how to do it yourself? I've also got both the domain vamospaella.co.uk and vamospaella.com. Is it better to have one of these redirecting to the other for UK traffic (at the moment .co.uk redirects to .com) Thanks
Technical SEO | | melissa10 -
Not sure which URL to use for 301 redirect
A client has new website design completed by another developer, was launched in April of this year. No 301 redirect was set up so duplicate content is an issue. Client has had a website with same domain name for about 10 years, but has not had any SEO work completed before or since his new site design. For non-www there are 6 referring links - 1 considered to have authority, for www there are also 6 but 3 considered to have authority. More links seem to coming from www than non-www. But for one of the clients keywords they are ranked #1 for their area and that links to their non-www address. And even though no redirects set up by developer, non-www has had far more visits according to Google Analytics. So many basics that still need to be done for site: no meta-descriptions on any page, H1 and page titles could use keywords, call to action moved above fold, etc. Considering this is a new site, and new SEO work and many more inbound links needed, does it matter which address I redirect to? _Cindy Barnard
Technical SEO | | CeCeBar0 -
Canonical tag used on several pages?
Is it a bad idea to use rel=canonical from several pages back to one (if you are planning on no-indexing them)? Does this concentrate the “link juice” from those several pages back to one?
Technical SEO | | nicole.healthline0 -
Joomla 301 redirects
hi i am using joomla sef404 and i have 7000 not found pages in my webmaster google tool page. the trouble is i got rid of a lot of pages and also components so now i am left with loads of not found pages. what i want to try and do is to create 301 redirects so that i do not lose page rank. can anyone please let me know how to do this step by step please
Technical SEO | | ClaireH-1848860 -
301 redirects inside sitemaps
I am in the process of trying to get google to follow a large number of old links on site A to site B. Currently I have 301 redirects as well a cross domain canonical tags in place. My issue is that Google is not following the links from site A to site B since the links no longer exist in site A. I went ahead and added the old links from site A into site A's sitemap. Unfortunately Google is returning this message inside webmaster tools: When we tested a sample of URLs from your Sitemap, we found that some URLs redirect to other locations. We recommend that your Sitemap contain URLs that point to the final destination (the redirect target) instead of redirecting to another URL. However I do not understand how adding the redirected links from site B to the sitemap in site A will remove the old links. Obviously Google can see the 301 redirect and the canonical tag but this isn't defined in the sitemap as a direct correlation between site A and B. Am I missing something here?
Technical SEO | | jmsobe0