Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 CategoryPage that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
New Alternate View Redirection
Hi, We are merging two sites, differentiated by the type of customer (consumer or corporate). Currently we have:
Intermediate & Advanced SEO | | seoeuroflorist
www.consumersite.com/product/
www.corporatesite.com/product/ When on the new site, the type of customer can be switched by clicking 'Corporate' or 'Customer' which adds ?user=Business or ?user=Private to the url which then redirects so the URL is the same but certain features have changed. We block ?user=Business and ?user=Private in URLs in robots.txt to prevent duplicating pages. Should we redirect like: www.corporatesite.com/product/ -> www.consumersite.com/product/ Or: www.corporatesite.com/product/ -> www.consumersite.com/product?user=Business (this will then redirect but the parameter is blocked by robots.) I'm concerned redirecting to a URL that is blocked from indexing is an obvious error. Any ideas are welcome. Thanks!0 -
Keyword rank and redirecting
I'm creating a new amazon affiliate site. I've researched other successful sites. I've noticed that they are ranking for 1000s of keywords, but many of these long tail keywords are redirected back to a main page. I can see how this can reduce the overall total amount of content pages on the site. How are you able to rank for the keyword in the first place if the the page is redirected?
Intermediate & Advanced SEO | | lkomontt760 -
Multilingual Redirection
Hey there awesome Mozzers, I have a site that it automatically redirects people by using geolocation ( i know that probably is not good ) to the various languages of the site. I just wanted to know Is 301 or 302 the best option? ( I've heard that for language re-directions 302 is the best case scenario ) My main page for example is www.example.com and it automatically redirects with a 301 to www.example.com/en for any language that is not there. What is the best case scenario? Leave it to redirect to /en or just leave it go to the root page www.example.com.
Intermediate & Advanced SEO | | Angelos_Savvaidis0 -
301 Redirect? How to leverage the traffic on our old domain.
I've seen multiple questions about this but there's a few different answers on ways to approach it. Figured I'd personally ask for our situation. Any advice would be appreciated. We formed a new company with a new name / domain while at the same time buying an existing company in our industry. The domain and site of the company we acquired is ranking for some valuable keywords and still getting a significant amount of traffic (about half of what our new site is getting). A big downside has been, when they moved that site to a different server, something happened to where the site became uneducable so it's full of bad pricing and information. Because of that, we've had a maintenance page up for a little bit because it was generating calls to our sales team (GOOD) but the customer was having seen incredibly incorrect information (BAD) Rather than correcting those issues or figuring out why the site is un-editable, we just want to find a way where we can leverage that traffic and have them end up at our new site. Would we 301 redirect the entire domain to our new one? If we did that would the old domain still keep the majority of it's page rank?
Intermediate & Advanced SEO | | HuskyCargo1 -
302 redirects in the sitemap?
My website uses a prefix at the end to instruct the back-end about visitor details. The setup is similar to this site - http://sanfrancisco.giants.mlb.com/index.jsp?c_id=sf with a 302 redirect from the normal link to the one with additional info and a canonical tag on the actual URL without the extra info ((the normal one here being http://sanfrancisco.giants.mlb.com,) However, when I used www.xml-sitemaps.com to create a sitemap they did so using the URLs with the extra info on the links... what should I do to create a sitemap using the normal URLs (which are the ones I want to be promoting)
Intermediate & Advanced SEO | | theLotter0 -
301 redirections done too late - What do you suggest?
Hi, When pushing our new site live, most of the 301 redirections got done too late for several reasons. Understandably, our site rankings in google have taken a hit now. So far we have just tried to perfectly optimize the pages that used to rank well (They weren't even optimized before and were still ranking) , to get our positions back. But does anyone have an idea about what else we could do? Is there a recommended "action plan" when someone is late with their 301 redirections?
Intermediate & Advanced SEO | | JohanMattisson0 -
SEO friendly way to redirect users based on IP address
Hi There I have an issue where I need to redirect all visitors from a specific country to a sub-domain, which has content explaining to those users that our service is not offered in that country. I would like to know the most SEO friendly way to do this? Thanks Sadie
Intermediate & Advanced SEO | | dancape0 -
My site penalized after 301 Redirect or redesign?
Hi, I have a question regarding my site (http://www.pokeronlineitalia.com) that has, all of a sudden, lost rankings on several keywords; plus, Google Analytics and the plug-in Clicky installed on my site (the site is built on WordPress) claim that my site has no visitors/visits anymore. I would like to provide a little background of what has happened. Three weeks ago I asked my web hosting company to do a 301 redirect from http://pokeronlineitalia.com to http://www.pokeronlineitalia.com. At the same time I asked a web hosting company to to a redesign of the site. Strangely, the day after the new redesigned site went online Google Analytics and the Clicky plug-in showed that my site, from one day to the other, had no visitors/visits anymore (I had installed Google Analytics and Clicky before the 301 redirect). In addition, I noticed that I had lost positions on many keywords for which I used to rank on the second page. However, the PR of the site has remained intact and Google is indexing it without problems. Plus, I still rank high for a keyword. I tend to believe that because of this, my site was not penalized by mighty Google...but I'd like an SEO expert to tell me what he thinks about it. In particular, please answer this: has my site lost rankings because of the 301 redirect? Has my site been penalized because of the redesign? Is this only a temporary situation? Thank you very much for your help. Sal
Intermediate & Advanced SEO | | salvyy0