Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 Category Page that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Advice on redirects for a category I want to reuse.
Hi, We have a current category set up that is starting to rank OK but we are going through a site re-build and this category URL will now better describe a new category of products. My dilemma is if I 301 redirect the current url to my new category I won't be able to use the URL for the new one. But if I don't redirect it will the pages that have already been ranked under this url then confuse customers and search engines. For example - Products and sub-categories under the URL /personalised-toys will now become /personalised-toys-for-boys but I want to use the /personalised-toys URL for a different set of sub categories and products. Any assistance or ideas or just definitely don't do it in a particular way would be greatly appreciated
Intermediate & Advanced SEO | | neil_stickergizmo0 -
Ranking after redirecting two URLs to a new domain
I run two websites which operate in similar business sectors. Each has a calculator tool that offers the same functionality. The pages rank 2nd and 5th for the key search term. I'd like to improve the functionality of this and have thought about setting up a new domain for this calculator to move it away from the main sites. If I did this and 301 redirected both pages to the new domain do you think I'd maintain a strong ranking position for this search term on the new domain? Thanks for any advice.
Intermediate & Advanced SEO | | craigramsay0 -
What are the best practices with website redesign & redirects?
I have a website that is not very pretty but has great rankings. I want to redesign the website and loose as little rankings as possible and still clean up the navigation. What are the best practices? Thanks in advance.
Intermediate & Advanced SEO | | JHSpecialty0 -
Multiple 301 Redirect Query
Hello all, I have 2 301 redirects on my some of my landing pages and wondering if this will cause me serious issues. I first did 301 directs across the whole website as we redid our url structure a couple of months ago. We also has location specific landing pages on our categories but due to thin/duplicate content , we have got rid of these by doing 301's back to the main category pages. We do have physical branches at these locations but given that we didnt get much traffic for those specific categories at those locations and the fact that we cannot write thousands of pages of unique content content , we did 301's. Is this going to cause me issues. I would have thought that 301's drop out of serps ? so is this is an issue than it would only be a temporary one ?.. Or should I have 404'd the location category pages instead. Any advice greatly appreciated. thanks Peter
Intermediate & Advanced SEO | | PeteC120 -
301 Redirect from unused domain
Hi All First question here so go easy.. I have a property site which is working well so far considering it;s early days, unfortunately some of my earlier efforts did not go so well and one in particular I pretty much destroyed in my attempts to improve the site SEO. Lucky enough my SEO skills have improved quite a bit lately, largely thanks to the great tools, tutorials and experts here at Moz 🙂 My question is whether I can use a 301 redirect to pass the domain authority and any link equity from an unused site to the one that ive done a better job on? it would seem a little sketchy to me and I would prefer not to get slapped and penalized "again" for doing something dodgy... Thanks everyone and thanks for all the help over the last 6 months or so.. Wes Dunn
Intermediate & Advanced SEO | | wesdunn19771 -
DNS or 301 Website Redirect
We are running a marketplace site, so we have thousands of vendors selling their products on our site. Each vendor has a Profile page and we are soon to launch a premium store-front that is white label. Many of these vendors will want to point a custom url to their premium store-front (which is a sub domain of the marketplace) and we are trying to get an understanding of how we should instruct them to point their url in a way that will give the main marketplace site the seo juice. We also want to understand what will show up in the address bar. Will it be their url or our sub domain? Will any of the marketplace seo juice boost their url local listing status?
Intermediate & Advanced SEO | | bloomnation0 -
301 redirect for ip address in SERPs
Hi, I've recently had the misfortune of my site's ip address being crawled and indexed by Google, which is causing some duplicate content issues. Due to the nature of the site we're not able to implement a canonical tag to fix this at present. Would a 301 redirect do the trick, and if so, could someone point me to what I'd need to add to our .htaccess file? Many thanks Chris
Intermediate & Advanced SEO | | ChrisHillfd0 -
Fetch as Googlebot
"With Fetch as Googlebot you can see exactly how a page appears to Google" I have verified the site and clicked on Fetch button. But how can i "see exactly how a page appears to Google" Thanks
Intermediate & Advanced SEO | | seoug_20050