Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 CategoryPage that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Is Indexing my 301 Redirects to Other sites
Long story but now i have a few links from my site 301 redirecting to youtube videos or eCommerce stores. They carry a considerable amount of traffic that i benefit from so i can't take them down, and that traffic is people from other websites, so basically i have backlinks from places that i don't own, to my redirect urls (Ex. http://example.com/redirect) My problem is that google is indexing them and doesn't let them go, i have tried blocking that url from robots.txt but google is still indexing it uncrawled, i have also tried allowing google to crawl it and adding noindex from robots.txt, i have tried removing it from GWT but it pops back again after a few days. Any ideas? Thanks!
Intermediate & Advanced SEO | | cuarto7150 -
How many redirects are too many?
Hello Everyone, I currently have a dynamic site and it is my understanding that switching to a static site would be beneficial. I already have some 301's in place from when my site had a .php extension to the new extension now with ./?... etc. Is it okay to re redirect them? How many redirects are too many? Thank you in advance for suggestions. Have a Fabulous Friday! Sandra
Intermediate & Advanced SEO | | rankmenow0 -
301 Redirection and apostrophes in URLs
Hi I am experiencing trouble getting any redirects with apostrophes in the URLs to 301 redirect in order to eliminate 404 errors. I have tried replacing the instance of the apostrophe in the source URL field to %27 and variations of this but to no avail. The site is a wordpress site (the old URLS are legacies from the old Business Catalyst site) and I am using the redirection plug in. I have gone into some detail with a helpful soul here http://wordpress.org/support/topic/how-to-deal-with-apostrophes-in-source-url but unfortunately to no result. If anyone has any idea how to solve this puzzle I would be grateful for the help. Example: http://www.tesselaars.com/blog/Inside_Flowers/post/Online_Marketing_for_Florists_Part_1%E2%80%93_A_Website_You_Won%27t_Regret/
Intermediate & Advanced SEO | | Seamoose0 -
Big 301 Redirect Help!
Hey guys I need a little help with setting up a big 301. Background: It's a bit of a mess as the old site is a total mess after being online for 10 years plus. It has html and php pages, and a mod rewrite to redirect old html links to the newer php version of those pages. It's now moving to a new site and as the domain name and URL structure has changed we can't use any fancy regex and have to do a page to page redirect. There are 1500 pages to redirect. However, the old site has thousands of linking root domains, and some of these are to the old html pages (which currently redirect to the php pages) and some to the newer php pages. Question: My initial plan was to leave the mod rewrite and only redirect the php pages. That means 1500 individual redirects instead of 3000 if I individually redirect both the php and html pages. I'm not sure what's best to be honest. We don't really want multiple hops in the redirect (html>php>new site), but surely 1500 redirects is better than 3000! Does anyone have any advice on which option may be best, or even a better option? Thanks 🙂
Intermediate & Advanced SEO | | HarveyP0 -
301 redirect rule
Hi there, I have a website that has hundreds of links with a "question mark" at the end of URLs. For example: http://www.domain.com/directory/page.html?
Intermediate & Advanced SEO | | iQandil
http://www.domain.com/directory/another-directory?
http://www.domain.com/directory/yet-another-directory/? I'm want to place a wildcard redirect on the .htaccess file but don't know what exactly to add. Ideally I want the URLs above to be: http://www.domain.com/directory/page.html
http://www.domain.com/directory/another-directory/
http://www.domain.com/directory/yet-another-directory/ Any help is most appreciated. Thanks
Issa0 -
Duplicate titles but redirecting anyway (without redirects set up!!!)
Google has done a crawl of my site and is flagging up duplicate titles on my wordpress site. This appears to be due to the face that some posts are tagged in more than one category. I have just gone to make sure that each post just has one category and add redirects and I've noticed that all the duplicate title issues google has notified me about appear to redirect anyway. For example: http://www.musicliveuk.com/latest-news/live-music-boosts-australian-economy and http://www.musicliveuk.com/live-music/live-music-boosts-australian-economy have duplicate titles apparantly but the 1st url redirects to the 2nd one. I use the redirection plug in but have no redirection set up for that url so I'm a bit confused. And if they're redirecting anyway then why is google flagging up duplicate titles? Any help would be much appreciated.
Intermediate & Advanced SEO | | SamCUK1 -
How long should a domain redirect take?
Hi, I know that this is a 'How long is a piece of string?' type question but at what point should the ranking value of site A pass over to site B following a domain 301 redirect? I have shifted a domain over to a new URL, same hosting server, same IP address. I haven't made any URL changes or any content changes other than to change the site logo to match the new domain name. Domain B is basically an exact clone of domain A. I have redirected Domain A to domain B using the following line at the top of the .htaccess file:- Redirect 301 / http://www.newdomain.com/ I have submitted a sitemap for the new domain via google webmaster tools. It looks like the original domain as been completely indexed by google following the redirect as all rankings have been dropped from the results and there are no results for a site:olddomain.com search. Surely the rankings should have switched over at this point? Any help would be much appreciated.
Intermediate & Advanced SEO | | AdeLewis
Ade.0 -
How can I create a redirection on IIS?
Hi, I want to know how to create a server side 301 redirection on IIS7. I found on google the below details; IIS Redirect In internet services manager, right click on the file or folder you wish to redirect Select the radio titled "a redirection to a URL". Enter the redirection page Check "The exact url entered above" and the "A permanent redirection for this resource" Click on 'Apply' But couldn't find anything like that. Can anyone help me out. thanks & regards
Intermediate & Advanced SEO | | VipinLouka780