Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 Category Page that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I have over 12,000 permanent redirects. Is this okay?
Hey guys, I have moved my website to http://www to https:/www. I use a plugin called Really Simple SSL on Wordpress. Now I have 12,000 permanent 301 redirects. It seems like my whole site has been moved. Is this the right way to do it? I am quite new to SEO and I quite don't understand if this is good or bad. From my intuition, it seems like a lot of redirects for Google bot to crawl through. Is there a better way to move to HTTPS without all these redirects? Thank you. Paul
Intermediate & Advanced SEO | | paultg0 -
How can i solve this redirect chain issue?
Hey, can somebody explain to me how can I solve this redirect chain issue? I tried redirecting /contact-us directly to **/contact/ **but this causes a chain of redirects of infinite dimensions. I don't know how to find the old redirect and delete it. Is there such an option? Thank you! byA7l
Intermediate & Advanced SEO | | Popos1 -
Should I redirect 404s or should I eliminate them?
Hello! I am now checking a website that has been migrated months ago from osCommerce to Prestashop.
Intermediate & Advanced SEO | | teconsite
While I was checking crawl errors in search console I found a lot of 404s coming from the last website. The urls are mainly 4 types: popup_image.php?pID=125&osCsid=507c27261ba5ca2568f06ce5bad2ebc9 product-friendly-url-pr-125%3FosCsid.... product-friendly-url-p-125%3FosCsid..... products_new.php?page=228 I've have realized that the parameter pId, and the number that comes after pr- and p- is the product Id in the new website, so I think our team will be able to create an script to redirect those. My question is: Is it ok to send several urls to the same url?. I mean, the popup_image.php was not the product page, as its name says it's more like a popup page. We don't have now a pop up page for images, so I was thinking to send that url to the product page. the one with the pr- was product review page the one with the p- was the product page I was thinking on redirecting the 3 of them to the product page? Should I? Or should I just redirect the last one (p-) and eliminate the others from the index? And... the ones with products_new.php?page=228 I was thinking to redirect all to the page 1 of new products. Is it ok? thank you!0 -
Https Homepage Redirect & Issue with Googlebot Access
Hi All, I have a question about Google correctly accessing a site that has a 301 redirect to https on the homepage. Here’s an overview of the situation and I’d really appreciate any insight from the community on what the issue might be: Background Info:
Intermediate & Advanced SEO | | G.Anderson
My homepage is set up as a 301 redirect to a https version of the homepage (some users log in so we need the SSL). Only 2 pages on the site are under SSL and the rest of the site is http. We switched to the SSL in July but have not seen any change in our rankings despite efforts increasing backlinks and out put of content. Even though Google has indexed the SSL page of the site, it appears that it is not linking up the SSL page with the rest of the site in its search and tracking. Why do we think this is the case? The Diagnosis: 1) When we do a Google Fetch on our http homepage, it appears that Google is only reading the 301 redirect instructions (as shown below) and is not finding its way over to the SSL page which has all the correct Page Title and meta information. <code>HTTP/1.1 301 Moved Permanently Date: Fri, 08 Nov 2013 17:26:24 GMT Server: Apache/2.2.16 (Debian) Location: https://mysite.com/ Vary: Accept-Encoding Content-Encoding: gzip Content-Length: 242 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=iso-8859-1 <title>301 Moved Permanently</title> # Moved Permanently The document has moved [here](https://mysite.com/). * * * <address>Apache/2.2.16 (Debian) Server at mysite.com</address></code> 2) When we view a list of external backlinks to our homepage, it appears that the backlinks that have been built after we switched to the SSL homepage have been separated from the backlinks built before the SSL. Even on Open Site, we are only seeing the backlinks that were achieved before we switched to the SSL and not getting to track any backlinks that have been added after the SSL switch. This leads up to believe that the new links are not adding any value to our search rankings. 3) When viewing Google Webmaster, we are receiving no information about our homepage, only all the non-https pages. I added a https account to Google Webmaster and in that version we ONLY receive the information about our homepage (and the other ssl page on the site) What Is The Problem? My concern is that we need to do something specific with our sitemap or with the 301 redirect itself in order for Google to read the whole site as one entity and receive the reporting/backlinks as one site. Again, google is indexing all of our pages but it seems to be doing so in a disjointed way that is breaking down link juice and value being built up by our SSL homepage. Can anybody help? Thank you for any advice input you might be able to offer. -Greg0 -
Redirection not working
http://elmanarah.com/ to http://www.elmanarah.com/ I have mistakenly created 5 databases for one wordpress installation.In order to get rid of them I mistakenly even deleted the right one.Now created the new one but the URL is showing with www Even now if It type in http://elmanarah.com/ it sends me to http://www.elmanarah.com/ I also check in URL D.A and P.A in OSE it shows like I have redirected it fine.Can anyone Check in and guide me either I have done it right and It pass on my previous work effort or it was total loss for me?
Intermediate & Advanced SEO | | csfarnsworth0 -
My site penalized after 301 Redirect or redesign?
Hi, I have a question regarding my site (http://www.pokeronlineitalia.com) that has, all of a sudden, lost rankings on several keywords; plus, Google Analytics and the plug-in Clicky installed on my site (the site is built on WordPress) claim that my site has no visitors/visits anymore. I would like to provide a little background of what has happened. Three weeks ago I asked my web hosting company to do a 301 redirect from http://pokeronlineitalia.com to http://www.pokeronlineitalia.com. At the same time I asked a web hosting company to to a redesign of the site. Strangely, the day after the new redesigned site went online Google Analytics and the Clicky plug-in showed that my site, from one day to the other, had no visitors/visits anymore (I had installed Google Analytics and Clicky before the 301 redirect). In addition, I noticed that I had lost positions on many keywords for which I used to rank on the second page. However, the PR of the site has remained intact and Google is indexing it without problems. Plus, I still rank high for a keyword. I tend to believe that because of this, my site was not penalized by mighty Google...but I'd like an SEO expert to tell me what he thinks about it. In particular, please answer this: has my site lost rankings because of the 301 redirect? Has my site been penalized because of the redesign? Is this only a temporary situation? Thank you very much for your help. Sal
Intermediate & Advanced SEO | | salvyy0 -
Probelms w/ 301 Redirects
I am trying to 301 redirect my old site to the new site (under the same domain. Most of the 404 urls are 301 redirects from 404 pages but they are still able to redirect. I need help understanding why I cannot redirect half of my list. <colgroup><col width="636"> <col width="342"> <col width="64"></colgroup>
Intermediate & Advanced SEO | | Melia
| OLD URL | 301 To: | Status |
| http://www.meliacaribetropical.com/en/groups.html | http://www.meliacaribetropical.com/en/groups.html | done |
| http://www.meliacaribetropical.com/entertainment/ | http://www.meliacaribetropical.com/en/services.html | done |
| http://www.meliacaribetropical.com/es/all_inclusive.html | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/es/galeria/ | http://www.meliacaribetropical.com/es/visor.html | done |
| http://www.meliacaribetropical.com/es/los-picapiedra/ | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/es/outside-us-telephone-listing.php | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/es/restaurantes/ | http://www.meliacaribetropical.com/es/gastronomia.html | done |
| http://www.meliacaribetropical.com/es/reuniones-eventos/ | http://www.meliacaribetropical.com/es/grupos.html | done |
| http://www.meliacaribetropical.com/es/spa/ | http://www.meliacaribetropical.com/es/servicios.html | done |
| http://www.meliacaribetropical.com/es/terms-condition.php | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/gallery/beach.html | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/gallery/dining.html | http://www.meliacaribetropical.com/en/gastronomy.html | done |
| http://www.meliacaribetropical.com/gallery/pools.html | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/lowest-rate.php | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/press/ | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/spanish/ | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/spanish/services/ | http://www.meliacaribetropical.com/en/services.html | done |
| http://www.meliacaribetropical.com/spanish/services/ | http://www.meliacaribetropical.com/es/servicios.html | done |
| http://www.meliacaribetropical.com/terms-condition.php | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/es/galeria/piscinas.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/galeria/playa.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/galeria/restaurantes.html | http://www.meliacaribetropical.com/es/gastronomia.html |
|
| http://www.meliacaribetropical.com/es/prensa/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/prensa/family-facilities-amenities.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/prensa/melia-caribe-tropical-announces-fall-promotion.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/prensa/melia-international-brand-overhaul.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/press/family-facilities-amenities.html | http://www.meliacaribetropical.com/en/index.html |
|
| http://www.meliacaribetropical.com/press/melia-caribe-tropical-announces-fall-promotion.html | http://www.meliacaribetropical.com/en/index.html |
|
| http://www.meliacaribetropical.com/press/melia-international-brand-overhaul.html | http://www.meliacaribetropical.com/en/index.html |
|
| http://www.meliacaribetropical.com/spanish/accommodations/ | http://www.meliacaribetropical.com/es/habitaciones.html |
|
| http://www.meliacaribetropical.com/spanish/dining/ | http://www.meliacaribetropical.com/es/gastronomia.html |
|
| http://www.meliacaribetropical.com/spanish/entertainment/ | http://www.meliacaribetropical.com/es/servicios.html |
|
| http://www.meliacaribetropical.com/spanish/events/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/flintstones/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/ | http://www.meliacaribetropical.com/es/visor.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/beach.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/dining.html | http://www.meliacaribetropical.com/es/gastronomia.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/pools.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/press/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/press/melia-caribe-tropical-announces-fall-promotion.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/press/melia-international-brand-overhaul.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/royal/ | http://www.meliacaribetropical.com/es/servicio-real.html |
|
| http://www.meliacaribetropical.com/spanish/sitemap.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/spa/ | http://www.meliacaribetropical.com/es/servicios.html |
| gastronomia.html0 -
301 redirect every pages?
Good evening, my question might sound stupid but please forgive me, I am still learning SEO. If I build a new site that will replace an existing site. Is there any point to do a 301 redirect for pages that had no inbound link so, no juice to pass? I kind of think that it would be a better practice to 301 redirect each pages to a page that make sense on the new web site .... but here is why I think that. Why I say that If I am lucky, many of my old web site pages will be indexed, many of them having no inbound links. So once the new web site online, until all my new web sites pages are indexed, I could imagine Google would send people to the index pages (the old ones that do not exist anymore)... I am right? So in that case, if I do a 301 redirect only for pages that have inbound links, the user would end up on a 404 page. Could you tell me if it make sense how I think? Thanks a lot !! Nancy P.S. I would not redirect if it make no sense to the user. I fully understand that we must always keep the user experience in mind in any 404 and 301 redirect decisions. But to simplify the question, just suppose it is ok from a user perspective to map every old site pages to a page in new web site.
Intermediate & Advanced SEO | | EnigmaSolution0