Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 Category Page that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Googlebot on steroids... Why?
We launched a new website (www.gelderlandgroep.com). The site contains 500 pages, but some pages (like https://www.gelderlandgroep.com/collectie/) contains filters (so there are a lot possible url parameters). Last week we mentioned a tremendous amount of traffic (25 GB!!) and CPU usage on the server. 2017-12-04 16:11:57 W3SVC66 IIS14 83.219.93.171 GET /collectie model=6511,6901,7780,7830,2105-illusion&ontwerper=henk-vos,foklab 443 - 66.249.76.153 HTTP/1.1 Mozilla/5.0+(Linux;+Android+6.0.1;+Nexus+5X+Build/MMB29P)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/41.0.2272.96+Mobile+Safari/537.36+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - - www.gelderlandgroep.com 200 0 0 9445 501 312 We find out that "Googlebot" was firing many, many requests. At first we did a nslookup for the IPadres where it actually seems to be googlebot. Second we visited Google Searchconsole and I was really surprised... Googlebot on steroids? Googlebot requested 922.565 different url's and made combinations for every filter/ parameter combination on the site. Why? The sitemap.xml contains 500 url's... The authority of the site isn't very high, no other signal that this is a special website... Why so much "Google resources"? Of course we will exclude the parameters in SearchConsole, but I never saw a Googlebot activity for a small website like this before! Does anybody have any clue? Regards Olaf searchconsole.png nslookup.png
Intermediate & Advanced SEO | | Olaf0 -
If you do 302 redirect then change to 301 redirect do you lose all link juice?
Hello everyone, I was wondering if you could help me with understanding the following story: A website has been moved from its HTTP version to a HTTPS version. The SEO manager has advised developers that they needed to do 301 redirects. However, in the end, 302 redirects have been put in place instead. Now, 301s should be put in place ASAP. The million dollar question is: has the website lost all of its link juice already given the nature of the redirects? Also, does it depend on whether Google has indexed the new 302 pages or does it depend on something else? Many thanks!
Intermediate & Advanced SEO | | MarketingGH0 -
Directories that Redirect - Do They Pass Link Juice?
I did some searching before asking but could not quite find what I was looking for. There are valid directories out there that provide business as well as links that provide SEO value. My question is whether or not having a redirect in place negates passing any link juice. When I use Open Site Explorer for Old Monterey Inn, this directory (CABBI) does not show up on their list. However, their website dropped from Google Analytics altogether for some time because of an issue in how they built their site. Their "fix" is this redirect which was integrated a short time ago. I do see traffic in Google Analytics now but wonder about the link juice. Example: <a href="[/redirect?type=website&inn=34211&url=http%3A%2F%2Fwww.oldmontereyinn.com](view-source:https://www.cabbi.com/redirect?type=website&inn=34211&url=http%3A%2F%2Fwww.oldmontereyinn.com)" target="<a class="attribute-value">_blank</a>">www.oldmontereyinn.coma>p> What say you? Thanks to anyone that responds.
Intermediate & Advanced SEO | | ColoradoMarketingTeam0 -
Page A Best for Users, but B Ranks
This is real estate MLS listings related. I have a page "B" with lots of unique content (MLS thumbnails mixed with guide overview writing, pictures etc) which outranks "A" which is a page simply showing MLS thumbnails with map feature included. I am linking from "B" to "A" with anchor "KEYWORD for sale" to indicate to search engines that "A" is the page I want to rank, even though "B" has more unique content. It hasn't worked so far.
Intermediate & Advanced SEO | | khi5
Questions: Should I avoid linking from "B" to "A" as that could impact how well "B" ranks? Should I leave this setup and over time hope search engines will give "A" a chance to rank? Include some unique content on "A" mostly not viewable without clicking "Read more" link? I don't foresee many users will click "Read more" as they are really just looking for the properties for sale and do rarely care about written material when searching for "KEYWORD for sale". Should I "no index, follow" A as there are limited to none unique content and this could enhance chance of ranking better for B? When I write blog posts and it includes "KEYWORD for sale" should I link to "A" (best for users) or link to "B" since that page has more potential to rank really well and still is fairly good for users? Ranking for "B" is not creating a large bounce rate, just that "A" is even better. Thank you,
Kristian0 -
Redirects in .htaccess file
I am using yoast seo plugin for wordpress. I have done several link changes and placed approximately 80-100 301 redirect into .htaccsses file. Is this good practice? Do they have to be there forever? Is it not slowing down site? Or should this be done in different way? Thank you very much Iris
Intermediate & Advanced SEO | | Rebeca10 -
How long for a 301 redirect to pass PR?
Hi, How long does it take for a 301 redirect to pass PR/Juice to the new domain it's redirecting to? From what I understand you tell Google in Webmaster tools this domain is now going to this domain and then setup a file on the old domains hosting to redirect to the new. And that's it! If that is correct how long does it take? 2 days, 2 weeks, months, maybe never??? Cheers
Intermediate & Advanced SEO | | activitysuper0 -
Are there any negatives to channeling my links through a 301 redirect?
I'm channeling 1000's of links through another url with a 301 redirect. I've thought this through and can't see any downside to doing this, but I want to get your opinion. Can you see any downside to doing this? With regards to passing anchor text, PR, PA, etc? Since this is done with sites all the time when they change urls, I can't see Google being able to penalize me for this....can you? What do you think?
Intermediate & Advanced SEO | | Brian-M0 -
How To 301 Redirect .html pages
I need to redirect a page/URL that is purely .html to a new location. I don't know how to do this. All the redirects I can find are for server side code pages .php/.aspx etc. From my understanding I can't put a server side redirect in a .html file. I am hosting on a microsoft server, however the new page I am redirecting to is .php. I am running some WordPress (.php) files on the server. I need to make it redirect before the old page loads so visitors don't start reading something that is about to get redirected Can someone please help me?
Intermediate & Advanced SEO | | MyNet0