Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 CategoryPage that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I have over 12,000 permanent redirects. Is this okay?
Hey guys, I have moved my website to http://www to https:/www. I use a plugin called Really Simple SSL on Wordpress. Now I have 12,000 permanent 301 redirects. It seems like my whole site has been moved. Is this the right way to do it? I am quite new to SEO and I quite don't understand if this is good or bad. From my intuition, it seems like a lot of redirects for Google bot to crawl through. Is there a better way to move to HTTPS without all these redirects? Thank you. Paul
Intermediate & Advanced SEO | | paultg0 -
Duplicate without user-selected canonical excluded
We have pdf files uploaded in the media of wordpress and used in our website. As these pdfs are duplicate content of the original publishers, we have marked links to these pdf urls as nofollow. These pages are also disallowed in robots.txt Now, Google Search Console has shown these pages Excluded as "Duplicate without user-selected canonical" As it comes out we cannot use canonical tag with pdf pages so as to point to the original pdf source If we embed a pdf viewer in our website and fetch the pdfs by passing the urls of the original publisher, would the pdfs be still read as text by google and again create duplicate content issue? Another thing, when the pdf expires and is removed, it would lead to 404 error. If we direct our users to the third party website, then it would add up to our bounce rate. What should be the appropriate way to handle duplicate pdfs? Thanks
Intermediate & Advanced SEO | | dailynaukri1 -
Is it possible to avoid redirect of penalties for 301 forwards?
We have been doing a good amount of competitive research lately and have noticed sites that have been changing their TTLD quite often to escape manual penalties / DCMA filings. An example evolution: brandterm.com -> brandterm.bz -> brandterm.me These competitors are able to quickly rank for money keywords in the top 3 soon after another domain switch. What we have noticed is that while its obvious they received Google penalties they continue to 301 redirect the old domains to the new ones. We have experienced first hand that penalties travel along domains with 301 redirects. Does anyone have an explanation how these companies are able to achieve quickly high volume of organic search while 301-redirecting from burnt domains? The only option I see is to disavow all previous domains in GWT to be able to employ 301 redirects without risking carrying over the penalty. Are there other theories ppl can think of? T
Intermediate & Advanced SEO | | petersocapro0 -
SEO for a redirected domain name
Our client is a law firm with a name that is challenging to spell. We have procured a domain name for them that is catchy, easy to spell, and plays well into their brand, or at least the current campaign. We're using the campaign domain to direct traffic to their website with a 301 redirect. We have placed the campaign domain in a variety of offline mediums including print and outdoor. The client is currently in the number 1 spot for a good number of our highest priority keywords, so I do not want to do anything to jeopardize that. I'm also not sure this campaign will be their "brand" long-term so I don't want to risk making a switch and making it back. So for now, I'm most comfortable leaving the campaign domain as a redirect to their primary domain. Recently, the client approached me complaining (legitimately) that when people google the campaign domain, they are brought to search results for an entirely different domain because Google "corrects" the domain name for them. This is obviously a bad thing, with many users defaulting to entering urls into Google instead of the address bar. If you tell Google that it was wrong about the autocorrection, our site is in the number 1 position. I liken the situation to Overstock.com using O.co as their offline domain, but overstock.com as their online domain. But imagine if you googled o.co and google brought you to a list of results for "on.co" because it assumed you fat-fingered it. Is there anything I can do to prevent the domain name from getting corrected by Google?
Intermediate & Advanced SEO | | steverobinson0 -
Homepage 302 redirect - Which option makes most sense?
Edit: Here's a simplified version of this issue and how I have fixed it thus far. On AmbitionSnowskates.com, there is a video section. There is no content on this page, it 302 redirects to the newest video. If you access ambitionsnowskates.com/video/, you are redirected to ambitionsnowskates.com/video/safari-time/. The original post (OP) was about a site with recurring events. There is a cycle between to which subpage the homepage should redirect. For this reason, I was wondering if I should redirect mysite.com to mysite.com/active-subpage/ or the other way around (and have the content directly on the homepage). I was also wondering how this will affect the result in SERPs. It turns out Google shows the title and description of the destination page, but shows the URL of the original URL (the homepage). Knowing this, I can tailor my meta descriptions to be about both the company and the current event; a mix of the two means I won't have to switch or duplicate meta descriptions between active events. I do appreciate the real solution though: in my opinion there should be unique content on the homepage with according CTA. I'm trying to push this as the best fix, with redirections being an alternative, but albeit more complex, solution. Again sorry for being so unclear. I wish I had had an example from the beginning. 🙂 I'm leaving this opened in case someone wants to chime in. Ben Hey guys, I need a hand on this one 🙂 We have a website with 3 events and we want the homepage to show the upcoming event. Event 1 is in February Event 2 is in April Event 3 is in June These events are recurrent year after year. Currently the homepage shows the content of event 1 at the root level (site.com/) . The other events have a unique URL (site.com/event-2, site.com/event-3). Later in the year, after event 1 is over, we change the homepage content to event 2 and move event 1 to its own URL. In other words... Current structure Today: Event 1: site.com/ Event 2: site.com/event-2 Event 3: site.com/event-3 In March: Event 1: site.com/event-1 Event 2: site.com/ Event 3: site.com/event-3 And so on. I want to make sure each event has its own URL and is properly indexed. Option A I can redirect the homepage to the right event: site.com -> 302 -> site.com/event-1. If that's the way to go, what will be the SEO impact, i.e. what content will show up in SERPs? The destination page's content/meta description and title? Option B What I could also do is keep the current structure (content moved to the root), but redirect temporarily the event's unique URL to the homepage: Today: Event 1: site.com/ Event 1: site.com/event-1 -> 302 -> site.com/ Event 2: site.com/event-2 Event 3: site.com/event-3 In March: Event 1: site.com/event-1 Event 2: site.com/ Event 2: site.com/event-2 -> 302 -> site.com/ Event 3: site.com/event-3 And so on. Again, if that's the way to go, how will this impact SERPs, which title and description will I see for the homepage and individual events? If you have other options, I'm all ears! Thanks a lot! (I mean it) -Ben
Intermediate & Advanced SEO | | BenoitQuimper0 -
How many pages to 301 Redirect
Hi Mozzers, My site has 11,200 pages indexed in Google and I'm looking to remove some of the lesser content which should probably have been picked up by Panda. However these pages work out to about 1,100 in total and I'm not sure whether to remove these bit by bit or just do it in one fell swoop? Does Google not like a site's indexed pages fluctuating too quickly? Are there any other considerations I should be aware of? Thanks!
Intermediate & Advanced SEO | | panini0 -
301 redirect for duplicate content
Hey, I have just started working on a site which is a video based city guide, with promotional videos for restaurants, bars, activities,etc. The first thing that I have noticed is that every video on the site has two possible urls:- http://www.domain.com/venue.php?url=rosemarino
Intermediate & Advanced SEO | | AdeLewis
http://www.domain.com/venue/rosemarino I know that I can write a .htaccess line to redirect one to the other:- redirect 301 /venue.php?url=rosemarino http://www.domain.com/venue/rosemarino but this would involve creating a .htaccess line for every video on the site and new videos that get added may get missed. Does anyone know a way of creating a rule to rewrite these urls? Any help would be most gratefully received. Thanks. Ade.0 -
Regarding 301 Redirect!
Hello, I heard that 301 redirect can be good for newly registered domain names can i buy a old domain name and put 301 redirect on it to my newly registered niche market domain name. Shall i buy only 1 domain name and put 301 redirect to my newly registered domain names or i can do this for more than 1 old domains i purchased?
Intermediate & Advanced SEO | | anand20100