I have a site that has a 302 redirect loop on the home page (www.oncologynurseadvisor.com) i
-
i am trying to do an audit on it using screaming frog and the 302 stops it. My dev team says it is to discourage Non Human Traffic and that the bots will not see it. Is there any way around this or what can I tell the dev team that shows them it is not working as they state.
-
As I said use robots.txt or htaccess rules for bots, where you would be able to allow access to specific bots like seofrog (at least temporary). Or, maybe they can allow seofrog to crawl it temporarily or something.
As for Google bot - try to "fetch as google" within Google Search console. See what response there is.
-
Thank you.
In response to your question, they want to block any Non Human Traffic. This is another site for a single sign on. So that when someone signs in to one site, they are signed in to all of the sites that we offer. They have said that the bots doe not see the redirect, yet I am trying to crawl it with Screaming Frog and SEO Powersuite that will crawl a site as if it were Googlebot and I still get the same response and both tools go no further unless i ask them to crawl the orphan pages.
-
Howdy.
This is very strange. So, does your dev team block all bots? How about search engine bots? 0.o
Anyway, the 302 redirect is not the way to go. If you'd like to manage bot access, use robots.txt, htaccess rules or simply filter it out afterwords in reporting tools.
Hope this helps
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site structure and Visual Sitemaps
Aside from mind mapping software are there any tools ( recommended) to build a visual sitemap of the internal linking structure of a URL? I've been trying to 'show' clients the structure of a website as it pertains to internal and external links. Here is one I've tried it's "Close" - http://site-visualizer.com/ . I've been using the excel export function, import into mind meister and building it. It's a teeny bit time consuming for large websites. Site structure I feel is a valuable portion of SEO and a down and dirty visual explanation would be great. Don't get me wrong, it offers other benefits as well- it's just I'd like to free up the time it takes. Thank you in advance. Screen shots are available on the website of the organization.
Web Design | | TammyWood0 -
Existing URL structure and how to handle new pages before migration
Hi there! Currently, our site uses underscores "_" within the url structure. We are moving to Wordpress soon (the site is currently static html) but it will be a couple of months before the migration. Here is an example of the current structure: www.oldsitestructure.com/about_us/success_stories/custom_vinyl_banners When we do change, our url structure will have hyphen's "-" to separate terms, so the preferred new structure will be: www.oldsitestructure.com/about-us/success-stories/custom-vinyl-banners The entire site (with the exception of our Wordpress blog) currently uses the old structure. We have about 10 - 15 pages we will add before our migration, my question is: Should we use the preferred url structure starting NOW or stick with the old one? And set up 301 redirects are part of the migration process? Many thanks!
Web Design | | SEOSponge
Jon0 -
Site Speed for Magento Site
We have our site all ready for test users but the site speed is painfully slow. Our developers are a bit stymied about what to do. Is anyone a site speed expert or know of anyone good that can evaluate our site and tell us what's going wrong? Thanks!
Web Design | | EcomLkwd1 -
How do we get search engine bots to the item detail pages?
The problem we have is that we have lots of inventory pages. These inventory pages have a bunch of links at the top linking to different styles of the item, up to 56 links in some cases. Then each item listed has a link to the item's detail page and a link to the item's shop owner's page. So if a page has 50 items shown, there are really 100 links just for the inventory. This is not taking into account the header links, footer links, sidebar links to other sections on the site. We have all these links to help consumers move through the site. The problem is that every item detail page on the site is not getting indexed and actually I think it's more like over 50% of the item detail pages are not indexed because the search engines are too busy following all these other links. Should we nofollow, index the links to the different styles of the item, the shop owner page? Or what should we do to get the search engine bots to index our item detail pages?
Web Design | | CFSSEO0 -
Why do site links appear under one keyword and not another? Any ideas?
Hi everyone, I have a client whose website is doing the strangest thing. When I search the branded keyword (the company name), Google doesn't show any site links under the result. However, when I search for the company name plus Inc., I do get site links. Now, the website is ranking first in both searches, so that's not the issue. And, as near as I can tell, the site only contains one or two uses of the company name plus the word "inc." Most of the text on the page and all of the meta data only uses the company name, and most of the links that connect to the site use only the company name as well. Even the Who Is for the site doesn't use the term "inc." And ideas what might be going on? I know Google says that the process is still automated but for the life of me I can't figure out what kind of automated process would result in these results. Thanks! Megan (Rebecca's minion)
Web Design | | RebeccaRalston0 -
International SEO issues for multiple sites
We currently have 3 websites: oursite.co.uk oursite.fr oursite.ch We also own Oursite.com, and that URL currently redirects to Oursite.fr. We are considering a complete site redesign and a possible merge of the 3 sites. Assumptions: ** the 3 sites currently receive organic search traffic to varying degrees
Web Design | | darkgreenguy
** Oursite.ch is almost identical to Oursite.fr in terms of the site content
** Our target market is NOT the USA for English-language searches. It is the UK. With a re-design, we see our options as follows: Merge the 3 sites and make Oursite.com the "main site" and then have subfolders as follows: /uk /fr /ch Keep the 3 sites as they are. We see Option 1 as the best in terms of saving time when updating the site, and saving money paid to the site developers (1 site vs 3 sites). We see Option 2 as the best in terms of ability of the site to rank, as well as confidence of searchers when seeing our site in the search results (in other words, a person searching in France would be more likely to buy and/or submit a form on our site if they saw Oursite.fr vs Oursite.com/fr). I guess we're looking for some suggestions/guidance here. Are we missing any big issues? Does anyone have experience with an issue such as this? Thank you in advance...
-Shawn0 -
What is the optimal URL Structure for Internal Pages
Is it more SEO friendly to have an internal page URL structure that reads like www.smithlawfirm.com/personal-injury/car-accidents or www.smithlawfirm.com/personal-injury-car-accidents? The former structure has the benefit of showing Google all the sub-categories under personal injury; the later the benefit of a flatter structure. Thanks
Web Design | | rarbel0 -
Redirecting 301 Redirects -- Will Search Engines Notice?
Hello Mozzers, We're currently evaluating a client site where the previous web developer redesigned the site and got lazy, 301 redirecting hundreds of pages to the home page instead of to their respective new URLs. Ugh. In any case, we will probably fix this for the sake of implementing best practices. But I am curious how search engines treat 301'd URLs, as they are supposed to be permanent redirects. Will search crawlers ever visit the old URLs again to find that we've re-redirected them? Or have they written them off as moved to the home page for good, meaning that there's no way to direct the authority of the previous URLs to their rightful targets? Thanks!
Web Design | | SEOTeamSF0