Bing Won't Index Site - Help!
-
For the past few weeks I’ve been trying to figure out why my client's site is not indexed on bing and yahoo search engines.
My Google analytics is telling me I’m getting traffic (very little traffic) from Bing almost daily but Bing webmaster tools is telling me I’ve received no traffic and no pages have been indexed into Bing since the beginning of December.
At once point I was showing ranking in Bing for only one keyword then all of a sudden none of my pages were being indexed and I now rank for nothing for that website. From Google I’m getting over 1200 visits per month. I have been doing everything I can to possibly find the culprit behind this issue.
I feel like the issue could be a redirect problem. In webmaster tools on Bing I’ve used “Fetch as Bingbot” and every time I use it I get a Status of “Redirection limit reached.”. I also checked the CRAWL Information and it’s saying all the URL’s to the site are under 301 redirect. A month or so ago the site was completely revamped and the canonical URL was changed from non www to www. I have tried manually adding pages to be indexed multiple times and Bing will not index any of the sites pages. I have submitted the sitemap to Bing and I am now at a loss. I don’t know what’s going on and why I can’t get the site listed on Bing.
Any suggestions would be greatly appreciated.
Thanks,
Stephen -
You have been extremely helpful. Thank you for all your help. I will get right on this to get resolved.
-
Okay, so the crawler is seeing it for the www. version but not for the non www version.
If we look in a browser at:
http://dunninggolf.com it redireects to www.dunninggolf.com
If I look in webbug at the headers it does not redirect and returns a 200 Okay status code for the non www version of the URL.
I think this could be the issue here and that for some reason, when bing visits the site is giving out some strange 301 response which then sends out a loop of 302 redirects.
Okay, there is a tool here that lets you visit the site as if you were bing bot and get the HTTP headers:
http://view-http-headers.gishan.net/fetch-as-bing-bot-tool.jsp
You have to select bingbot rather than IE6 from list of user agents.
If you visit the www.dunninggolf.com version then it returns a 200 OK
If you visit the http://dunninggolf.com version then it returns a 301 redirect (which in turn seemingly issues the loop of 302 temp redirects that never go anywhere).
Okay, there is your issue. The site has some issue with bingbot (and possibly others) when visited via the http://dunninggolf.com URL. This issue is not visible in webbug or a browser. That said, we do seem to get some redirection from http://dunninggolf.com to http://www.dunninggolf.com.
So, taking a stab in the dark, I would say there is some kind of meta refresh or something going on after the page has been returned as a HTTP 200 result. I am not on my windows PC so can't examine the headers more closely but this is certainly your issue here.
You need to find out the following.
1. How are they doing the redirect?
2. why are they redirecting this way?
3. why are they serving up these 301 / 302 loops via http://dunninggolf.com
Ultimately, this is an issue for the developers. You should advise them on correct protocol (a 301 via the HTTP headers from http://dunninggolf.com to http://www.dunninggolf.com and not via this whacky redirect they currently have in place).
Wowzers, that was a tricky one!
Hope this helps and sorts out your headache!
Marcus
-
I have used SEO analyzer before and i did it again just now and it gave me some weird error.
This is the result:
<label class="error" for="seoUrl">The URL has too many redirects</label>
| ! Redirects detected for the current URL |
I also have submitted the URL in the "submit URL toolbox page" a couple times.
I will get working on the robots.txt and make sure it's put in place.
I don't do any of the development work on the site i just perform the SEO duties. Seeing those redirect errors is part of what im talking about with it saying there are to many redirects. Any idea what that's all about if its showing everything should be working properly?
I just scanned it again and based on the URL i enter i get different results.
those redirect errors come when i analyze dunninggolf.com. But when i analyzed www.dunninggolf.com I only 2 errors on homepage for missing alt tags on images and no H1 tag. -
Hmm, five months is a bit long.
Lets look at what we do know.
1. Bing is telling you that the page it is trying to fetch is redirecting and the bingbot does not follow redirects (1).
2. If I fetch your page in a browser or a even just the headers in webbug I get a HTTP 200 OK status code and no 301 redirect.
So, the site itself seems fine. The pages are serving up fine and they are returning the correct HTTP status codes.
So.. have you tried analysing the page in the SEO Analyser? http://www.bing.com/webmaster/help/seo-analyzer-97615e21
If so, what does that tell you? It should detail the HTTP response codes that bing is seeing.
One possiblity is that there is something whacky going on for the bing user agent but fetching the page as bingbot also returns a 200 status code.
There is no robots.txt so I would put that straight first up as suggested and at the very least go with:
User-Agent: *
Allow: /Without access tp webmaster tools and digging further it is hard to see what is going on but 5 months seems too long. That said, there are no issues preventing crawling or indexation in so far as a 15 minute look can identify.
Daft question but have you submitted the URL via:
http://www.bing.com/toolbox/submit-site-url
I have just submitted it just in case you have only done via webmaster submit.
Anyhow, it's not secret that bing can be horribly slow and it seems that it still believes you have 301 in place for some reason and I could not find one.
Maybe one for Bing Webmaster Support as it seems there is no good reason why they would not index this site and the tools at your disposal are not working as they should: http://www.bing.com/webmaster/help/webmaster-support-24ab5ebf
References
1. http://www.bing.com/webmaster/help/fetch-as-bingbot-fe18fa0d
-
I have been working on getting some new links and have gotten a couple but nothing substantial as of yet.
This client is new for me and i am only aware of this site being around since June or July of last year.
I've been waiting for bing to show search results for the site for almost 5 months now and nothing has changed. I am getting to the point where i feel the culprit lies somewhere else besides saying bing indexing slowly as being an excuse.
-
It looks like the site was down for a long period of time and was redirecting to another site: http://web.archive.org/web/20090615041707/http://dunninggolf.com/
It also then looks like there was a holding page up for a while.
I can't see the complete history on archive.org and nothing since Feb 2011 but it looks like these historical redirects are your problem.
Few questions
How long has the new site been up?
Have you won any new links?
It seems that Bing still believes this site is redirecting somewhere else so you may just have to wait a while till they update their index. You could ask them to remove all of the links that they do currently hold.
If Bing webmaster tools is anyting like Google Webmaster tools the information there is horribly out of date so it may just be a matter of waiting this out till they recrawl the site and refresh everything.
A good use of your time may be just getting your on page ducks in a row and making sure everything else is okay.
- The canonical looks fine.
- There are no 301's on the pages I visited
- You have no robots.txt file
- Google is indexing it okay
Ultimately, I think this is a case of waiting it out. If you want to speed things up, try to get some fresh links to the site from well ranking sites to help drive the crawlers your way.
Hope that helps!
Marcus -
Yeah, the URL is http://www.dunninggolf.com/
-
I did check a small handful of pages and i did see rel="canonical" in the page source code. I only saw it on product pages and category pages etc. blog and other pages not advertising product i didn't see it.
But like i said i only checked about 8 pages and 6 out of the 8 had that tag on it.
-
Hey Stephen
Can you post a URL so we can take a look? Anything else would be a complete shot in the dark.
Marcus
-
Double check that the recent rel=can update for the main URL / homepage (your non-www to www) has not been accidentally applied to ALL the pages. That single mistake can hoark a site's indexing 100%.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
More 'New users' than 'Users' in Google Analytics
Hey Mozzers, I'm seeing more 'New users' than 'Users' in Google Analytics (see screenshot). This is over a 3 year time period. Any idea why new users would be greater than users? Thanks! 6lYmU0j
Reporting & Analytics | | RWesley0 -
Need help with my "ghost" blog...
...when I fired my original web designer, did they sabotage coding? I have never checked my Alexa/Google Analytics, or any blog ranking until last night. Subsequently, I have spent the last 24 hours googling away, and finally joining MOZ b/c I'm desperate to find out WHY I'm not ranking. I've googled and found many answers to a problem directly opposite of mine: (How to increase traffic with a high ranking), but I already have quite a bit of traffic (via Wordpress Stats), but can not be found on any ranking system. So, fiddled with some NoFollow/NoIndex boxes in Genesis SEO settings thinking maybe when my domain name changed it messed everything up? Most the boxes HAD been checked, so I unchecked them all. Anyhow, basically signed up for the monthly service so i could ask this question on the forum. My site is hellowhitney.com **it's so weird---i have a LOT of organic direct hits coming directly to my blog (for instance a celebrity re-posted a post which gained a lot of traffic from Twitter to the page), but Google nor another ranking is seeing it. IN FACT, it stops any and all ranking data back to FEBRUARY 2016 when I changed my domain name from Myscriptedreality.com to HelloWhitney.com Ignorance is NOT bliss in this case--would appreciate any help! #ForeverGrateful
Reporting & Analytics | | hellowhitney0 -
How do I track a primary domain and a subdomain as single site in Google Analytics?
Our website consists of a primary domain (marketing focused) and subdomain (ecommerce platform). The two sites look and function as one site even though they are using different technology. I would like to track the primary domain (example.com) and the subdomain (shop.example.com) as a single site in Google Analytics. The subdomain will be set up with GA ecommerce tracking as well. Can someone provide an example of the GA snippet that each would need?
Reporting & Analytics | | Evan340 -
Free Media Site / High Traffic / Low Engagement / Strategies and Questions
Hi, Imagine a site "mediapalooza dot com" where the only thing you do there is view free media. Yet Google Analytics is showing the average view of a media page is about a minute; where the average length of media is 20 - 90 minutes. And imagine that most of this media is "classic" and that it is generally not available elsewhere. Note also that the site ranks terribly in Google, despite having decent Domain Authority (in the high 30's), Page Authority in the mid 40's and a great site and otherwise quite active international user base with page views in the tens of thousands per month. Is it possible that GA is not tracking engagement (time on site) correctly? Even accounting for the imperfect method of GA that measures "next key pressed" as a way to terminate the page as a way to measure time on page, our stats are truly abysmal, in the tenths of a percentage point of time measured when compared with actual time we think the pages are being used. If so, will getting engagement tracking to more accurately measure time on specif pages and site signal Google that this site is actually more important than current ranking indicates? There's lots of discussion about "dwell time" as this relates to ranking, and I'm postulating that if we can show Google that we have extremely good engagement instead of the super low stats that we are reporting now, then we might get a boost in ranking. Am I crazy? Has anyone got any data that proves or disproves this theory? as I write this out, I detect many issues - let's have a discussion on what else might be happening here. We already know that low engagement = low ranking. Will fixing GA to show true engagement have any noticeable impact on ranking? Can't wait to see what the MOZZERS think of this!
Reporting & Analytics | | seo_plus0 -
Submitting an 'HTTPS' sitemap.xml to Bing
I have been trying to submit my sitemap to Bing [via their webmaster tools] for well over a week and it continues to report 'pending' My site is HTTPS and the sitemap is accepted by Google. I questioned Bing about this and got this response: To set your expectations, our Sitemap fetchers use a different pipeline and because of this, we cannot crawl Sitemaps in HTTPS format. We require that you submit an HTTP version of sitemap in order for Bing to properly crawl the file. Please go ahead and delete the current Sitemap and resubmit a new one in HTTP. Currently I don't and can't have a HTTP version of my site & sitemap and my developers are telling me that 3hrs worth of dev time will go into coming up with a work-around which I'm not sure I want to invest in [I have more important things to concentrate my spend on!]. Has anyone been faced with this problem and is there any quick/cheap alternative or do I just accept that Bing won't crawl my site until they update their end?!
Reporting & Analytics | | cityxplora.com0 -
404 errors on page urls that don't even exist
I am getting a lot of errors on pages with urls that aren't even legit. Like for example: /videos/support/index.asp No such path even exists like this on the site. I have a /videos and /support off root but no place on the site is there any reference or file at location /videos/support/index.asp so I get a lot of 404 duplicate page errors. This is just one example of several. How do I stop this?
Reporting & Analytics | | GKLWL0 -
Measuring events to external sites
Im having problem measuring click on ads by using events in GA or Jetpack. For example when I checked out yesterday this is what I read: 1. In GA events it says 12 clicks 2. In Jetpack it says 9 clicks But when I look at Referrals to the actual site directly it says 18 clicks Which one is the rights one? I need this because I use this to invoice clients end of month! and it cant be any "maybe".something. cheers, R
Reporting & Analytics | | rrrobertsson0 -
Can you help me figure out what happened to my website search results in Google?
On or about the 24th of April I noticed an abrupt decrease in traffic to my website:
Reporting & Analytics | | rdominey
http://www.getyourphotosoncanvas.com Sorry this might be long but I’m trying to be as thorough as possible. I thought that I had been hacked, a virus, maybe penalized by Google I don’t know what ? I submitted a reconsideration request to Google and they responded with the following: Reconsideration request for http://www.getyourphotosoncanvas.com/: No manual spam actions found
May 10, 2012
Dear site owner or webmaster of http://www.getyourphotosoncanvas.com/,
We received a request from a site owner to reconsider http://www.getyourphotosoncanvas.com/ for compliance with Google's Webmaster Guidelines. - - - - - -
We reviewed your site and found no manual actions by the webspam team that might affect your site's ranking in Google. There's no need to file a reconsideration request for your site, because any ranking issues you may be experiencing are not related to a manual action taken by the webspam team.
Google Search Quality Team I have ran all kinds of web crawl tests, Google webmaster, talked with SEO “Experts” and still can not figure out what is happening. I decided to use a couple of SEOmoz tools to try to help me explain what is happening. I figured that if I could take a very specific and unique KeyPhrase and run it on a specific page that I might be able to better explain what is happening. Basically, We appear to be no longer searchable by key words or phrases on google?
Here is an example:
Key Phrase: Free Services to Help Improve Your Photos on Canvas
Website: http://www.getyourphotosoncanvas.com/free-photo-canvas-retouching
Attached are some screen shots of the actual search results on Bing, Yahoo and Google along with the ranking tool results from SEOmoz and the on page grade for the key phrase.
Anybody got any Ideas? I am hurting; the internet and Google search is about 40% of by business. http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/Bing-Free-Services.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/Yahoo-Free-Services.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/Google-Free-Services.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/SEOmoz-Ranking.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/SEOmoz-Report-Card.jpg [" target="_blank">iframe>](<iframe class=) Bing-Free-Services.jpg Yahoo-Free-Services.jpg Google-Free-Services.jpg SEOmoz-Ranking.jpg SEOmoz-Report-Card.jpg0