Bing's indexed pages vs pages appearing in results
-
Hi all
We're trying to increase our efforts in ranking for our keywords on Bing, and I'm discovering a few unexpected challenges. Namely, Bing is reporting 16000+ pages have been crawled... yet a site:mywebsite.com search on Bing shows less than 1000 results.
I'm aware that Duane Forrester has said they don't want to show everything, only the best. If that's the case, what factors must we consider most to encourage Bing's engine to display most if not all of the pages the crawl on my site?
I have a few ideas of what may be turning Bing off so to speak (some duplicate content issues, 301 redirects due to URL structure updates), but if there's something in particular we should monitor and/or check, please let us know. We'd like to prioritize
Thanks!
-
Yep, if Bing Webmaster Tools doesn't show problems with the sitemap, I'd focus on the points I highlighted back in mid-June on this thread (make content robust, unique, and make sure text is in HTML).
Good luck,
Kristina
-
Hello again Kristina
Bing's showing 38,885 pages indexed... and I've noticed the amount of pages vary after clicking through several pages.
So I guess the problem isn't why aren't they indexing, but rather why aren't they showing all pages. I'd assume this is related to page quality (content, on-page ranking factors, etc)?
-
I haven't heard of Bing keeping historically submitted sitemaps and confusing them, although I know that they're very picky about the number of inaccuracies they find in a sitemap, so it's possible they keep the latest one around so they can refer to it if the current one seems to have holes.
That said - when you search for your site, are the same pages coming up on the first page? What about the second? Third? The number of pages that come up when you search for site:mysite.com are approximations and can vary even as you scroll through the results pages. The more important question is, how many pages does Bing say are indexed in Bing Webmaster Tools?
-
Just an update:
Bing reported a successful crawl after submitting a new one, then rejected it based on an error that it didn't describe. Took it down, made a change to URL itself (somehow the .gz extension wasn't there) and resubmitted on 7/7/13.
Since then, Bing has reported a successful crawl, then reported a successful crawl on 6/30/13 (7 days before submission?), then reported a failed crawl on 7/5/13 (2 days before submission?) and now today again reporting a successful crawl on 7/7/13.
So my question now is... does Bing keep record of historically submitted sitemaps and confuse them with new submissions of the same ones? I've yet to see Bing actually index what's in the sitemaps, as a site: operator search is still a daily fluctuation between 1200 and 3300 results, sometimes going up to 4400. But again, this is daily. Right now, searching site:roadtrippers.com on Bing reports 4,420 results. Later today, I imagine it'll be around 3,300 or 1,200.
Any suggestions at all would be greatly appreciated.
-
Good luck!
If these tips don't work, you should follow up here again, but include a little more information about your site. It's possible that Bing IS crawling all of your pages properly, but something about them is making Bing think that they aren't valuable enough to be in their indexes. I'd particularly look to see if:
- Content seems to be duplicate, either within your site or if it's duplicated elsewhere
- Content is extremely thin (less than 100 words on a page/no unique text above the fold)
- Content is unreadable by Bing: check the cached version of a page that's not indexed and make sure you can read the unique content
Hope this helps! I'm going to mark this question as "answered," only because if you have a follow up question, it'll probably be more specific now that you have more information, and I'd like all of that info to be included in the original question.
Best,
Kristina
-
Hey Kristina
It has not unfortunately.
Bing reports successful crawls, however it's not crawling it - at all.
After reading more about Bing's sitemap preferences, there are a few things left to try. I'm using this post on Bing's forums http://www.bing.com/blogs/webmaster/f/12248/t/659635.aspx#9602607 as a reference for now. We're going to make a temporary separate sitemap for Bing to test what is suggested in that link. Hopefully something sticks and we can make some progress going forward!
Brandon
-
Hi Brandon,
Just wanted to check in - did using 1 sitemap work?
Kristina
-
I believe I've found the solution - as recently as 2009, Bing was only crawling one sitemap per website. It also said Bing would only crawl the most recently submitted sitemap but it doesn't appear that was the case for our site.
So I've since removed the old sitemap and am waiting to see some evidence of our new sitemap being crawled and indexed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long does Google indexes the website after algorithm update?
I've noticed that the page ranks for some queries were improved unexpectedly, without any actions from my side. Is it possible that this improvement is connected with with Google algorithm update Sep, 4-5?
Algorithm Updates | | AurigaPR0 -
On page vs Off page vs Technical SEO: Priority, easy to handle, easy to measure.
Hi community, I am just trying to figure out which can be priority in on page, off page and technical SEO. Which one you prefer to go first? Which one is easy to handle? Which one is easy to measure? Your opinions and suggestions please. Expecting more realistic answers rather than usual check list. Thanks
Algorithm Updates | | vtmoz0 -
Is it bad from an SEO perspective that cached AMP pages are hosted on domains other than the original publisher's?
Hello Moz, I am thinking about starting to utilize AMP for some of my website. I've been researching this AMP situation for the better part of a year and I am still unclear on a few things. What I am primarily concerned with in terms of AMP and SEO is whether or not the original publisher gets credit for the traffic to a cached AMP page that is hosted elsewhere. I can see the possible issues with this from an SEO perspective and I am pretty sure I have read about how SEOs are unhappy about this particular aspect of AMP in other places. On the AMP project FAQ page you can find this, but there is very little explanation: "Do publishers receive credit for the traffic from a measurement perspective?
Algorithm Updates | | Brian_Dowd
Yes, an AMP file is the same as the rest of your site – this space is the publisher’s canvas." So, let's say you have an AMP page on your website example.com:
example.com/amp_document.html And a cached copy is served with a URL format similar to this: https://google.com/amp/example.com/amp_document.html Then how does the original publisher get the credit for the traffic? Is it because there is a canonical tag from the AMP version to the original HTML version? Also, while I am at it, how does an AMP page actually get into Google's AMP Cache (or any other cache)? Does Google crawl the original HTML page, find the AMP version and then just decide to cache it from there? Are there any other issues with this that I should be aware of? Thanks0 -
Dates appear before home page description in the SERPs- HUGE drop in rankings
We have been on the first page of Google for a number of years for search terms including 'SEO Agency', 'SEO Agency London' etc. A few months ago we made some changes to the design of the home page (added a blog feed), and made changes to the website sitemap. Two days ago (two months after last site changes were made) we dropped subsantially in the SERPs for all home page keywords. Where we are found, a date appears before the description in the SERPs, dating February 2012 (which is when we launched the original website). The site has been through a revamp since then, yet it still shows 2012. This has been followed by a few additional strange things, including the sitelinks that Google is choosing to show (which including author bio pages showing in homepage site links), and googling our brand name no longer brings up sitelinks in the SERPs. The problem only affects the home page. All other pages are performing as standard. When Penguin 4.0 came out we saw a noted improvement in our SERP performance, and our backlinks are good and quality, largely from PR efforts. Of course, I would be interested in additional pairs of eyes on the back links to see if anyone thinks that I have missed anything! We have 3 of our senior SEOs working on trying to figure out what is going on and how to resolve it, but I would be very interested if anyone has any thoughts?
Algorithm Updates | | GoUp3 -
Is it stil a rule that Google will only index pages up to three tiers deep? Or has this changed?
I haven't looked into this in a while, it used to be that you didn't want to bury pages beyond three clicks from the main page. What is the rule now in order to have deep pages indexed?
Algorithm Updates | | seoessentials0 -
Drop in Page Indexing, Small rise in Search Queries
Hello, I have a news based website so i am creating multiple new posts daily. I changed a lot of the site and got rid of old potentially duplicate content back in feb and had a sharp drop in pages indexed. I know this was because I removed a lot of pages though. However I still have a good 20,000 + pages on my site and my indexing has dropped a further three times since then. From 9,000 to 2,000 a coupe of months ago and then slowly down since April to just 133. It doesn't seem to have affected my search queries yet but surely will if it continues. I am really confused as to how this might happen & how to turn it around. We dont use any dodgy SEO tricks either.
Algorithm Updates | | luwhosjack0 -
Very Strange Search Results!
Having just done a search on Google.co.uk for 'payday loans' I am baffled as to why the top two organic results (image attached) are even associated to the keyword. The KW isn't present in the title, metas, or content. Nor do any backlinks use relevant anchor text. I'm guessing this is an algorithmic 'f*ck up', do you agree? uGdk7Cw92Rme
Algorithm Updates | | Webpresence0 -
Local Vs. National Targeting - Pointless?
Hi, I have a client, in Maryland, that wants to rank on the first page for "yoga teacher training". But to my understanding Google (don't know about Yahoo! & Bing at the moment) is prioritizing local listings first based on the user's IP (or preferred city if using a logged in Google Account). Am I right about the priority of local listings? Is that a common occurrence now? Is it possible for my client to rank for yoga teacher training for someone searching in New York, for example, or is this a huge chalenge? Thanks for any advice!! Kindly, Michael
Algorithm Updates | | netex720