Bing's indexed pages vs pages appearing in results
-
Hi all
We're trying to increase our efforts in ranking for our keywords on Bing, and I'm discovering a few unexpected challenges. Namely, Bing is reporting 16000+ pages have been crawled... yet a site:mywebsite.com search on Bing shows less than 1000 results.
I'm aware that Duane Forrester has said they don't want to show everything, only the best. If that's the case, what factors must we consider most to encourage Bing's engine to display most if not all of the pages the crawl on my site?
I have a few ideas of what may be turning Bing off so to speak (some duplicate content issues, 301 redirects due to URL structure updates), but if there's something in particular we should monitor and/or check, please let us know. We'd like to prioritize
Thanks!
-
Yep, if Bing Webmaster Tools doesn't show problems with the sitemap, I'd focus on the points I highlighted back in mid-June on this thread (make content robust, unique, and make sure text is in HTML).
Good luck,
Kristina
-
Hello again Kristina
Bing's showing 38,885 pages indexed... and I've noticed the amount of pages vary after clicking through several pages.
So I guess the problem isn't why aren't they indexing, but rather why aren't they showing all pages. I'd assume this is related to page quality (content, on-page ranking factors, etc)?
-
I haven't heard of Bing keeping historically submitted sitemaps and confusing them, although I know that they're very picky about the number of inaccuracies they find in a sitemap, so it's possible they keep the latest one around so they can refer to it if the current one seems to have holes.
That said - when you search for your site, are the same pages coming up on the first page? What about the second? Third? The number of pages that come up when you search for site:mysite.com are approximations and can vary even as you scroll through the results pages. The more important question is, how many pages does Bing say are indexed in Bing Webmaster Tools?
-
Just an update:
Bing reported a successful crawl after submitting a new one, then rejected it based on an error that it didn't describe. Took it down, made a change to URL itself (somehow the .gz extension wasn't there) and resubmitted on 7/7/13.
Since then, Bing has reported a successful crawl, then reported a successful crawl on 6/30/13 (7 days before submission?), then reported a failed crawl on 7/5/13 (2 days before submission?) and now today again reporting a successful crawl on 7/7/13.
So my question now is... does Bing keep record of historically submitted sitemaps and confuse them with new submissions of the same ones? I've yet to see Bing actually index what's in the sitemaps, as a site: operator search is still a daily fluctuation between 1200 and 3300 results, sometimes going up to 4400. But again, this is daily. Right now, searching site:roadtrippers.com on Bing reports 4,420 results. Later today, I imagine it'll be around 3,300 or 1,200.
Any suggestions at all would be greatly appreciated.
-
Good luck!
If these tips don't work, you should follow up here again, but include a little more information about your site. It's possible that Bing IS crawling all of your pages properly, but something about them is making Bing think that they aren't valuable enough to be in their indexes. I'd particularly look to see if:
- Content seems to be duplicate, either within your site or if it's duplicated elsewhere
- Content is extremely thin (less than 100 words on a page/no unique text above the fold)
- Content is unreadable by Bing: check the cached version of a page that's not indexed and make sure you can read the unique content
Hope this helps! I'm going to mark this question as "answered," only because if you have a follow up question, it'll probably be more specific now that you have more information, and I'd like all of that info to be included in the original question.
Best,
Kristina
-
Hey Kristina
It has not unfortunately.
Bing reports successful crawls, however it's not crawling it - at all.
After reading more about Bing's sitemap preferences, there are a few things left to try. I'm using this post on Bing's forums http://www.bing.com/blogs/webmaster/f/12248/t/659635.aspx#9602607 as a reference for now. We're going to make a temporary separate sitemap for Bing to test what is suggested in that link. Hopefully something sticks and we can make some progress going forward!
Brandon
-
Hi Brandon,
Just wanted to check in - did using 1 sitemap work?
Kristina
-
I believe I've found the solution - as recently as 2009, Bing was only crawling one sitemap per website. It also said Bing would only crawl the most recently submitted sitemap but it doesn't appear that was the case for our site.
So I've since removed the old sitemap and am waiting to see some evidence of our new sitemap being crawled and indexed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What happens when a de-indexed subdomain is redirected to another de-indexed subdomain? What happens to the link juice?
Hi all, We are planning to de-index and redirect a sub domain A to sub domain B. Consequently we now need to d-index sub domain B also. What happens now to the link juice or page rank they gained from hundreds and thousands of backlinks? Will there be any ranking impact on main domain? Backlinks of these sub domains are not much relevant to main domain content. Thanks
Algorithm Updates | | vtmoz1 -
Meta robots at every page rather than using robots.txt for blocking crawlers? How they'll get indexed if we block crawlers?
Hi all, The suggestion to use meta robots tag rather than robots.txt file is to make sure the pages do not get indexed if their hyperlinks are available anywhere on the internet. I don't understand how the pages will be indexed if the entire site is blocked? Even though there are page links are available, will Google really index those pages? One of our site got blocked from robots file but internal links are available on internet for years which are not been indexed. So technically robots.txt file is quite enough right? Please clarify and guide me if I'm wrong. Thanks
Algorithm Updates | | vtmoz0 -
Help Me Change My Client's Mind
My client wants to build a second site to provide targeted links for SEO to his main site. He's interested in buying a TLD with some near topic authority/links and then build the second site's authority up from there. He is clear that this could get him in trouble for a link scheme, but thinks it can all be hidden from Google. Off the top of my head I was able to recall a few of the pain-in-the-neck things you'd have to do to not get caught, but he seemed unconvinced. I recall you'd have to have: Different registrar Different contact/WhoIs Different site host Different G/A, GWT Logging into second's site's G/A, GWT with different IP address not used for main domain With the exception of the last one, he didn't seem to think it would be too hard. Aren't there more difficult maneuvers required for hiding this from Google? I want to be able to point out to him how ridiculous this low integrity effort will be, without losing the client. Thanks! Best... Darcy
Algorithm Updates | | 945010 -
Search Bar Under Brand In Search Result
I must have missed this or been living under a rock but when I type 'amazon' in a google search. Below Amazon.com I see a "search amazon.com" search bar. How do you get this? Thanks
Algorithm Updates | | Mike.NW1 -
Our root domain is no longer appearing in search results
Hi all The root domain for our site, roadtrippers.com, has been disappearing from Google's search results. Subfolders and subdomains still appear, but our root domain isn't found at all. I believe I've verified this by searching "-inurl:trips -inurl:byways -inurl:support -inurl:blog -inurl:places -inurl:guides -inurl:destinations site:https://roadtrippers.com/" in Google and our root domain is nowhere to be found. This may or may not be related to another issue we've had, where the root domain is appearing with a seemingly rotating set of parameters. Sometimes it'll be ?mod=, sometimes it'll be ?tag=translation. Originally they appeared to simply displace our ranking root domain, but now they and our root domain are completely disappearing. Our dev team believes they fixed the problem with recent 301 tags to any unapproved parameter being added to the root domain, but this hasn't fixed the original problem. Any insight into this is greatly appreciated! Brandon
Algorithm Updates | | brandonRT0 -
Website dropping from page 1google uk
Hi all, Firstly let me stress I am not really SEO minded, I know the very basics and that is about it. I am a driving instructor in the UK and have had my website (Wordpress) on page 1 for about 3 years now round the position 3 mark but for the last few months it has been dropping and is now right at the bottom of page 1 so no doubt a few more days and it will vanish completely from page 1 of google.co.uk . I was wondering if someone could just have a quick look at the page to see if they can see anything obvious that wouldn't be seen by me! The search term is " Driving Lessons Worcester " and the page that has always show on page 1 of google is .. http://www.passlee.com/driving-lessons-worcester.html I also had another site on page 1 for about 2 years which was www.drivinglessonsworcester.com this has also vanished to page 5 over the last 3 to 4 months. What really hurts is I made a website for another local instructor using WP and with similar setup to mine and that is now showing as No.1 on page 1 of google.co.uk!! So how is it the website I did for him is doing amazing yet mine is dieing a death when they are setup the same way but obviously different content! As I said I barely know the basics , I am sure a lot of you are thinking " Just go research " which I know I should and no doubt will, but I just wanted someone to have a very quick look to see if there was anything obvious! Kind Regards Lee Francis
Algorithm Updates | | germinus0 -
SERP Rankings: Breadcrumb appears near URL
Hi mozzers, I was checking at the "carpet cleaning" kw national search and an usual result appeared(image attached): -Title Tag -Url + Breadcrumbs following The Breadcrumb showing up near the url is the first time I see that happening! Anyone has an idea why? Do you think it is a Google new trick or do you guys think it is the webmaster who added a hack to it? Thanks for letting me know Tf52L.png
Algorithm Updates | | Ideas-Money-Art0 -
Elaborate Search Results
Please see the attached image. My question : How can I have such results showing for my own website? I mean what should I do to make this happen? 3ZQfr.jpg
Algorithm Updates | | KS__0