Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

Why does Bing bot crawl so aggressively?

Technical SEO

1293

Roverandom last edited by

We observer that the Bing bot is crawling our site very aggressively. We set Bing's crawl control so that it should not crawl us during heavy traffic hours, but that did not change a thing. Does anyone have the problem and even better a solution?
1 Reply Last reply
Reply Quote 1
Alick300 last edited by
Hi Thomas,

I didn't tested this ever for my website but you can give this a try by mentioning in robots.txt file

User-agent: bingbot
Crawl-delay: 1
- 1 – Slow
- 5 – Very Slow
- 10 – Extremely Slow
Thanks
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Google stopped crawling my site. Everybody is stumped.

This has stumped the Wordpress staff and people in the Google Webmasters forum. We are in Google News (have been for years), and so new posts are crawled immediately. On Feb 17-18 Crawl Stats dropped 85%, and new posts were no longer indexed (not appearing on News or search). Data highlighter attempts return "This URL could not be found in Google's index." No manual actions by Google. No changes to the website; no custom CSS. No Site Errors or new URL errors. No sitemap problems (resubmitting didn't help). We're on wordpress.com, so no odd code. We can see the robot.txt file. Other search engines can see us, as can social media websites. Older posts still index, but loss of News is a big hit. Also, I think overall Google referrals are dropping. We can Fetch the URL for a new post, and many hours later it appears on Google and News, and we can then use Data Highlighter. It's now 6 days and no recovery. Everybody is stumped. Any ideas? I just joined, so this might be the wrong venue. If so, apologies.
Technical SEO | | Editor-FabiusMaximus_Website

0
Error report in Bing Evaluated size of HTML....

Hi Whilst checking Bing's SEO analyser I got this error message for our page www.tidy-books.co.uk/childrens-bookcases "Evaluated size of HTML is estimated to be over 125 KB and risks not being fully cached. (Issue marker for this rule is not visible in the current view)" Just wondering what needs to be done about it and what it actually means? Thanks
Technical SEO | | tidybooks

0
Hreflang Tag great for Google, what about Bing or others?

I've read that the Hreflang Tag is all the rave for International solutions on a per page basis. I haven't read much about what International agencies are using for non-Google search engines such as Bing. Is the common language meta tags the only solution? would love to see an article that addresses this
Technical SEO | | MikeSEOTruven

0
Site not ranking in Google but comes up #1 in Yahoo and Bing

Hi everyone, I've been working on SEO for this site for about 2 years and for some reason the site has just tanked in google. However it shows up #1 in yahoo and bing for the same search. http://www.nfsmn.com Phrase: "commercial foundation repair mn" If anyone can shed some light on the issue I would really appreciate it. They do have a sister-site: american-waterworks.com that may be causing issues as they link a lot of content to amww but not the other way around. Thanks Eric
Technical SEO | | reynoldsdesign

0
Is Google caching date same as crawling/indexing date?

If a site is cached on say 9 oct 2012 doesn't that also mean that Google crawled it on same date ? And indexed it on same date?
Technical SEO | | Personnel_Concept

0
When is the last time Google crawled my site

How do I tell the last time Google crawled my site. I found out it is not the "Cache" which I had thought it was.
Technical SEO | | digitalops

0
Does Google pass link juice a page receives if the URL parameter specifies content and has the Crawl setting in Webmaster Tools set to NO?

The page in question receives a lot of quality traffic but is only relevant to a small percent of my users. I want to keep the link juice received from this page but I do not want it to appear in the SERPs.
Technical SEO | | surveygizmo

0
Bing Cache

How can you see what pages are cached by bing. I'm basically looking for these google approaches for bing: cache:domain.com site:domain.com Thanks Tyler
Technical SEO | | tylerfraser

1