How to Set Custom Crawl Rate in Google Webmaster Tools?
-
This is really silly question to set custom crawl rate in Google webmaster tools. Any one can find out that section under setting tab. But, I have confusion to decide number for request per second and second between requests text field.
I want to set custom crawl rate for my eCommerce website.
I checked my Google webmaster tools and find out as attachment. So, Can I use this facility to improve my crawling?
-
Hi Daniel,
I was thinking about it due to very less number of indexing. Thanks for your suggestion.
-
It's better to just leave it on default - custom crawl rates are only needed if you're experiencing traffic problems from the server(as stated in your screenshot).
Yyou will also use more bandwidth on your own server by the google bot coming more often so this should also be something you think about. Also worth mentioning is that after 90 days it will go back to default settings.
http://googlewebmastercentral.blogspot.com/2006/10/learn-more-about-googlebots-crawl-of.html
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can Google Crawl & Index my Schema in CSR JavaScript
We currently only have one option for implementing our Schema. It is populated in the JSON which is rendered by JavaScript on the CLIENT side. I've heard tons of mixed reviews about if this will work or not. So, does anyone know for sure if this will or will not work. Also, how can I build a test to see if it does or does not work?
Intermediate & Advanced SEO | | MJTrevens0 -
Google Indexing Stopped
Hello Team, A month ago, Google was indexing more than 2,35,000 pages, now has reduced to 11K. I have cross-checked almost everything including content, backlinks and schemas. Everything is looking fine, except the server response time, being a heavy website, or may be due to server issues, the website has an average loading time of 4 secs. Also, I would like to mention that I have been using same server since I have started working on the website, and as said above a month ago the indexing rate was more than 2.3 M, now reduced to 11K. nothing changed. As I have tried my level best on doing research for the same, so please if you had any such experiences, do share your valuable solutions to this problem.
Intermediate & Advanced SEO | | jeffreyjohnson0 -
URL Parameter Setting Recommendation - Webmaster Tools, Breadcrumbs & 404s
Hi All, We use a parameter called "breadCrumb" to drive the breadcrumbs on our ecommerce product pages that are categorized in multiple places. For example, our "Blue Widget" product may have the following URLs: http://www.oursite.com/item3332/blue-widget
Intermediate & Advanced SEO | | Doug_G
http://www.oursite.com/item3332/blue-widget_?breadCrumb=BrandTree_
http://www.oursite.com/item3332/blue-widget_?breadCrumb=CategoryTree1_
http://www.oursite.com/item3332/blue-widget_?breadCrumb=CategoryTree2_ We use a canonical tag pointing back to the base product URL. The parameter only changes the breadcrumbs. Which of the following, if any, settings would you recommend for such a parameter in GWT: Does this parameter change page content seen by the user? Options: Yes/No
How does this parameter affect page content? Options: Narrows/Specifies/Other Currently, google decided to automatically assign the parameter as "Yes/Other/Let Googlebot Decide" without notifying us. We noticed a drop in rankings around the suspected time of the assignment. Lastly, we have a consistent flow of products that are discontinued that we 404. As a result of the breadcrumb parameter, our 404s increase significantly (one for each path). Would 800 404 crawl errors out of 18k products cause a penalty on a young site? We got an "Increase in '404' pages' email from GWT, shortly after our rankings seemed to drop. Thank you for any advice or suggestions! Doug0 -
The best tool
Hi friends !! I have a huge question . Which is the best tool for SEO? I am using a lot of tools but I would like to know more ways to position my website in the top . I hope that you can help me! Regards , Carlos Zambrana
Intermediate & Advanced SEO | | CarlosZambrana1 -
Can links indexed by google "link:" be bad? or this is like a good example by google
Can links indexed by google "link:" be bad? Or this is like a good example shown by google. We are cleaning our links from Penguin and dont know what to do with these ones. Some of them does not look quality.
Intermediate & Advanced SEO | | bele0 -
Google places
Is there away to get to the top of google places? Can it be manipulated?
Intermediate & Advanced SEO | | dynamic080 -
HELP - got the following message - Google Webmaster Tools notice of detected unnatural links
Hi All, While trying to grow we used several freelancers and small companies for guest blogging, article submissions etc. We lost about 90% of traffic from our peek at December. We don't know if it is related but we got the following message last week:
Intermediate & Advanced SEO | | BeytzNet
"Google Webmaster Tools notice of detected unnatural links to www.domain.com" Is it related (getting this message after two months of losing traffic)? What to do???? (P.S
We fired most of the companies we used months ago since we noticed they used bad methods. We didn't believe it can hurt us - just thought it would be useless...) Please Help...0 -
Understanding Google's keyword tool...
When I type in Google a keyword like : boot camp I get results that show Bootcamp (one word) traffic in the tens of thousands. I see many words combined. Does this mean that tens of thousands of people every month are misspelling that keyword? How should I interpret this in terms of anchor texting? I would hate to deliberately misspell it on my website just to get traffic. For those interested, my website is: http://ultimatebasictraining.com/admin/ (currently revaming my http://ultimatebasictraining.com website)
Intermediate & Advanced SEO | | StreetwiseReports0