Firefox Add-On for crawl frequency??
-
Hi all,
a short one: is there a firefox add-on available, which lets you see the crawl frequency of your page(s)? Would be interesting to see if google bot comes around more lately...
There are some statistics in the webmaster tools, but I don't find them very attractive
I know there is something for Wordpress, but we don't use it...
I don't to put up an excel-sheet and check the cache-version for myself. And I would love to see how deep the crawler gets and which sites do not get crawled...
So, any existing add-ons or tools that are for free??
Thanx....
-
Hi,
Your best bet is Web master Tools or any other server side tool like the ones available for Word Press like you've mention. Depending on your server (unix or microsoft based) and/or your platform for the backend you can install / add additional features to get more info for this - google bot visits / crawl rates and formats.
There are no fireFox addons since the browser dosen't have access to your site / server in order to see when google bot is visiting what pages and how is it performing - as that 's a behind the scene type of process.
If you have access to your server you might try newrelic.com  (I am not affiliate in any way with them - I just love the tool). However there are several others in the same spectrum that will give you more then the Google bot stats and data.
Hope it helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz crawler is not able to crawl my website
Hi, i need help regarding Moz Can't Crawl Your Site i also share screenshot that Moz was unable to crawl your site on Mar 26, 2022. Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster.
Technical SEO | | JasonTorney
my robts.txt also ok i checked it
Here is my website https://whiskcreative.com.au
just check it please as soon as possibe0 -
Dropdown content on page being crawled
Hi, will the content within a dropdown on a page be crawled? I.e. if the page visitor has to click to reveal the content as a dropdown will it be crawled by bots. Thanks
Technical SEO | | BillSCC1 -
New theme adds ?v=1d20b5ff1ee9 to all URL's as part of cache. How does this affect SEO
New theme I am working in ads ?v=1d20b5ff1ee9 to every URL. Theme developer says its a server setting issue. GoDaddy support says its part of cache an becoming prevalent in new themes. How does this impact SEO?
Technical SEO | | DML-Tampa0 -
Google's ability to crawl AJAX rendered content
I would like to make a change to the way our main navigation is currently rendered on our e-commerce site. Currently, all of the content that appears when you click a navigation category is rendering on page load. This is currently a large portion of every page visit’s bandwidth and even the images are downloaded even if a user doesn’t choose to use the navigation. I’d like to change it so the content appears and is downloaded only IF the user clicks on it, I'm planning on using AJAX. As that is the case it wouldn’t not be automatically on the site(which may or may not mean Google would crawl it). As we already provide a sitemap.xml for Google I want to make sure this change would not adversely affect our SEO. As of October this year the Webmaster AJAX crawling doc. suggestions has been depreciated. While the new version does say that its crawlers are smart enough to render AJAX content, something I've tested, I'm not sure if that only applies to content injected on page load as opposed to in click like I'm planning to do.
Technical SEO | | znotes0 -
Why is my site crawled so much more in Bing then in Google?
I recently setup Cloudflare so I can see how much my site is being crawled. It looks like Bing is crawling me about 3 times as much as Google. Any ideas on why that would be or what I should check?
Technical SEO | | EcommerceSite0 -
New pages need to be crawled & indexed
Hi there, When you add pages to a site, do you need to re-generate an XML site map and re-submit to Google/Bing? I see the option in Google Webmaster Tools under the "fetch as Google tool" to submit individual pages for indexing, which I am doing right now. Thanks,
Technical SEO | | SSFCU
Sarah0 -
Sitewide Links delete or add nofollow
This question has been asked before, and I’ve read most of the answers. However, things are somewhat different, as we are a web hosting company and have many clients that link to us site wide in the footer, as well we have a website builder application where we control the footer links on our end user's websites. Most use just our “domain name” or “Powered by Domain” Should we remove them? It does provide visitors some value as they can tell where the website is hosting, has been developed or how to sign up for our website builder or web hosting services. Right now, they are all follow, and we are working on cleaning up our link profile so looking for some great advice on how to proceed. Our link profile is very large since we are a web hosting company that has been around for 10 plus years. Thanks in advanced for your recommendations.
Technical SEO | | goodhost0 -
Is there an easy tool to add all keywords I need to seomozpro?
In my website there are 3,300 products. www.theprinterdepo.com. I have about 200 keywords on the seomoz account. Every item has about 5 or more variations of keywords. examples: hp 1320 1320 printer hp 1320 refurbished 1320 hp printer hp laser 1320 and so on. The question is: Is there an easy way to add all the keywords related to my products and all its variations to the seomoz dashboard so I can more easily track them? Â If not, what do you suggest me to do?
Technical SEO | | levalencia10