Keyword Grader & OmniExplorer_Bot Question
-
I found this in several of my clients Robots.txt
"User-Agent: OmniExplorer_Bot
Disallow: /"
Right now I cannot grade any of them that are on a particular platform that has that written in.
Would this block the keyword grader?
Thanks!
-
I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?
-
User-Agent: bingbot
Crawl-delay: 5
User-Agent: msnbot
Crawl-delay: 5
User-Agent: Googlebot-Mobile
Allow: /mobi
User-Agent: OmniExplorer_Bot
Disallow: /
-
Hey,
Mike is correct that our crawler is named RogerBot.
I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.
This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
I hope that helps track down the issue.
Cheers,
Joel. -
Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On-Page Grader giving wrong results for missing keyword in image Alt Attribute
I'm trying to optimize for the Keyword "River City Apartments" on this page: https://3fconstruction.net/case-studies/river-city-apartments-chicago-condo-deconversion-renovation/ When I use On-Page Grader to see how I'm doing it's telling me I'm missing the Keyword in the Image Alt Attribute. However, it's clearly in there. <strong>River City Apartments Aerial</strong> What am I missing?
Moz Bar | | Drew.Friestedt0 -
"Keywords Showing" total doesn't match "Tracked Keywords" total
My "Tracked Keywords" total under "Add & Manage Keywords" does not match the total for "Keywords Showing" on the main "Rankings" tab. Why is there a difference? For example, I have 453 tracked keywords attached to my campaign, but under the "Tracked Keywords Overview," there are only 254 listed. I'm sure there's a simple explanation but I can't find it anywhere!
Moz Bar | | MSoleski0 -
Keyword Difficulty tool showing Google search volume or Bing?
Hi, I plugged in a bunch of keywords to know their search volume, chose Google from the drop down and then chose English from the next dropdown and clicked on Check dificulty. It shoes me "Bing Search Volume (Exact Match)" on top but shows the name of the keyword and "for Google CA" on the left. Is the search volume for Google? or Bing? and the difficulty as well? Google or Bing? "
Moz Bar | | SpaMedica0 -
Google ranking by keyword for website
Hi, i am wondering if there is a function to check page ranking on google for a specific keyword/s for our website using MOZ. Thanks
Moz Bar | | andrewlos0 -
Idea for new feature on Moz Q&A
Hi Moz Can we please have a new feature to moz when answering questions please, the ability to save as a draft. I am currently on the train from London back home and I was trying to answer a few questions and give my opinion and after writing a really long and detailed response to an area I actually have quite a lot to say on, my internet froze and the page refreshed losing all the content. It doesn’t help that my Mac I am working on isn’t the fastest, but my new one isn’t quite ready yet. But if there was a simple save draft feature, it would be really great as I wasn’t ready to post my answer as it wasn’t finished and without checking wouldn’t have made sense. Yes I know I could of done it in word and pasted it across, but usually when I am travelling and answering questions I do it on my Ipad which isn’t great for multitasking. While I understand you don’t want lots of ‘draft answers’ all over the place as this would take storage space up on your servers and doesn’t actually help the person answering the question, your system could purge all draft answers which have not been edited in the last 24 hours. Just a little suggestion to help make this great forum even better. Thanks Andy
Moz Bar | | Andy-Halliday4 -
URL inaccessible for On Page Grader
I am trying to use the on page grader however it is not working with my website. The URL is as follows: https://capbeast.com. I have been trying to read in older posts to see if https is now supported or not but have not found anything. I know there is no robots.txt issue as I am able to run the crawl test on our website fine. Is the issue on my end in regards to configuration or is due to DDos attacks? Any help would be appreciated. Thanks
Moz Bar | | MisterStitches0 -
On-Page Grader is not counting H2 and H3 Keywords
I used on-page grader in one of my webpages and it showed the limit of 15 keywords used. The problem is that it counted only when the keyword appears in the Title, URL, Meta Description, H1, Body and IMG ALT. If you take into account the H2 and H3 tags, the count will go to 22. Is there a problem with the On-Page Grader Tool or it's not a problem to go above 15 keywords if they are in H2 and H3? Here's the link for the page: http://www.tororadar.com.br/acoes/bbse3-bb-seguridade/cotacoes-bovespa Keyword: BBSE3 Thanks in advance.
Moz Bar | | Guilhermealvesc0