Keyword Grader & OmniExplorer_Bot Question
-
I found this in several of my clients Robots.txt
"User-Agent: OmniExplorer_Bot
Disallow: /"Right now I cannot grade any of them that are on a particular platform that has that written in.
Would this block the keyword grader?
Thanks!
-
I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?
-
User-Agent: bingbotCrawl-delay: 5
User-Agent: msnbotCrawl-delay: 5
User-Agent: Googlebot-MobileAllow: /mobi
User-Agent: OmniExplorer_BotDisallow: /
-
Hey,
Mike is correct that our crawler is named RogerBot.
I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.
This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
I hope that helps track down the issue.
Cheers,
Joel. -
Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Keyword Volume - as compared to keyword planner and other tools.
Hi in the rankings for my keywords, the monthly range is so much lower than other keyword tools I use, including google keyword planner, semrush and longtail pro (which all match and show a number much higher than Moz). Support said it's because those tools may group like keywords or singular/plural versions where Moz truly separates each term as a separate term, hence more specific and precise. My question is: If google does somewhat group plural/singular and close match keywords, then wouldn’t it not matter anyways - if someone types a close match keyword and my keyword ends up triggering/displaying as if they typed my targeted keyword - then wouldn’t it still fall under the larger number if in the end it works in displaying my website as a result? If google changes search "blue sky" to "sky that is blue", and you get the same ranking for either of those two keywords then in the end does it matter? I'm thrown for a loop as I spent time optimizing my website based on numbers I received from the other tools and now seeing a much lower number for said keywords and possibly a little higher for a different variation of it, I don't know what to do.
Moz Bar | | camoeverafter.com0 -
Keyword Explorer - Unable to complete searches for data?
Hey guys, I'm in the process of completing some keyword research using Keyword Explorer but every time I complete a search I'm told that explorer is unable to perform the search at this time and that I should refresh the page, or search with a new query. Is anyone else having this issue/had this problem in the past?
Moz Bar | | daniel-brooks0 -
On page grader tool - ip address
Does anyone know Moz's IP address for the above tool so I can give it access to a staging area of our site?
Moz Bar | | Jennie050 -
What does the green check mark mean in the KW Difficulty & SERP Analysis Report mean?
Hi, I just generated a Keyword Difficulty and SERP Analysis Report and I find that many of the metrics have a green checkmark besides them. What does that indicate? Similarly, there are many metrics which appear in green. However rest are in grey. Also help what does that indicate? Refer the attached link to the screenshot of the report. I tried looking at a couple of places for the answer but in vain.. Thanks.. DQSlssU
Moz Bar | | Shalin.TJ0 -
Keyword Difficulty Report - Any way to have it send weekly?
I need the SERP data found in the Keyword Difficulty research tool each week for a search phrase. We are not tracking any specific URL - just really need the SERPS page 1 for the term weekly. Is there a way to have this report sent to use each week automatically? We have RSS feeds for the search but the end result is not visually appealing. If not this, does anyone know of any service that could provide this? Thanks
Moz Bar | | S2RSolutions0 -
Keyword Tool using Bing instead of Google
I'm trying to analyze some keywords, and I've selected 'Google' and 'United States' - but when the ranking comes back, it is showing up as the rankings for Bing. Any idea why this continues to happen? Thanks!
Moz Bar | | Marketing-Maven0 -
Inconsistent On-Page Grader results
Hello everyone! I am fairly new to the MOZ tools but I'm finding some inconsistencies in the On-Page grader.. I have tested several pages (some of which I know have doos and bad SEO) and the results I keep getting are very volatile. For example, I have looked at testing http://www.sportingwebdesign.com for "Sports Website Design" - one minute it gave me a grade F, then then 2 minutes after I refresh the page and it gave me a grade A.. I am very confused and a bit worried that this tool may be a bit volatile... Anyone have any thoughts?
Moz Bar | | Social_Media_Mentors0
