Keyword Grader & OmniExplorer_Bot Question
-
I found this in several of my clients Robots.txt
"User-Agent: OmniExplorer_Bot
Disallow: /"Right now I cannot grade any of them that are on a particular platform that has that written in.
Would this block the keyword grader?
Thanks!
-
I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?
-
User-Agent: bingbotCrawl-delay: 5
User-Agent: msnbotCrawl-delay: 5
User-Agent: Googlebot-MobileAllow: /mobi
User-Agent: OmniExplorer_BotDisallow: /
-
Hey,
Mike is correct that our crawler is named RogerBot.
I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.
This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
I hope that helps track down the issue.
Cheers,
Joel. -
Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What do you do when there is significant movement when checking a Ranking & Algorithm tracking tool?
I've checked various Ranking & Algorithm tracking tools (SEMrush, Mozcast etc.) and since 12th Jan 18 there has been a lot of 'turbulence' in terms predicted Google algorithm changes. This has been 'high' for the past 2 weeks. Question is, would you 'hang tight and ride the wave' or 'act fast'? Our actual rankings haven't really been affected positively or negatively - so I'm thinking I should just ride the wave to see what happens but keep a close eye on the rankings?
Moz Bar | | Jack_Jahan1 -
Does Organic CTR from Keyword Explorer show percentage of 1st position clicks? If not, how to calculate?
I have two questions about the Organic CTR number from the Moz Keyword Explorer tool. I have read through the description here, but it is not very clear on what I need to know: https://moz.com/community/q/what-is-a-good-keyword-opportunity-score Does it calculate the percent of searchers that are likely to click on ANY of links from page one of the traditional organic search results? Or does it calculate the percent of searchers that are likely to click on the NUMBER 1 position of the traditional organic search results? I suspect the answer will be that it calculates the percentage of clicks for any of the traditional organic search results from page one. Which leads me to question two... I'm trying to calculate the estimated search traffic from each keyword if there was a number 1 ranking. And if the Organic CTR shows the percentage of clicks going to any of the traditional organic search results I need to add a modifier to my formula which will calculate how many clicks the number 1 ranking receives. So I think the formula would will look something like this: Monthly website traffic from keyword = monthly search traffic * Organic CTR * Number 1 Position CTR. In this case, what would I use for the "Number 1 Position CTR" modifier?
Moz Bar | | elementpaints0 -
Keyword Explorer search tips&tricks e.g. A&B , A&B Not C, AorB
I'm new to MOZ and I like the Keyword Explorer a lot. I'm curious if there is any additional helpful information that I can look up in order to optimized SEO? What if I want to search for a keyword combination? Apple and Oranges but not Kiwi Thanks in advance, Elmar
Moz Bar | | ElmarW0 -
Label existing keywords
I have keywords that were added without a label that I would now like to associate with an existing label. I don't want to lose my tracking data by deleting them and re-adding. Is there a way to easily label them with an existing label?
Moz Bar | | AppliedMarketingScience2 -
Use of Keyword Analysis Tool (espcially regarding search volume)
Hello! I am fairly new to Moz and just starting to explore the available Research Tools. At first glance, I was very excited about the Keyword Analysis tool, but I quickly realized that (a) I wasn't receiving the information I needed and (b) I didn't know what to do with the information it gave me. So... Two specific questions: Is there any way to see the Google Search Volume instead of the Bing Search Volume? The overwhelming majority of our organic site traffic comes from Google, so it seems silly to look at the Bing search volume. How do you deal with the fact that this tool only returns search volume based on an exact match? As we know, there are some many ways the same general query could be entered, but it seems ridiculous for me to sit there and run the tool against every iteration I can think of just to get a general sense of the search volume. Two general questions: What do you find helpful about the Keyword Analysis tool and how does it inform your SEO work? What other tools do you recommend to help determine Google search volume for keywords? Thanks in advance for your help and advice!
Moz Bar | | Marketing_Dept0 -
Can the Moz tool identify variations of a Chinese language branded keyword?
We've recently started a trial of the Moz Pro service and are tracking a selection of keywords. Our primary band / product name is 功夫英语（Kungfu English), so we've set a rule that any keywords containing those four Chinese characters (功夫英语) should be marked as a branded keyword. However, in the "Non-Paid Keywords Sending Search Visits" section of our traffic report, we see a few variations on our brand name that are not being marked as branded keywords. (See attached Images). Based on our rules, shouldn't these variations also be marked as branded keywords without our needing to manually add them as such? Or have I misunderstood the intent of this rule? For the English text, the brand rule about words containing "kungfuenglish" seems to have resulted in all of "www.kungfuenglish.com", "http://www.kungfuenglish.com", and kungfuenglish being labelled as branded keywords. However, I'm not seeing the same sort of result with variations on the Chinese keyword, 功夫英语. KedTi1D.png z5bcUIt.png
Moz Bar | | PaulCoffey0 -
Keyword Difficulty Showing ONLY Bing Search Volume (Exact Match)
Hi I am using the "Keyword Difficulty" tool and selecting "Google US". But the report that gets generated shows "Bing Search Volume (Exact Match). Is there any way to get "Google Search Volume (Exact Match)" being shown in the report? Regards
Moz Bar | | rholt0
