Keyword Grader & OmniExplorer_Bot Question
-
I found this in several of my clients Robots.txt
"User-Agent: OmniExplorer_Bot
Disallow: /"
Right now I cannot grade any of them that are on a particular platform that has that written in.
Would this block the keyword grader?
Thanks!
-
I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?
-
User-Agent: bingbot
Crawl-delay: 5
User-Agent: msnbot
Crawl-delay: 5
User-Agent: Googlebot-Mobile
Allow: /mobi
User-Agent: OmniExplorer_Bot
Disallow: /
-
Hey,
Mike is correct that our crawler is named RogerBot.
I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.
This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
I hope that helps track down the issue.
Cheers,
Joel. -
Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Keyword Explorer how can I uncover all phrases relating to a single keyword?
From what I can see in Keyword Explorer you cannot enter a single keyword and uncover all search phrases that include this word. Or at least when I try to "Display keyword suggestions that" and select "only include keywords that contain all the query terms" and then enter a keyword. I cannot find a phrase in the list even when I can find it via a different search when if I type it directly.
Moz Bar | | GrouchyKids0 -
Keyword Difficulty Search Results: Can't tell which search engine
I ran a keyword difficulty search for 4 search engines: Switzerland in Italian, Switzerland in English, Switzerland in French and Switzerland in German. Now the results are in but it shows "Google.ch" for all of them so I can't tell which result is for which language. How can I check that? I ran the basic not full SERP.
Moz Bar | | WagMoreBarkLess0 -
Tool for seeing targeted keywords?
Hello Friends, Is there a tool that provides a list including targeted keywords and there ranking for those keywords? For example if I type in a website url it shows me which keywords they targeted, on which page and their ranking position. Thanks for helping out a foreign noobie;)
Moz Bar | | grobro0 -
On-Page Grader not working?
I've been using the On-Page Grader tool for a while now, and have always had great success using it to better optimize my pages. But just recently I've noticed that it's not detecting any changes I make to any of my pages.
Moz Bar | | CMinc
For example: One of my pages was coming up with Exact Keyword is Used in Page Title. So I changed it and uploaded it to the site and then clicked the re-grade page button. The only problem is that it's not seeing any of the changes that I made.1 -
Keyword volume
I started using the keyword difficulty tool, I can see the keyword difficulty but how can I see the search volume of each keyword?
Moz Bar | | amielsosa0 -
Change an "Answered" question to "unanswered?"
Some times Moz marks my questions as 'answered" when they are not. I know that if I mark something as a "good answer" or select "answered" (obviously) it will change, but occasionally (like the last question I asked) it gets changed and I'm not sure how to change it back to "unanswered." Anyone know if that's possible? Thanks, Ruben
Moz Bar | | KempRugeLawGroup1 -
Need a Reliable Free Bulk Keyword Rank Checker
I have 2,871 keywords that I need to check Google rank for on 4 separate domains. Does anyone know of any FREE tools or plugins available that will allow for this volume that wont get my IP banned by Google? Even the Moz Rank Checker only allows to enter 1 keyword at a time for up to 200 per day. Who would seriously enter in 200 keywords, one by one all day, every day?
Moz Bar | | dsinger0 -
How is this possible, Keyword Difficulty tool has some problems
Hello guys, I have some concerns or misunderstanding. If I am about to look for [software development] using Google first, it shows that the Exact match for the keyword is 16,574, then I choose Bing and I had the same number 16,574. Yahoo is the showing the same figure. Do I miss something in here or this tool doesn't work? odd_zps2388b5a9.png
Moz Bar | | Askalexis0