Keyword Grader & OmniExplorer_Bot Question
-
I found this in several of my clients Robots.txt
"User-Agent: OmniExplorer_Bot
Disallow: /"
Right now I cannot grade any of them that are on a particular platform that has that written in.
Would this block the keyword grader?
Thanks!
-
I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?
-
User-Agent: bingbot
Crawl-delay: 5
User-Agent: msnbot
Crawl-delay: 5
User-Agent: Googlebot-Mobile
Allow: /mobi
User-Agent: OmniExplorer_Bot
Disallow: /
-
Hey,
Mike is correct that our crawler is named RogerBot.
I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.
This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
I hope that helps track down the issue.
Cheers,
Joel. -
Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What does "Optimal Use of Keywords in Header Tags" really mean?
I get this explanation on the grader: Although using targeted keywords in H1 tags on your page does not directly correlate to high rankings, it does appear to provide some slight value. It's also considered a best practice for accessibility and helps potential visitors determine your page's content, so we recommend it. Over-using keywords, however, can be perceived as keyword stuffing (a form of search engine spam) and can negatively impact rankings, so use keywords in H1 tags two or fewer times. To adhere to best practices in Google News and Bing News, headlines should contain the relevant keyword target and be treated with the same importance as title tags. But I have the keyword in an H1 tag, only once. So why am getting this message on the grader? The URL is https://www.armsecuritysystems.com/business-security-systems/ The keyword is "business security systems" Any insight would be greatly appreciated.
Moz Bar | | bizmarquee1 -
Issue with Keyword List Export
I am having an issue with the Keyword list export. I exported a keyword list, then went back in a made changes to the list (deleted keywords, changed priority, etc). Then when I re-export the list it still downloads the original list. For instance, I had a keyword list of 120 keywords. I exported it. Then I reduced the list to 30 keywords. Now when I export that list it is still the original 120 keywords. Any one else having this issue? Is there a way to fix this?
Moz Bar | | Sundaram1 -
"Keywords Showing" total doesn't match "Tracked Keywords" total
My "Tracked Keywords" total under "Add & Manage Keywords" does not match the total for "Keywords Showing" on the main "Rankings" tab. Why is there a difference? For example, I have 453 tracked keywords attached to my campaign, but under the "Tracked Keywords Overview," there are only 254 listed. I'm sure there's a simple explanation but I can't find it anywhere!
Moz Bar | | MSoleski0 -
On Page Grader inconsistent
Why does the on page grader not update it's grades based on the other factors, other then the title tag. i.e. I can have the keyword 'burger' in H1 tags, the URL, ALT attributes, in the body text a couple times and in the meta section of the site and receive an F grade from the on page grader, but as soon as I add the word 'burger' to the title I receive an A grade. Is there a reason why the only factor that has that influence is the title tag, and why is it that a keyword for a page cant get say a B or C grade if it has the other factors covered (i.e. they have a tick next to them) but not the title tag? Cheers Again
Moz Bar | | sharpleaddesign0 -
Q&A Question showing as answered when it is not
Yeh, that question at http://moz.com/community/q/onpage-grader-says-site-unreachable is not actually answered even though someone has marked it as such. Don't know how to mark it as not. Just annoying, as it may prevent someone else chime in with new information now I have updated it with some more info myself as I am just trying to get this resolved.
Moz Bar | | TheWebMastercom0 -
Create a report with keyword, label, difficulty, global search volume, and ranking?
Is it possible to create a report with containing keyword, label, difficulty, global search volume, and ranking? Currently in order to get the data, it seems like I need to manage two lists, the keyword list we are tracking and the keyword list in the Difficulty tool then somehow manually combine the data. Is there an easier way?
Moz Bar | | promfgsystems2 -
Inconsistent On-Page Grader results
Hello everyone! I am fairly new to the MOZ tools but I'm finding some inconsistencies in the On-Page grader.. I have tested several pages (some of which I know have doos and bad SEO) and the results I keep getting are very volatile. For example, I have looked at testing http://www.sportingwebdesign.com for "Sports Website Design" - one minute it gave me a grade F, then then 2 minutes after I refresh the page and it gave me a grade A.. I am very confused and a bit worried that this tool may be a bit volatile... Anyone have any thoughts?
Moz Bar | | Social_Media_Mentors0 -
Moz Keyword Ranking Report
Hi, Moz Keyword Ranking report shows different results then what i see on my desktop for some keywords. There wasn't much difference on most of them until now so ignored some of the differences but now i see huge differences. For instance, on my Google UK I see the keyword ranking 1 or 2 but Moz reports show 22 or higher than that . Search Engine is Google UK. Would you please advise if I am doing anything wrong here? Thank you,
Moz Bar | | Rubix0