Keyword Grader & OmniExplorer_Bot Question
-
I found this in several of my clients Robots.txt
"User-Agent: OmniExplorer_Bot
Disallow: /"Right now I cannot grade any of them that are on a particular platform that has that written in.
Would this block the keyword grader?
Thanks!
-
I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?
-
User-Agent: bingbotCrawl-delay: 5
User-Agent: msnbotCrawl-delay: 5
User-Agent: Googlebot-MobileAllow: /mobi
User-Agent: OmniExplorer_BotDisallow: /
-
Hey,
Mike is correct that our crawler is named RogerBot.
I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.
This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
I hope that helps track down the issue.
Cheers,
Joel. -
Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Issue with Keyword List Export
I am having an issue with the Keyword list export. I exported a keyword list, then went back in a made changes to the list (deleted keywords, changed priority, etc). Then when I re-export the list it still downloads the original list. For instance, I had a keyword list of 120 keywords. I exported it. Then I reduced the list to 30 keywords. Now when I export that list it is still the original 120 keywords. Any one else having this issue? Is there a way to fix this?
Moz Bar | | Sundaram1 -
What is a Good Keyword Priority Score?
Howdy gang, This is my last discussion post in the series on keyword metrics in KW Explorer & Moz Pro (previously on Keyword Difficulty, Opportunity, & Volume). In this one, let's chat about the "Priority Score," a feature you'll find in Keyword Explorer on any lists you build. Priority was conceived to help aggregate all the other metrics - Difficulty, Opportunity, Volume, and (if you choose to use it) Importance. We wanted to create an easy way to sort keywords so the cream would rise to the top -- cream in this case being keywords with low difficulty, high opportunity, strong volume, and high importance (again, if you choose to use it). Thus, when it comes to Priority Score, there's no particular number you should necessarily seek out, but higher is better. When you get into the ranges of 80+ (which is quite rare, Single Malt Scotch is one of the few examples I could find, and only because it's volume is so high and there's only a couple SERP features), you're generally talking about keywords with high demand (lots of monthly searches), the difficulty isn't too crazy (a website in the 55-80 DA range might have a shot), and the CTR Opportunity is decently strong (usually not too many SERP features that take clicks and attention away from the organic web results). Below that score range, you're usually finding keywords where one or more of those isn't true -- there's either lower volume, heavier competition, or lots of SERP features with the accompanying lower estimated CTR. When you're building KW lists, my view is that there's no "good" or "bad" Priority scores, only relative scores. Priority should be used to help you determine which terms and phrases to target first -- it's like a cheat code to unlock the low hanging fruit. If you build large lists of 50-100 or more keywords, Priority is a powerful and easy way to sort. It becomes even more useful if you use the Importance score to help add an estimation of value to you/your business/your client in to the mix. In that case, Importance can cut Priority by up to 2/3rds (if you set it at 1) or raise it by a little more than 3X (if you set it at 10). This is hyper-useful to nudge keywords with middling scores up if they're super-important to your marketing efforts. Look forward to your feedback, and thanks for checking these out!
Moz Bar | | randfish8 -
Learn how to use Rank Tracker to analyze rank progress for any given keyword. Get your Daily SEO Fix!
Moz's Rank Tracker tool retrieves search engine rankings for pages and keywords, and stores them for easy comparison later. With it, you can analyze and track rank progress for a keyword or set of keywords that you may not want to include within a Moz Analytics campaign. In today’s Daily SEO Fix, James dives into the tool to show you how to use Rank Tracker to start keeping tabs on the ranking history of a given keyword. This video is part of The Moz Daily SEO Fix tutorial series--Moz tool tips and tricks in under 2 minutes. To watch all of our videos so far, and to subscribe to future ones, make sure to visit the Daily SEO Fix channel on YouTube.
Moz Bar | | kellyjcoop1 -
How does the grader tool treat keyword "stuffing" in ecommerce
We recently started using Moz on our ecommmerce site because I'm concerned that our SEO company doesn't really know what they are doing and I want to see what I can do on my own with the little bit of knowledge I have. It's helping in a number of ways but here's a big question mark: The Grader Tool keeps telling us that our product category pages have too many keywords on them. We are only using them in the content once or twice, but the sub-category buttons on the page show the category + sub repeated. Could this be what's causing it? Does Google distinguish this for ecommerce? We've taken a huge hit in rankings for key phrases and keywords over the past six months and I'm wondering if this is part of it?
Moz Bar | | Creative-Web-Stores0 -
Keyword Rankings vs. Rank Tracker research tool
I have a campaign tracking several keywords. On my Keyword Rankings report I'm not in the top 50 for anything. When I use the Rank Tracker tool, my rankings are quite high (I have 2 in the top 5). When I use the Rank Tracker tool I've noticed that I'm able to point to specific URLs on my website (not the main page). When looking on my Keyword Rankings report, the URL is usually empty. So what am I missing? Have I configured something incorrectly? Can I update the URLs on the Keyword Rankings report so the rankings will be reflective of the Rank Tracker tool? OSQdLmb nMnOQim
Moz Bar | | roywest630 -
Keyword Difficulty Search Engine Selection
I'm using the Keyword Difficulty tool. The results generated are Bing search engine results, even though I have selected Google for the search engine. I cannot get this feature to work, and I've tried it in both Chrome and Firefox. What's going on here...??
Moz Bar | | AcroPilot1 -
Rankings Tool Shows 'Wed' for Most Keywords
Hi Folks, I have 300 keywords in the moz rankings tool but I would say over 90% of them simply show a 'WED' icon instead of the icon. It has been like this for nearly 3 weeks now. Any ideas why this is? Cheers Gaz
Moz Bar | | PurpleGriffon0
