Rogerbot will not crawl my site! Site URL is https but keep getting and error that homepage (http) can not be accessed. I set up a second campaign to alter the target url to the newer https version but still getting the same error! What can I do?
-
Site URL is https but keep getting and error that homepage (http://www.flogas.co.uk/) can not be accessed. I set up a second campaign to alter the target url to the newer https://www.flogas.co.uk/ version but still getting the same error! What can I do? I want to use Moz for everything rather than continuing to use a separate auditing tool!
-
It will probably help if you delete the following line from your robots.txt file:
User-Agent: rogerbot
Disallow: /Dirk
PS You also block /js/ for all bots - Google doesn't really recommend that
-
I just check and see that http is 301 to https. But https works.
Can you mail this issue to help@moz.com?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz on-page grader can't see keyword in H1
Hi, Trying to do grade for whatiswhere.com, keyword 'poi search'. The tool can't see keyword in h1, but It is there. Could you please check what is the problem? Thank you, Andrei.
Moz Bar | | Anazar_20010 -
Tracked Ranking Keywords in Campaign and Ranking Keywords in Keyword Explorer are coming up with very different results - does anybody know why?
We have a campaign set up that is tracking keywords with varying monthly volumes and the website is ranking for them. I have gone into the new Keyword Explorer Feature (looks great!) and I have tested it by putting in the root domain address mentioned above. My problem is that none of the keywords that I am tracking and ranking for are appearing in the "ranking keywords" report in Keyword Explorer. Does anybody know why this might be happening?
Moz Bar | | RedAntSolutions0 -
Moz Bar doesn't show any data and keeps asking me to log in when actually I'm logged in.
Hi all, I've been using Moz Bar for years. It ran well until about three weeks ago. It suddenly failed to show the DA and PA of sites that I open after I log in. And it keeps asking me to log in when I did. I tried to uninstall the Mozbar extension and reinstalled it several times. Nothing worked. I also tried to uninstall Chrome and clear the cookies, still, nothing changed. Did anyone experience this? How do you solve it and make it run on the track? Any information will be appreciated. [admin edited support category]
Moz Bar | | Bennie22339 -
What does the external links column mean in the crawl report , thanks
Hi, Ran a report for www.dare2b.com report, and it showing 34780 external links. What does this mean Thanks Jeff
Moz Bar | | jefffox0 -
How do you block keywords in On-Page Grader for certain URLs?
For the on-page grader, I rank A's for 9 keywords. I struggle with F's I have because it is searching keywords on pages that are not supposed to be searched. For example, I have a "bracelets" page, so I didn't optimize it for "rings", so I got an F. However, it graded me an A for the bracelets keyword, which is great. To be sure I am correct, each page should have it's own keyword, such as bracelet. So why is the grader checking my "Bracelets" page for "Rings"? Maybe there is something I am missing. Just trying to see why this happening.
Moz Bar | | tiffany11030 -
Crawl Test cannot be seen on my PC. Using Windows 8.
I received and downloaded my Crawl Test. When I try to open it, my pc says "This app can't run on your PC. To find a version for your PC, check with the software publisher". I'm running Windows 8. Can I view my Crawl Test with my PC? Is there a work-around for this issue? Update I can apparently open my Crawl Test and view it as an Excel Spreadsheet. But when I download it and choose Save As, it saves it as a MS-DOS Application. This is my very first Crawl Test and I am not sure if I am doing everything right.
Moz Bar | | jameskoby010 -
Moz not crawling opencart product pages
hi, i have waited for over 2 weeks now and the crawler only got 8 pages, and is not getting all the open cart pages and products. any idea of what can be wrong? im using joomla 2.5.11 and mijoshop 2.0.5 (which uses opencart 1.5.5.1). thanks
Moz Bar | | marlvass10 -
Where's Rogerbot?
Could someone please tell me where Rogerbot lives now?! Unless I'm having distorted memories, I used to be able to crawl websites with Rogerbot (that are not set-up as Campaigns). Could someone please let me know where to find this now? p.s. I didn't really want to Q&A this, but after a while clicking around moz I'm now even questioning myself!
Moz Bar | | GregDixson0