Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Rogerbot not showing in logs

Moz Pro

856

CraigWiltshire last edited by

Hi All

Rogerbot has recently thrown up 403 errors for all our pages - no changes had been made to the site so I asked our ISP for assistance.

They wanted to have a look at what rogerbot was doing and so went to the logs but rogerbot was not listed anywhere in the logs by name - any ideas why?

Regards

Craig
1 Reply Last reply
Reply Quote 0
kenneth_martin last edited by

Hey Craig,

Sorry about the issues you are experiencing with the crawler. I just opened up a customer support ticket for you so we can investigate this on our end to see what's going wrong.

My apologies for the inconvenience!

Kenny
1 Reply Last reply
Reply Quote 0
Adam.Whittles last edited by

Have you performed a crawl test? That might give you a clearer idea of what's going on.
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Rogerbot's crawl behaviour vs google spiders and other crawlers - disparate results have me confused.

I'm curious as to how accurately rogerbot replicates google's searchbot I've currently got a site which is reporting over 200 pages of duplicate/titles content in moz tools. The pages in question are all session IDs and have been blocked in the robot.txt (about 3 weeks ago), however the errors are still appearing. I've also crawled the page using screaming frog SEO spider. According to Screaming Frog, the offending pages have been blocked and are not being crawled. Webmaster tools is also reporting no crawl errors. Is there something I'm missing here? Why would I receive such different results. Which one's should I trust? Does rogerbot ignore robot.txt? Any suggestions would be appreciated.
Moz Pro | | KJDMedia

0
Site Explorer shows links as followable but they have nofollow tags

Hello, I am looking at site explorer and sites linking to my site moneyfact.co.uk. I've got thousands of links showing as 'followable' but when i check them they have rel="nofollow" tags. e.g: http://www.dianomioffers.co.uk/partner/moneyfacts.co.uk/brochures.epl?partner=93&partner_id=93&partner_variant_id=33 Why would they show as followable when the links are nofollowed? Thanks Steve
Moz Pro | | SteveBrumpton

0
No internal links showing up in OpenSiteExplorer report

Hi there A simple one. I was just looking at our site in opensiteexplorer and notice that it says we have no internal links at all: https://www.evernote.com/shard/s244/sh/dd8cc88f-fee4-4ba2-8ec0-f3f8d5c408db/8d3fc3d2aa6fc9d5406051bc91731402 Rather odd as we do. Is this a bug with OSE that anyone knows about or something else? Any ideas / thoughts gratefully received.
Moz Pro | | ArenaFlowers.com

0
Can't log into Firefox MozBar

I just downloaded and installed the MozBar for Firefox, but it will not let me login to my account. The Log In button is gray and none of the buttons do anything when I click on them. Please help! Thank you,
Meghan
Moz Pro | | Instabill

0
What is the correlation between the SEOMOZ PR number and where my site is showing up in an organic search?

For years, our company placed 3 or 4 in an organic search on Google for the keyword I am inquiring about. A couple of months ago, we went down to number 15. I then subscribed to SEOMOZ and began a concerted effort to improve our ranking for this keyword. Slowly, our listing came up higher in searches and on our reports- back onto the first page #8. Our weekly SEOMOZ reports reflected this improvement, but then it dropped down again to page 2, #11. My most recent report, this week, gave us a PR 8 for this keyword, but our listing is showing upon page 2, #11. I do have my personal search and browser history disabled. I am unclear as to why there is this discrepancy.
Moz Pro | | gfiedel

0
Added to product title showing the discount, will it affect the product rankings?

I've added the savings info to a few products to see if I get any more visits from google merchant, example; After Party | Save £5.40. will it harm the products ranking or merchant feed? http://www.thehairroom.co.uk/tigi-bed-head-styling-products/tigi-bed-head-after-party.htm Thank you Mark
Moz Pro | | smoki666

0
Why does SEO moz show so many more duplicate page titles than webmaster

My SEO moz duplicate page titles are more than double the webmaster duplicate page titles. Plus, some of the items appearing in SEO moz are not really duplicates. Anyone else seeing this?
Moz Pro | | Stamats

0
Why would opensite explorer show a link from baidu result page with baidu as a linking root domain?

I found an old domain that has very high domain authority and one of its top linking root domains is baidu I clicked the link in ose and it took me to a baidu serp????? please explain...im happy to clarify if need be or give you the site in question just let me know what you think. I checked ose for the page authority of the serp page it was 50. would that page be helping the site in question to rank? or am I just dumb for asking this question One more thing I couldnt actually find a link to the page I was looking at in ose on the baidu result page.. thanks
Moz Pro | | duncan274

0