Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

My website does not allow all crawler to crawl, Now my question is that whether i need to give permission to moz crawler if yes then whaat is moz bot name?

Getting Started

1565

irteam last edited by

My website does not permit all crawler to crawl website. Whether ii need to give permission to moz bot to crawl website or not? If yes what is the moz bot name?
1 Reply Last reply
Reply Quote 0
MonicaOConnor last edited by

I don't know that you need to add permission, can you add a picture of your robots.txt file?
1 Reply Last reply
Reply Quote 1
Alick300 last edited by

Hi,

moz bot name is 'Rogerbot'.

Thanks
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Page Count per campaign - Crawl Usage 500,000 Pages

How to you find the page crawl count per campaign? I have 3 campaigns and Moz stats I have used 150,000 pages from 500,000. I want to check this. Thanks
Getting Started | | SJMDT

0
Weekly Introductory Webinar - NEED RECORDED VERSION

Hi there, I'm looking to give my entire company an introduction to Moz software. I would like to show them a recorded version of the introductory webinars you host every Friday. I need a recorded version because our team meetings are on a Monday. Please can you point me in the right direction?Thanks!Mike
Getting Started | | The_Mezzanine_Group

1
Moz Private Message Restriction

Why does Moz only let you send 2 private messages a day? I'm a Moz Pro subscriber! Sorry Message Not Sent You are currently over your quota of 2 threads per day.
Getting Started | | deelo555

0
Campaign.crawl-seed.bad-response

I am trying to set up a new campaign for a website, but I keep getting this error message... campaign.crawl-seed.bad-response 😞 I have no idea what the problem is. Can you tell me what I am suppose to do to fix this? The URL I am trying to set up is www.aboutplcs.com
Getting Started | | ChadC

0
Changes to Moz

I used to think Moz was rich and full but man, there is so Much new stuff! I love it! I still need to explore everything. Does anybody know if its going to be an option for our clients to log in and see just their account? are changes in the works for reports we can pull and send to clients? thank you!! Matthew
Getting Started | | Mrupp44

1
How to give access, invite additional users?

We have Standard subscription to Moz Analytics. I can't find option to how to give other team members access to view the analytics. I need other team members just to view same analytics profile, no need for them to create or modify profiles. Can anyone point me where do I invite other team members? Thanks.
Getting Started | | romanr

0
How to remove campaign from Moz.com

I've accidentally created a duplicate campaign, which I am unable to delete, please help
Getting Started | | shopperlocal_DM

0
MOZ Starter Crawl not happeneing

Hi I added a new site 48hours + ago and the starter crawler has not even begun collecting data. Any help would be appreciated. cheers Isaac
Getting Started | | sodafizz

0