Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

My website does not allow all crawler to crawl, Now my question is that whether i need to give permission to moz crawler if yes then whaat is moz bot name?

Getting Started

1565

irteam last edited by

My website does not permit all crawler to crawl website. Whether ii need to give permission to moz bot to crawl website or not? If yes what is the moz bot name?
1 Reply Last reply
Reply Quote 0
MonicaOConnor last edited by

I don't know that you need to add permission, can you add a picture of your robots.txt file?
1 Reply Last reply
Reply Quote 1
Alick300 last edited by

Hi,

moz bot name is 'Rogerbot'.

Thanks
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Changing the Moz Crawl Date

Hello, I am wondering where I can change the date of Crawl by Moz. I would like to change this crawl period from one week to 2 or even 3 weeks for Moz to crawl my website. Hope to hear from anyone soon. Kind regards, Koen.
Getting Started | | Koenniiee

1
Can anyone tell me about spam score of website

how can i check spam score and tf cf of my mygiftcardsite
Getting Started | | johnvella

0
What does Location National means in Moz Pro Campaigns?

I understand the concept of locations in Moz Pro Campaigns. What does National exactly means? Is it like globally for a United States Google Search Rank or what? Please explain.
Getting Started | | ds9.tech

1
Moz can't crawl my site.

Moz cannot carry out the site crawl on my online shop. Not really sure what the issue is, it has no problem getting onto my site when you use www. before the address, but it needs to be able to access bluerinsevintage.co.uk Stuck as what to do, we are a shopify store. Anyone else had this problem, or know what i need to change so they can crawl the site? thjis is the page they are getting when trying to get on bluerinsevintage.co.uk but if they use www.bluerinsevintage.co.uk the site comes up. Adam
Getting Started | | bluerinsevintage

0
Can these question be deleted?

No point having it on here if it doesn't get answered. Thanks
Getting Started | | CamperConnect14

0
New to Moz Pro? Join our latest free webinar this Friday!

Hello everyone! We'll be holding a webinar on tomorrow to help new members learn about what all Pro has to offer, show some off our most popular tools, and get you comfortable with the dashboard. Register here: https://www3.gotomeeting.com/register/355765902 Date: Friday, September 26th (this Friday!) Time: 10:00 AM - 11:00 AM PDT Hope to see you all there!
Getting Started | | jennita

1
Can you help me with setting up a campaign? I get stuck after I put in the domain name and the other information doesn't go through.

Can you help me with setting up a campaign? I get stuck after I put in the domain name and the other information doesn't go through. Regards Roshni M
Getting Started | | harmanseo

0
MOZ Starter Crawl not happeneing

Hi I added a new site 48hours + ago and the starter crawler has not even begun collecting data. Any help would be appreciated. cheers Isaac
Getting Started | | sodafizz

0