Restrict rogerbot for few days
-
Hi Team,
I have a subdomain that built in Zendesk's CRM system. Now, I want to restrict Moz crawler (rogerbot) for crawling this complete subdomain for a few days, but I am not able to edit the robots.txt file of the subdomain, because this is a shared file and Zendesk is not allowing to edit it. Could you please let me know the alternative way to restrict rogerbot to crawl this subdomain?
I am eagerly awaiting your quick response.
Thanks
-
Do you have FTP access and / or Yoast installed (if this is a WordPress website)?
Thanks,
Zack -
Not from what I could think about for how a specific user-agent is being treated just like you would want with Rogerbot.
-
Is there any way to restrict via Google Tag Manager?
-
Hey there!
Thanks for reaching out to us!
Unfortunately you would need access to the robots.txt file in order to block us from crawling a certain subdomain. I'm sorry I can't be of anymore help here. Please do reach out to help@moz.com if you have any further questions,
Best,
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Are tags in blogs worth anything in SEO value these days and can i remove duplicate title tags?
I have around 12 duplicate tags in the blog and the general consensus is to noindex./nofollow the tag pages which is not an awful way to do things. Should I just arrange my tags better or remove them all together? I believe it's the fact that the same page can be accessed by 3 different url's because of the tags it's under and that in the tag urls, only teaser descriptions of the posts are being display, not the full post, but the seomoz tools still consider it as duplicated content.
Moz Pro | | SEM_at_Lees0 -
What are the restrictions/limitations to running SEO/Adwords in these countries?
What are the limitations or restrictions to running SEO/Adwords campaigns in countries such as China, South Korea, Japan, Brazil, Portugal, Spain, and Mexico?
Moz Pro | | ThomasCenterInc0 -
Is there a way to view Facebook book stats for last 30 days instead of a week?
I'm trying to Facebook stats (engagement, exposure, etc) for the last 30 days instead of the last week...when I look at Facebook admin stats I'm only seeing the last week. Does anyone know if there is a tool (or a way to do it within Facebook) to see the stats for a 30 day time slot instead of just a week? Thanks!
Moz Pro | | NoahsDad0 -
How can i download more than 10 reports per day (in Excel)? Thanks!
I am using Followerwonk to analyze twitter following. When I tried to download data into Excel files, i realized that there is a limit of 10 downloads per day. Can Moz allow more than 10 downloads per day on a special request? Thanks
Moz Pro | | rajansingh960 -
Its been over a month, rogerbot hasn't crawled the entire website yet. Any ideas?
Rogerbot has stopped crawling the website at 308 pages past week and has not crawled the website with over 1000+ pages. Any ideas on what I can do to get this fixed & crawling?
Moz Pro | | TejaswiNaidu0 -
Data Update for RogerBot
Hi, I noticed that rogerbot still give me 404 for http://www.salustore.com/capelli/nanogen-acquamatch.html refferal form http://www.salustore.com/protocollo-nanogen even I made changes since a couple of week. Same error with one "Title Element Too Short" on our site. Any suggestion on how to refresh it? Best Regards n.
Moz Pro | | nicolobottazzi0 -
Does Rogerbot respect the robots.txt file for wildcards?
Hi All, Our robots.txt file has wildcards in it, which Googlebot recognizes. Can anyone tell me whether or not Rogerbot recognizes wildcards in the robots.txt file? We've done a Rogerbot site crawl since updating the robots.txt file and the pages that are set to disallow using the wildcards are still showing. BTW, Googlebot is not crawling these pages according to Webmaster Tools. Thanks in advance, Robert
Moz Pro | | AC_Pro0 -
Can you change crawl day of week?
Can I somehow sync the day of the week for each of my campaigns' crawls, so that all campaigns are updated on the same day?
Moz Pro | | ATShock0