Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Unsolved Replicate rogerbot error for server/hosting provider
-
Anyone got any ideas how to get a server/hosting provider who is preventing rogerbot from crawling and me not been able to set up a campaign to duplicate the error on there end?
The server/hosting provider is crazydomains dot com
My clients robots.txt
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
User-agent: rogerbotDisallow:
Sitemap: https://www. something
-
thanks guys for this post
-
A friend of mine told me about online gaming, and I got curious. I never tried these kinds of sites before, so I figured I’d check it out. I wanted something that wasn’t too complicated, and that’s when I came across 1win-game.co.in. The site looked cool and was easy to navigate, so I gave it a shot. They’ve got a bunch of cool offers like sign-up bonuses and special promotions, which made me feel like I was getting some extra value right off the bat. I explored a few games, and the whole experience was smooth and enjoyable. The bonuses were easy to understand and use, which was perfect for a beginner like me. What really stood out was how everything was straightforward and hassle-free. I’m glad I found this site.
-
вфівіфвіфвфівфів
-
hello i like this post !
-
his thing is great thanks for help
the information provided,
they helped me a lot Edu connect. I hope you have more information Great blog. -
I'm glad you found the information helpful on. Thanks for the information provided,
they helped me a lot trust I hope you have more information Great blog. -
@sam0090 I am a student myself and don't know much about Math but after doing a lot of research I came across a reliable website called calculadora-de-horas(https://calculadora-de-horas.com/como-calcular-44-horas-de-trabalho-de-segunda-a-sexta-feira/) from here I got information about calculators. I. In this, I learned about the difficult calculation, and here is a noteworthy point I had such problems with my site too, I corrected it after seeing your reply. thanks
-
-
Webugol approached our project with a strategic mindset, ensuring that every aspect of the website aligned with our business goals. Their effective planning and execution have resulted in a site that not only looks great but also drives our business forward. We are extremely satisfied with the results and grateful for their hard work. Let's check this site https://webugol.com/blog/innovative-marketing-ideas-for-chiropractors-to-stand-out
-
@sam0090 Certainly! The issue with Cookiebot, GTM, and Typo3 seems to be related to script blocking. Make sure essential scripts for your hover menu are correctly categorized in Cookiebot’s settings to prevent them from being blocked. Also, review GTM’s integration to ensure it adheres to Cookiebot's consent settings.
-
@monkaw said in Replicate rogerbot error for server/hosting provider:
I've been fond of football since I was a kid, and recently I decided to place online bets on football matches on this site - https://betvisa.guru/Slope Game - and as it turned out, it's a great way to win money in Bangladesh! I recommend this site to you too if you love sports and want to place online bets on them.
Thank monkaw. This is great advice and you are very generous in sharing it.
-
When my AC started acting up, I was unsure how to handle it. I found https://serviceservotech.com/air-conditioning/ac-repair-installation/ and was impressed by their comprehensive advice on both repair and installation. The information was clear and easy to follow, allowing me to address the issues effectively. If you need reliable guidance for your air conditioning system, this site is a great resource.
-
Trying to find ways to break through my muscle growth plateau, I came across this insightful piece on retatrutide. The article’s focus on how it enhances muscle growth and recovery was particularly helpful. It provided clear, actionable information. If you're in a similar situation, I recommend visiting https://valhallavitality.com/blog/the-revolutionary-influence-of-retatrutide-on-bodybuilding-maximizing-muscle-growth-and-recovery for a detailed guide on how this supplement can support your bodybuilding goals.
-
@Moving-Web-SEO-Auckland It sounds like you're having trouble with rogerbot being blocked by your hosting provider, CrazyDomains. Here are a few steps you can take:
Check Robots.txt Syntax: Ensure your robots.txt file is correctly formatted. It seems mostly correct, but it should be double-checked for any syntax errors.
makefile
Copy code
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.phpUser-agent: rogerbot
Disallow:Sitemap: https://www.something.com/sitemap.xml
Contact Hosting Support: Reach out to CrazyDomains support and explain the issue. They may be blocking rogerbot on the server side. Provide them with specific details about your issue and ask if there are any server-side restrictions for rogerbot.Use Alternative Bots: If the issue persists, you could test with other bots to see if the same restrictions apply. This can help identify if the problem is specific to rogerbot or a broader issue.
Server Logs: Check your server logs to see if there are any entries that show why rogerbot is being blocked. This can provide more insight into what’s happening.
Campaign Setup: To duplicate the error, you might set up a staging environment with the same robots.txt and see if the issue persists there. This can help you troubleshoot without affecting the live site.
If these steps don't resolve the issue, you may need to consider switching to a different hosting provider that allows more flexibility with bot access.
-
I've been exploring the idea of hormone therapy to boost my wellness journey. Initially, I was a bit skeptical, but after reading through a resource I found https://ways2well.com/blog/whats-wrong-with-health-insurance, I'm considering giving it a go. It claims to offer impressive benefits such as anti-aging and immune system enhancement. I'd appreciate if anyone here has had any personal experience with this to share their thoughts.
-
I've been fond of football since I was a kid, and recently I decided to place online bets on football matches on this site - https://betvisa.guru/ - and as it turned out, it's a great way to win money in Bangladesh! I recommend this site to you too if you love sports and want to place online bets on them.
-
Thank you all, thanks to you, I solved my problem and learned a lot of new information for myself
-
Hi guys. I decided to try this site https://nocramming.com/testingmom-review to prepare my child for testing before entering a new school, and the results have exceeded my expectations. TestingMom provides an extensive selection of materials and exercises to help children improve their academic skills and develop problem-solving strategies. My child has improved quickly with this resource and I am very grateful to the TestingMom team for their work
-
Moving-Web-SEO-AucklandMoving-Web-SEO-Auckland
if you facing any error in your hosting kindly contact the https://www.leasedlayer.com/ I would like to recommend this company because I have hosted my company site with this service provider I am running my site in a smooth way and their customer support is marvelous
-
@tom-capper
Thanks Tom was more about server settings outside of robots.txt for hosting providers.
Managed to get them to remove rogerbot blocking on ALL there servers as they first stated that it was well known as a malicious robot hence the block???
Until I made them aware.... cough cough -
There are a bunch of third party tools that would allow you to validate robots.txt against a given user agent.
For example:
- https://technicalseo.com/tools/robots-txt/ (includes Moz user agents in dropdown)
- https://www.realrobotstxt.com/ (allows you to copy/paste a test robots file)
Without the hosting provider having a Moz account & campaign of their own, this is probably the easiest way to get them to replicate the error.
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved 403 errors for assets which work fine
Hi,
Moz Tools | | Skites2
I am facing some issue with our moz pro account
We have images stored in a s3 buckets eg: https://assets2.hangrr.com/v7/s3/product/151/beige-derby-cotton-suit-mb-2.jpg
Hundreds of such images show up in link opportunities - Top pages tool - As 403 ... But all these images work fine and show status 200. Can't seem to solve this. Thanks.0 -
Unsolved How to check the Domain Age?
Check the Domain Age on THE SEO TOOLS KING
Moz Tools | | seotoolsking
Get Free seo tools only on THE SEO TOOLS KING4 -
Unsolved Rogerbot blocked by cloudflare and not display full user agent string.
Hi, We're trying to get MOZ to crawl our site, but when we Create Your Campaign we get the error:
Moz Pro | | BB_NPG
Ooops. Our crawlers are unable to access that URL - please check to make sure it is correct. If the issue persists, check out this article for further help. robot.txt is fine and we actually see cloudflare is blocking it with block fight mode. We've added in some rules to allow rogerbot but these seem to be getting ignored. If we use a robot.txt test tool (https://technicalseo.com/tools/robots-txt/) with rogerbot as the user agent this get through fine and we can see our rule has allowed it. When viewing the cloudflare activity log (attached) it seems the Create Your Campaign is trying to crawl the site with the user agent as simply set as rogerbot 1.2 but the robot.txt testing tool uses the full user agent string rogerbot/1.0 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+shiny@moz.com) albeit it's version 1.0. So seems as if cloudflare doesn't like the simple user agent. So is it correct the when MOZ is trying to crawl the site it uses the simple string of just rogerbot 1.2 now ? Thanks
Ben Cloudflare activity log, showing differences in user agent strings
2022-07-01_13-05-59.png0 -
Unsolved Why Moz shutdown on seprate website ?
My moz shutdown on some website like my website -
Moz Tools | | yuvrajsindhal
https://shotsblastingmachine.com/0 -
Unsolved error in crawling
hello moz . my site is papion shopping but when i start to add it an error appears that it cant gather any data in moz!! what can i do>???
Moz Tools | | valigholami13860 -
Solved how to do seo audit
hello . i am a user that runs parsp website . i really need to know how to run a site audit to keep my job clean and my site works well ! i need help and i am a newbie in this job . thanks moz !
Moz Tools | | valigholami13863