Unsolved Replicate rogerbot error for server/hosting provider
-
Anyone got any ideas how to get a server/hosting provider who is preventing rogerbot from crawling and me not been able to set up a campaign to duplicate the error on there end?
The server/hosting provider is crazydomains dot com
My clients robots.txt
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
User-agent: rogerbotDisallow:
Sitemap: https://www. something
-
Trying to find ways to break through my muscle growth plateau, I came across this insightful piece on retatrutide. The article’s focus on how it enhances muscle growth and recovery was particularly helpful. It provided clear, actionable information. If you're in a similar situation, I recommend visiting https://valhallavitality.com/blog/the-revolutionary-influence-of-retatrutide-on-bodybuilding-maximizing-muscle-growth-and-recovery for a detailed guide on how this supplement can support your bodybuilding goals.
-
@Moving-Web-SEO-Auckland It sounds like you're having trouble with rogerbot being blocked by your hosting provider, CrazyDomains. Here are a few steps you can take:
Check Robots.txt Syntax: Ensure your robots.txt file is correctly formatted. It seems mostly correct, but it should be double-checked for any syntax errors.
makefile
Copy code
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.phpUser-agent: rogerbot
Disallow:Sitemap: https://www.something.com/sitemap.xml
Contact Hosting Support: Reach out to CrazyDomains support and explain the issue. They may be blocking rogerbot on the server side. Provide them with specific details about your issue and ask if there are any server-side restrictions for rogerbot.Use Alternative Bots: If the issue persists, you could test with other bots to see if the same restrictions apply. This can help identify if the problem is specific to rogerbot or a broader issue.
Server Logs: Check your server logs to see if there are any entries that show why rogerbot is being blocked. This can provide more insight into what’s happening.
Campaign Setup: To duplicate the error, you might set up a staging environment with the same robots.txt and see if the issue persists there. This can help you troubleshoot without affecting the live site.
If these steps don't resolve the issue, you may need to consider switching to a different hosting provider that allows more flexibility with bot access.
-
I've been exploring the idea of hormone therapy to boost my wellness journey. Initially, I was a bit skeptical, but after reading through a resource I found https://ways2well.com/blog/whats-wrong-with-health-insurance, I'm considering giving it a go. It claims to offer impressive benefits such as anti-aging and immune system enhancement. I'd appreciate if anyone here has had any personal experience with this to share their thoughts.
-
I've been fond of football since I was a kid, and recently I decided to place online bets on football matches on this site - https://betvisa.guru/ - and as it turned out, it's a great way to win money in Bangladesh! I recommend this site to you too if you love sports and want to place online bets on them.
-
Thank you all, thanks to you, I solved my problem and learned a lot of new information for myself
-
Hi guys. I decided to try this site https://nocramming.com/testingmom-review to prepare my child for testing before entering a new school, and the results have exceeded my expectations. TestingMom provides an extensive selection of materials and exercises to help children improve their academic skills and develop problem-solving strategies. My child has improved quickly with this resource and I am very grateful to the TestingMom team for their work
-
Moving-Web-SEO-AucklandMoving-Web-SEO-Auckland
if you facing any error in your hosting kindly contact the https://www.leasedlayer.com/ I would like to recommend this company because I have hosted my company site with this service provider I am running my site in a smooth way and their customer support is marvelous
-
@tom-capper
Thanks Tom was more about server settings outside of robots.txt for hosting providers.
Managed to get them to remove rogerbot blocking on ALL there servers as they first stated that it was well known as a malicious robot hence the block???
Until I made them aware.... cough cough -
There are a bunch of third party tools that would allow you to validate robots.txt against a given user agent.
For example:
- https://technicalseo.com/tools/robots-txt/ (includes Moz user agents in dropdown)
- https://www.realrobotstxt.com/ (allows you to copy/paste a test robots file)
Without the hosting provider having a Moz account & campaign of their own, this is probably the easiest way to get them to replicate the error.
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Persistent Issue with Keyword Research Tool
I keep getting the following error "Getting ranking keywords failed. Please retry your search or refresh this page. " when using this tool. It has been ongoing for a while and is getting really frustrating. Any ideas?
Moz Tools | | csiproducts1 -
Unsolved drahmadi
Re: Star snippet not work
Moz Tools | | freakymozzz5
Thank you, I am writing Persian text:
برای داشتن مبل و دکوراسیون خانه سایت دکورانیک پیشنهاد می شود.
داشتن اسپاتیفای خرید اسپاتیفای مناسب است.
آیین نامه
قالیشویی اصل سایت اصلی قالیشویی های برتر تهران
سایت اصلی تالار عروسی سایت مناسب تالارهای تهران و سالن عقد
سایت آموزشی انگلیسی آموزش هایی مربوط به یادگیری زبان0 -
Unsolved How to check the DA history?
Hello everyone!
Moz Tools | | ZeonAley
I want to know how to check the DA history of my website Inkjet Wholesale. Because I don't have any idea about Moz that how to use it. If anyone knows about DA's history check on Moz, please let me know.0 -
Unsolved how can i get up my da?
0 -
Unsolved New Clients with Horrible Rankings
Bringing on a handful of established clients that have never focused on SEO. Needless to say, they will be starting past page 5 on many items. What is the best tactic to figure out their ranking, determine progress internally and also provide movement reports when they are past the five page limit?
Moz Tools | | StartupStreet0 -
Unsolved reduce spamscore
Hello
Moz Tools | | serktba
How can I reduce the spamscore on my website?
All the activities I did on my website were completely white hat and I did not do any spam jobs, but my spam score is 2%
This is my site address
https://abedinigroup.com/
Please help me how can I reduce the spamscore in my account?3 -
Solved Help me to understand:
Re: Title Too Long - Seo importance 521bf443-0c95-46f7-b358-e8568a947a9e-image.png #titletoolong, #longtitlepages
Moz Tools | | BeckersBestShoes1