Server blocking crawl bot due to DOS protection and MOZ Help team not responding
-
First of all has anyone else not received a response from the help team, ive sent 4 emails the oldest one is a month old, and one of our most used features on moz on demand crawl to find broken links doesnt work and its really frustrating to not get a response, when we're paying so much a month for a feature that doesnt work.
Ok rant over now onto the actual issue, on our crawls we're just getting 429 errors because our server has a DOS protection and is blocking MOZ's robot, im sure it will be as easy as whitelisting the robots IP, but i cant get a response from MOZ with the IP.
Cheers, Fergus
-
Hi dave thanks for your response ill let our server provider the user agent.
I was contacting you using the contact form, putting in my email which is the same as the one used to log into moz, but not the email that was used when creating the account, which i think might've been the issue.
-
Hi Dougal!
We don't have any record of support requests coming from the email associated with your Moz Pro account. Can you let us know how you are reaching out (contact form or emailing us directly at help@moz.com)?
Unfortunately, we do not use a static IP address or range of IP addresses, as we have designed our crawler to have a dynamic approach. This means we use thousands of dynamic IP addresses which will change each time we run a crawl. This approach gives us the best dynamic view of the web, but it can make us incompatible with some servers or hosting providers. The best option I'd be able to suggest would be to identify our crawler by User-agent: rogerbot
You can read more about rogerbot in our guide.
Look forward to hearing back!
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Localized pages with hreflang markup being reported by Moz as duplicate content
We have 5 websites, each targeted at a different geography. They are all in English, but targeted per country (e.g. US, Canada, UK, Australia). And we've properly implemented hreflang tags on each page, with the US site being the x-default, and with each country-specific page having a self-referencing hreflang tag as well as individual hreflang tags pointing to the other country-specific versions. All seems to be working properly with the search engines. But, each week in our Moz Pro campaigns, Moz reports the pages as "duplicate content". It seems that Moz does not regard the hreflang tags when deciding if content is duplicate or not. I'm not 100% sure that's what's causing Moz to report them as duplicate, but it's my best guess. To date, I've been marking these as "ignore". But, that creates two problems. First, we have new pages all the time, and so this gets to be laborious. Second, it makes it somewhat likely we might miss a real duplicate content issue. Can someone confirm whether Moz should be looking at hreflang tags before considering pages as dupliacte? And possibly offer any suggestions to us if Moz doesn't do that?
Feature Requests | | seoelevated0 -
In Moz's Keyword Explorer/Explore By Site, Can I Exclude a Keyword From the Results?
Hypothetical question - Let's say I sell tax preparation software and I want to see the keywords that "turbotax.com" ranks highest for. Based on that list I might want to optimize for similar keywords on my site. But if I use Keyword Explorer, select "Root Domain", and enter "turbotax.com" as the root domain, the first 200 or so keywords in the list have the word "turbo". None of those interest me. Can I exclude certain keywords (like "turbo") from the list? Thanks.
Feature Requests | | CurtisB0 -
Can Moz add an alert to email us when a competitor's site gets a new backlink?
This would be a very useful feature, and other sites are doing this, including Ahrefs.
Feature Requests | | rabbit5190 -
Moz is flagging my callto: links as 404s. I "fixed" with tel: & nofollow and lost 2 DA. Help?
I changed the callto: links to tel: links in the footer and nofollowed them and my domain authority dropped 2. I'm assuming this is because we lost 2.8K "external links" Advice on how to remove 404s for Phone#'s without affecting DA? Will just using the tel: format stop Moz from crawling them as 404's? It was adding links to the 404 list that look like this: www.ocp.org/missals/callto:########### I assumed it was because to the crawlers the links looked like relative links this is what our links look like now: <a <span="" class="html-tag">href</a><a <span="" class="html-tag">="</a>tel:###########" rel="nofollow" title="Call us">########### Old footer links: <a <span="" class="html-tag">href</a><a <span="" class="html-tag">="</a>callto:###########" title="Call us">###########
Feature Requests | | OCPmusic0 -
Moz's New Video Ads on YouTube
I've enjoyed Moz's new video ads and want to share them with colleagues, however am unable to find the vids anywhere other than when they're delivered before a video plays on YouTube. Are they available on demand? If anyone is unfamiliar with the videos I speak of, they are new ads for Moz products like Keyword Explorer, Mozbar, and MozCon. Thanks in advance!
Feature Requests | | Choice0 -
MOZ Campaigns Down?
MOZ Campaigns has been displaying a blank page in the last 2 hours. Is anyone else experiencing this outage? I have attempted to access MOZ via an iPad and an iPhone using Safari and Chrome. My geographic location is France.
Feature Requests | | Kingalan10 -
Suggestion to Improve Moz's Link Data
Hello All. The recent Index update (29th January) seems to have produced quite a few shocks to Moz Pro users. Index updates always cause panic and joy every time they are released. However, I feel this is a big improvement that could be made. In this last index, the DA for my site has remained the same. I was expecting a rise, simply because I have been doing some link building since October and was expecting the results to start showing around this time. However, the link metrics for my site looked worse this month than ever before. The index clearly didn't crawl sites and pages that it has crawled the last couple of indexes, and I noticed a few links disappeared from the metrics altogether, links that I know still exist. So I have a suggestion... If Moz has collected link data in previous indexes, can it not preserve that data rather than starting afresh with every new index. Regular checks could be conducted on existing link data to identify expired or broken links, but to disregard them and start anew with every index seems a little odd. That way, Moz would have much more accurate and comprehensive link data which would help to avoid the DA inaccuracy, and provide users with a better measure of how their site is performing against the rest of the web. This may be naive because I do not know the technical side to index updates. Any thoughts, advice, suggestions etc would be appreciated.
Feature Requests | | AHC_SEO0 -
Can Moz Reports Data be exported in Excel
Hi there, The moz reports are exported into PDF at the moment. Is there a way to export the data into excel? Thanks
Feature Requests | | Jacky.Bizcover0