Need to solve "Oops our crawlers were unable to access" url for new campaign
-
I'm putting the url designfirstkitchenandbath.com and getting the "oops! our crawlers were unable to access the site. Since this site is a potential client, which shows up online, I can't get access to fix the code, plus while I can write a little html I don't feel comfortable working with hard, live code on someonelse's site. Anyone have a simple solution?
-
Hey there!
Sam from Moz's Help Team here!
I've taken a look at that URL, and from what I can see it looks like we're running into a 403 error which means that ultimately, access to that URL is forbidden:
http://www.screencast.com/t/UbMwIrjuQFG
This 403 error is coming directly from your server, so at this point the best thing to do will be to send this information to your client/their web dev. They'll need to make sure our crawler (rogerbot) is whitelisted, so our tools can start working hard for you, and that you aren't blocking AWS (Amazon Web Services) on the server level.
I hope this helps - if you have any other questions, feel free to pop a message over to help@moz.com
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Possible to Migrate Campaign Settings?
We have completed a a domain migration. I am ready to archive the existing campaign but would like to migrate the settings (keywords, etcetera) to the new campaign for the new domain. All settings to remain the same except for the domain name. Can this be done in a simple manner. I am not technical so I hope it is really easy. Thanks,
Moz Bar | | Kingalan1
Alan0 -
Tracked Ranking Keywords in Campaign and Ranking Keywords in Keyword Explorer are coming up with very different results - does anybody know why?
We have a campaign set up that is tracking keywords with varying monthly volumes and the website is ranking for them. I have gone into the new Keyword Explorer Feature (looks great!) and I have tested it by putting in the root domain address mentioned above. My problem is that none of the keywords that I am tracking and ranking for are appearing in the "ranking keywords" report in Keyword Explorer. Does anybody know why this might be happening?
Moz Bar | | RedAntSolutions0 -
Only One Canonical URL Tag
HI, I'm an SEO novice - company owner with no money so doing it all myself with help from my web designer using wordpress. Ive just completed some seo and done the moz page scoring analysis for optimisation and gained 92% - however - there is one outstanding issue on canonical url tags - i.e. recommened fix = The canonical URL tag is intended to refer duplicate pages to a single canonical URL. To ensure the search engines properly parse the canonical source, your page should use only one version of this tag in the header. See Canonical URL Tag - the Most Important Advancement in SEO Practices Since Sitemaps Ive gone through the page code and can see I have 2 rel=canonical references - am I able to simply delete one - how do I do this if its been created by the yoast/wordpress plug-in? Many thanks in advance for any help!
Moz Bar | | M-J-Smith0 -
Onpage Grader not Finding Keyword in URL
I've noticed that the Onpage Grader is not including my keyword in the URL when the keyword is in the domain. If I grade an inner page and the keyword is in the sub-directory, it finds it. Is this intentional? If so, why does the grader not include my keyword in the domain as Keyword in the URL?
Moz Bar | | Dino640 -
MOZ crawler has been finding a lot of 803 and 804 errors
During last 3 weeks MOZ crawler has been finding a lot of 803 and 804 errors. Meanwhile all pages seem to be working fine. What could cause it?
Moz Bar | | Paruyr0 -
Idea for new feature on Moz Q&A
Hi Moz Can we please have a new feature to moz when answering questions please, the ability to save as a draft. I am currently on the train from London back home and I was trying to answer a few questions and give my opinion and after writing a really long and detailed response to an area I actually have quite a lot to say on, my internet froze and the page refreshed losing all the content. It doesn’t help that my Mac I am working on isn’t the fastest, but my new one isn’t quite ready yet. But if there was a simple save draft feature, it would be really great as I wasn’t ready to post my answer as it wasn’t finished and without checking wouldn’t have made sense. Yes I know I could of done it in word and pasted it across, but usually when I am travelling and answering questions I do it on my Ipad which isn’t great for multitasking. While I understand you don’t want lots of ‘draft answers’ all over the place as this would take storage space up on your servers and doesn’t actually help the person answering the question, your system could purge all draft answers which have not been edited in the last 24 hours. Just a little suggestion to help make this great forum even better. Thanks Andy
Moz Bar | | Andy-Halliday4 -
Accessing old reports
Hi there, Is there any way of accessing reports for a certain campaign that are a year old? To my knowledge the reports had not been generated, and if they were, they have not been saved in the custom reports section.
Moz Bar | | WebhouseISP
Is there anyway of generating a report from months ago? Thanks a million!0 -
Moz "Crawl Diagnostics" doesn't respect robots.txt
Hello, I've just had a new website crawled by the Moz bot. It's come back with thousands of errors saying things like: Duplicate content Overly dynamic URLs Duplicate Page Titles The duplicate content & URLs it's found are all blocked in the robots.txt so why am I seeing these errors?
Moz Bar | | Vitalized
Here's an example of some of the robots.txt that blocks things like dynamic URLs and directories (which Moz bot ignored): Disallow: /?mode=
Disallow: /?limit=
Disallow: /?dir=
Disallow: /?p=*&
Disallow: /?SID=
Disallow: /reviews/
Disallow: /home/ Many thanks for any info on this issue.0