Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Need to solve "Oops our crawlers were unable to access" url for new campaign
-
I'm putting the url designfirstkitchenandbath.com and getting the "oops! our crawlers were unable to access the site. Since this site is a potential client, which shows up online, I can't get access to fix the code, plus while I can write a little html I don't feel comfortable working with hard, live code on someonelse's site. Anyone have a simple solution?
-
Hey there!
Sam from Moz's Help Team here!
I've taken a look at that URL, and from what I can see it looks like we're running into a 403 error which means that ultimately, access to that URL is forbidden:
http://www.screencast.com/t/UbMwIrjuQFG
This 403 error is coming directly from your server, so at this point the best thing to do will be to send this information to your client/their web dev. They'll need to make sure our crawler (rogerbot) is whitelisted, so our tools can start working hard for you, and that you aren't blocking AWS (Amazon Web Services) on the server level.
I hope this helps - if you have any other questions, feel free to pop a message over to help@moz.com
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved New Google SERP Now Lazy Loading
Hello, Google has changed the way it displays it's SERPs with lazy loading new listings. Because of this the Moz Chrome Extension no longer shows results past first results. I also often find that when a site is ranked above the Maps, it doesn't display results. Is there a fix for this?
Moz Bar | | HercMagnus0 -
Does "Disallow: /xmlrpc.php" in robots.txt affect moz tools ability to fetch DA?
Just checked a website for Domain Authority using Moz' tool, however it returned 1 for DA, which should be unlikely. I have been trying to find the problem and found "Disallow: /xmlrpc.php" in robots.txt. Could this affect Moz' tools ability to get the required data?
Moz Bar | | Foli0 -
Confused by "Exact Keyword Used in Document Text at Least Once".
Hi, I am using the On-Page Grader to improve the SEO for certain keywords on certain pages. I have received the notification that the "Exact Keyword Used in Document Text at Least Once". Well from the looks of it I have already done this but I want to make sure that im checking the right thing. What does it mean when it says document text as I assumed it was text on the page but its not reading it as that. Any feedback would be appreciated
Moz Bar | | charlessimmons0 -
Moz crawler only crawls one page?!
Hello there, I'm using Moz for a while and I'm very pleased with the tool and community. But for the first time I encountered a problem. We are trying to run a crawler for a client's website but only one page (only the homepage) was crawled. We tried to do a test on a more detailed level (maybe there is something wrong with the homepage). My campaign test's crawl came back for the Producten folder (level deeper than homepage), and it was also only a 1 page crawl with a 200 status. I did look at the robots.txt file now, and it is very restrictive, but there is nothing that I can clearly see that would explain why the crawl isn't working. Hopefully someone can point us at the right direction. Thanks in advance, Jeremy
Moz Bar | | mediaxplain.nl0 -
What is Considered Duplicate Content by Crawlers?
I am asking this because I have a couple of site audit tools that I use to crawl a site I work on every week and they are showing duplicate content issues (which I know there is a lot on this site) but some of what is flagged as duplicate content makes no sense. For example, the following URL's were grouped together as duplicate content: | https://www.firefold.com/contact-us | https://www.firefold.com/gabe | https://www.firefold.com/sale | | | How are these pages duplicate content? I am confused on what site audit tools are considering duplicate content. Just FYI, this is data from Moz crawl diagnostics but SEMrush site auditor is giving me the same type of data. Any help would be greatly appreciated. Ryan
Moz Bar | | RyanRhodes0 -
How to change the search profiles in the new mozbar
Hi, I want to change the search profiles in the new mozbar, but clicking the + Add new profile does nothing. Is this a browser issue? I'm using chrome Version 35.0.1916.114 m Any light on this would be great because there's no point in me checking US results because my site is only geared towards UK customers, being a mortgage broker we can't target other countries except to cater for expats. Thanks, Amelia xx
Moz Bar | | CommT0 -
Need a Reliable Free Bulk Keyword Rank Checker
I have 2,871 keywords that I need to check Google rank for on 4 separate domains. Does anyone know of any FREE tools or plugins available that will allow for this volume that wont get my IP banned by Google? Even the Moz Rank Checker only allows to enter 1 keyword at a time for up to 200 per day. Who would seriously enter in 200 keywords, one by one all day, every day?
Moz Bar | | dsinger0 -
How to find all 301 redirect for URL xyz.com/products (internal and external)?
This is what we are thinking: Get all URL of the xyz.com/products using XENU software. Search those URL on google (site;xyz.com url ) to find out if they are crawled by google, do the same on bing (as currently google shows 4k URL and bing 11k ) Use opensiteexplorer (301 redirect ) and using (internal external) to get the desired result. Is this the right approach? If not, what is the best way to find the correct result? All suggestions are welcome.
Moz Bar | | tpt.com0