How do i get the crawler going again?
-
The initial crawl only hit one page. Set up another campaign for another site and it crawled 260 pages.
How can I get the crawler started up again or do I really have to wait a week ?
-
It's easiest for the Moz team to respond to individual questions like this via an email to help@seomoz.org or sending in a ticket through the help hub at seomoz.org/help.
-
Hi Keri,
Yeah, I've followed these steps. htaccess forwards to www if none and my campaign is using www.
The robots.txt is available in browser no problem.
my htaccess had massive country ip blocks as well as amazonaws..
I've removed all blocks that arent china or amazon.
Still waiting on a response from the MOZ team here on how to get the real crawl going as what I found in tools is not the same as the weekly scan.
-
Also, for why we only crawled one page, check out https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages- for some helpful tips.
-
Whenever you have crawl issues, the SEOMoz help team is the place to go. They're the only ones who can look into your account to see if there's an issue or such. (And they are really nice and quick to respond.)
-
Research Tools > Crawl Test at bottom left of page.
Not sure why it's taken me 2 days to find that but it did.
Thanks. Sorry for the dumb question...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawler errors or Page Load Time? What affect more to SEO
Hello, I have a page with a forum and at this moment the moz report says that have 15.1k of issues like url too long, meta noindex, title too long etc. But this page have a load time realy sloooow with 11 seconds. I know i need fix all that errors (i'm working on this) but... What is more important for SEO? The page load or that type of error like duplicate titles etc. Thank you!
Moz Pro | | DanielExposito1 -
My website was at the top of Google search for some years... suddenly I almost can't reach first page! Moz ranks my website better than the competitors... what might be going one? Could anybody help me out? Thanks!
Hello Guys! My website was at the top of Google search for some years... suddenly I almost can't reach first page! Moz ranks my website better than the competitors... what might be going one? Could anybody help me out? Moz rank us grade A... the competitors B or C .. I think we have better back links than they do... Would you need any kind of data or report to help me here? Thanks!
Moz Pro | | wesleyms0 -
Rel="canonical" tag is implemented in my product pages, but still getting canoncal error for products in Moz. What is the problem? me or MOZ?
I have included the rel="canonical" tag in all my product pages, but still getting canonical error in MOZ reports for more than 6 month ! I would like to know if my code is wrong or the MOZ report system is not working properly. Here is an example of my canonical code in line 84 rel="canonical" href="http://www.doornmore.com/slab-single-door-80-fiberglass-courtlandt-1-panel-arch-lite-glass.html" /> Thanks Shayann
Moz Pro | | Shayann0 -
Rogerbot's crawl behaviour vs google spiders and other crawlers - disparate results have me confused.
I'm curious as to how accurately rogerbot replicates google's searchbot I've currently got a site which is reporting over 200 pages of duplicate/titles content in moz tools. The pages in question are all session IDs and have been blocked in the robot.txt (about 3 weeks ago), however the errors are still appearing. I've also crawled the page using screaming frog SEO spider. According to Screaming Frog, the offending pages have been blocked and are not being crawled. Webmaster tools is also reporting no crawl errors. Is there something I'm missing here? Why would I receive such different results. Which one's should I trust? Does rogerbot ignore robot.txt? Any suggestions would be appreciated.
Moz Pro | | KJDMedia0 -
Why am I not getting my allowance of 10,000 inbound links in csv download file? 370 out of 4700??
Hi, I'm desparately trying to audit my backlinks to remove a penguin penalty on my site livefit.co.uk When I do the inbound link report i'm not getting all the links in the download. I know there is a limit of 25 links from each linking site so we get the full picture of links bu: I have 4700 links so why does it need to limit it when we are supposed to see up to 10,000? When you check the link profile on the report it doesn't seem there are many sites with anything close to 25, so surely that rule is invalid as an explanation here? Should I just work off OSE? But there is less useful info than on the csv.. I'd be very grateful for your thoughts. Thanks! James
Moz Pro | | LiveFit0 -
Is there a way to get the domain authority of a subdomain?
For example, www.storage-mart.com and us.storage-mart.com both show the same DA in OSE. I believe it's the DA of the root domain. How can I compare the DA of the 'www' and 'us' subdomains?
Moz Pro | | Whitespark0 -
Who wants to help go over my crawl diagnostics via skype?
I have run a crawl diagnostic on my site and have 194 errors and most of them are 404 errors in wordpress. Not sure why, but many of my pages had name changes (possibly a permalinks issue) but I have no idea how to fix it. I had 5 duplicate page titles, and 1 tile missing or empty. 72 crawl notices found (2 permanent redirect, 17 blocked by robots, 53 rel canonical) 19 Crawl warnings were found Who wants to have some fun?
Moz Pro | | starkSEO0 -
How are our competitors getting these inbound linking domains?
I'm currently managing SEO for my company's website, and I'm getting into link building for the first time. As part of the process, I'm using Open Site Explorer to see who's linking into our competitor sites, to get a better sense of what's available to us in our particular avenue of e-commerce. However, I'm finding that our competitors are getting inbound links from high-authority sites pretty far afield from selling jewelry - census.gov, parallels.com, warnerbros.com, and others. I try clicking through to these links, but each link starts a download of a file. I've seen .f4v, .7z, and .apk files listed as inbound links to our competitor. How is this happening? Again, I'm new to link building, so there may be a simple answer here, and if so I apologize for asking. However, this seems really strange to me, and a difficult situation to confront.
Moz Pro | | jozaksut0