My report only says it crawled 1 page of my site.
-
My report used to crawl my entire site which is around 90 pages. Any idea of why this would happen?
-
Here's a replacement post with some of the same information at http://moz.com/help/pro/why-can-t-rogerbot-crawl-my-site. You can also send a note to the help desk at help@moz.com and we'll help you figure out what's up.
-
Hi Keri:
I am trying to run a crawl on two different sites where this is happening. I've come across your response in two posts, six months apart. However the article you reference produces a 404 error. Can you point me to the new location of that information? Thank you.
-
We have had a few crawl hiccups lately. Some reasons why this happens are at https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages and if that doesn't work, please do go to http://www.seomoz.org/help and open up a ticket with us.
-
I have it set as a campaign. This would of been my second crawl for this month. I had relatively low errors, warnings, but all my errors are set to zero so that is a good thing :). It almost looks as if my homepage is the only one being indexed, but i checked my other pages and all seems to be well. I will put a ticket in.
Thanks,
-
Nate
A little more info would help. I looked at your url and all seems ok on the face of it. Was this a crawl test or a campaign? Did you get a summary page and did it show any crawl errors? Warnings? etc.?
If it was a campaign and if in the past ...used to crawl entire site... put a ticket in with support or email help@seomoz.org. They are quick to help.
Hope this helps,
-
From what I gather from reading up on this over the last few days, SEOmoz does a quick crawl of your site then does a re-crawl which is later on. Think it takes up to 7 days for a full crawl.
If you have just started, I wouldn't worry too much at this stage!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best way to handle URLs of the to-be-translated pages on a multilingual site
Dear Moz community, I have a multilingual site and there are pages with content that is supposed to be translated but for now is English only. The structure of the site is such that different languages have their virtual subdirs: domain.com/en/page1.html for English, domain.com/fr/page1.html for French and so on. Obviously, if the page1.html is not translated, the URLs point to the same content and I get warnings about duplicate content. I see two ways to handle this situation: Break the naming scheme and link to original English pages, i.e. instead of domain.com/fr/index.html linking to domain.com/fr/page1.html link to domain.com/en/page.html Leave the naming scheme intact and set up a 301 redirect so that /fr/page1.html redirects to /en/page1.html Is there any difference for the two methods from the SEO standpoint? Thanks.
Technical SEO | | Lomar0 -
What is the best way to redirect visitors to certain pages of your site based on their location?
One website I manage wants to redirect users to state specific pages based on their location. What is the best way to accomplish this? For example a user enters the through site.com but they are in Colorado so we want to direct them to site.com/colorado.
Technical SEO | | Firestarter-SEO0 -
Google is Still Blocking Pages Unblocked 1 Month ago in Robots
I manage a large site over 200K indexed pages. We recently added a new vertical to the site that was 20K pages. We initially blocked the pages using Robots.txt while we were developing/testing. We unblocked the pages 1 month ago. The pages are still not indexed at this point. 1 page will show up in the index with an omitted results link. Upon clicking the link you can see the remaining un-indexed pages. Looking for some suggestions. Thanks.
Technical SEO | | Tyler1230 -
Can I use high ranking sites to push my competitors out of the first page of search results?
I'm looking at a bunch of long tail low traffic keywords that aren't difficult to rank for. As I was idly doing a boring task my mind wandered and I thought.... Why don't I ask lots of questions about these keywords on sites such as Moz, Quora, Reddit etc where the high DA will get them to rank for the search term? The results on a SEO site or Q&A site won't be relevant and so I'd starve my competitors of some of their leads. Of course I'm not sure the effort would be worth it but would it work? (and no, none of my long tail keywords are included in this post)
Technical SEO | | Zippy-Bungle3 -
Effect of 302 redirects from empty parent page to sub page
A client's website has links to their service pages which then redirect (302 through a php "Location:" header) to that service's first sub-page. For example, our-services/service-x redirects to our-services/service-x/about-service-x I can only think this has been done because there is no actual content for the parent page and to maintain some kind of structure for navigation and URLs. Really there's no reason why the 'about-service-x' page can't be removed and its content transferred to the main 'service-x' page. Then the redirects can be removed also - it's not how a 302 should be used for a start. I'm just wondering what kind of effect this current redirection has on SEO, as I know 302s don't pass any link juice? Thanks for your help.
Technical SEO | | driftingbass0 -
Do we need to manually submit a sitemap every time, or can we host it on our site as /sitemap and Google will see & crawl it?
I realized we don't have a sitemap in place, so we're going to get one built. Once we do, I'll submit it manually to Google via Webmaster tools. However, we have a very dynamic site with content constantly being added. Will I need to keep manually re-submitting the sitemap to Google? Or could we have the continually updating sitemap live on our site at /sitemap and the crawlers will just pick it up from there? I noticed this is what SEOmoz does at http://www.seomoz.org/sitemap.
Technical SEO | | askotzko0 -
Will frequently adding and frequently removing pages from my site hinder any SEO?
Hi Guys, Just looking through our crawl diagnositcs and we have a ton errors, well over 5000 actually, on 404 pages that cannot be accessed. Our website runs a lot of "Hot Offers" that are time bound, so they expire at the end of each month and we remove the page via our CMS. It's making the crawl diagnositcs loook bad, but will this be hindering our seo and Google 'stuff' because they are finding thousands of 404 errors? Any advice would be greatly appreciated! Website: www.vospers.com Lee Greenhill
Technical SEO | | lee_greenhill0 -
Delete old site but redirect domain to a new domain and site
I just have a quick query and I have a feeling about what the answer is so just wanted to see what you guys thought... Basically I am working on a client site. This client has a few other websites that are divisions of their company. However these divisions/websites are no longer used. They are wanting to delete the websites but redirect the domains to their name main website. They believe this will pass on SEO benefits as these old division sites are old and have a good PR and history. I'm unsure for DEFINITE, which way is correct?
Technical SEO | | Weerdboil0