Way has the number of pages crawled plummeted?
-
Why has the number of pages crawled for our campaign plummeted in Moz Analytics – down to 729 from over 10k? Don't see any issues in Google Analytics with crawling our site.
-
Wow, that was like an hour of digging--trying to track down the linkscape crawler. I think I've got it now.
-
Rogerbot is the campaign crawler, and OSE has a different name. I know it can be a tad bit confusing at times.
-
Ah huh, I see. So does the campaign crawler have a different name or does it go by Rogerbot too?
-
Chris, what you're talking about is Open Site Explorer. I believe what Eyeglass Guy is talking about is the campaign crawl, which is an entirely separate crawl system.
-
Eyeglasses,
I wonder if your Mozrank or Moztrust has ratcheted down, as well, or if you've made any architectural/navigational changes to the site. With that many pages, it is possible that its not getting crawled as deeply as it had in the past if those dropped pages no longer met the value criteria for crawling by rogerbot--maybe you lost a good link or two or something like that. Also, if you changed any internal linking, it might be that those pages may have been pushed down in the architecture and not be as accessible as they once were.
-
How many pages does your site have? Did you possibly clear up a bunch of parameters and duplicate content, so the crawl is more accurate?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I have too many tittle tag issue for my site on moz site crawl error
I have too many tittle tag issue in site crawl error but when I checked manually for the error there is no title in source code. Please Help me to understand
Moz Bar | | Nileshaggarwal0 -
Why is Moz Crawling More Pages Than My Site Actually Has?
Hi I have a site that only has 5k pages but Moz has crawled 50K pages on the site when I initiated the site crawl. I don't exactly know why Moz is reporting me back so many pages but I was wondering why this is and if any of you out in the Moz community know anything about this. Thanks
Moz Bar | | drewstorys0 -
MozBot Finding Duplicate Pages That Aren't Duplicate
I've been reviewing the technical audits for my campaign in Moz, and noticed I had a number of duplicate content issues that I'm not really sure how to address. When I click on the links of what the duplicates are, they are all different links that have different content/images. Based on what I was seeing other's wrote in the forum, this could be because the code base is really the same between these pages, and many of these were using query parameters (I'm assuming that is why the code is almost exactly the same across these pages), so example: website.com/tags/KEYWORD1?type=KEYWORD2 is a duplicate of website.com/tags/KEYWORD3?type=KEYWORD4 I was reading that I can use that URL Parameters area in google search console, but my search console says that the googlebot isn't experiencing issues, so I wasn't sure if that was the right move. I can't do the canonicals because these pages all have different content on them, and I know duplicate content is a big SEO issue, so I really wasn't sure what my next steps should be. Thanks for the help!
Moz Bar | | amaray4030 -
How does a non-traditional TLD impact Moz's crawl test?
I have a client who moved from a .com to .academy domain 6 months ago, and their current crawl tests are coming back with a universal page authority of 1, along with 0 indexed backlinks. The previous version of the site had an average page authority of 35-40, the site architecture and content are nearly identical, and there are no other errors or red flags in the crawl report that would hold back their organic rankings. In fact, looking at the site's analytics account, I can see dozens of sites that provide current and properly functioning backlinks, non of which are listed on the crawl test. So the question is - is Moz currently unable to properly crawl a .academy (or any other non-traditional TLD) site, or is there some deeper issue with the site's SEO that I'm not seeing? Thanks!
Moz Bar | | ThinkAOR1 -
On-page SEO in Shopify
Can I successfully perform on-page SEO in Shopify much like I would in a WordPress website? Reasons why I am asking: No plugin I know of like SEO-All-in-one, yoast or any plugin I know of.... URL is www.yourwebsite.shopify.com - Will my on-page efforts be fruitless with this kind of URL. Will the MOZ 'on-page grader' work properly with pages at www.yourwebsite.shopify.com or www.yourwebsite.com? Any Advice, pitfalls of performing on-page and off-page SEO in Shopify? Can you recommend an SEO plugin/APP for Shopify? Will I need a developer/coder to perform SEO in Shopify - Title and Descriptions? Best tool to find long-tail keywords? Thanks. JOE
Moz Bar | | Joseph.Lusso0 -
I'm getting, "you're not using the rel="canonical" META attribute" in my crawl diagnotic
I'm running a campaign crawler through Moz on this particular page: http://www.henley.ac.uk/executive-education/leadership-and-management-programmes/ but I'm getting a notifcaiton from Moz saying, "you're not using the rel="canonical" META attribute" I don't understand what this means!! Has anyone else had this problem, or can they help me understand what this means and how to fix it? Oh, and Happy Thanksgiving from the UK! Virginia
Moz Bar | | blackboxideas0 -
Suggestion for Improving the Crawl Report on Canonicals
This came up in the answer to a question I gave here http://moz.com/community/q/canonicals-in-crawling-reports#reply_222623 Wanted to post here to put it in as a suggestion on how to improve the Moz Crawl reports Currently, the report shows FALSE if there is no canonical link on a page and TRUE if there is. IF you get a TRUE response, this shows up as a warning in your report. I currently use Canonical to Self on almost all my pages to help with some indexing issues. I currently use the EXACT function in excel to create a formula to see if my canonical link matches the URL of the page (as this is what I want it to do). I can then know that the canonical is implemented properly, or if I need to manually check pages to make sure the canonical that points to another page is correct. I would like to suggest that the Moz crawl tool does this. It can show FALSE is the canonical is missing, TRUE if the canonical is present and SELF if the canonical points to the URL of the page it is on. I think for the most part this would be much more actionable information. I would even suggest that TRUE would need to be more of a high priority alert, and SELF can't do any damage, so I would leave that info in the CSV but not have that as a warning in the web interface. Thanks for listening!
Moz Bar | | CleverPhD0 -
Comparing details of current on-page grade reports with previous reports
Hi Whilst i've set up reporting for on-page grader when it comes through on report it just shows any changes in grade and rank, what i also want to look at is changes in the detail/components of current report compared to previous reports, is this possible historically (over the previous reporting periods) ? There doesn't seem to be any kind of archive that i can find to previously carried out on-page grade report ? Cheers Dan
Moz Bar | | Dan-Lawrence0