Wild fluctuation in number of pages crawled
-
I am seeing huge fluctuations in the number of pages discovered the crawl each week. Some weeks the crawl discovers > 10,000 pages and other weeks I am seeing 4-500.
So, this week for example I was hoping to see some changes reflected for warnings from last weeks report (which discovered > 10,000 pages). However, the entire crawl this week was 448 pages.
The number of pages discovered each week seems to go back and forth between these two extremes. The more accurate count would be nearer the 10,000 mark than the 400 range.
Thanks.
Mark
-
No problem!
Glad to see Cyrus' response!
-
Hi Mark,
I used to troubleshoot these types of problems (mysteries
when I worked on the SEOmoz help team.
The best thing to do would be contact the Help Team (help@seomoz.org) and include information both your account, url and campaign. They can take this information and see if there is anything odd about your website, or if there is a bug in the crawling software, or finally if there is some strange quirk of incompatibility causing this behavior.
If you would rather, you can PM me with the info and I can try to troubleshoot it myself, but the Help Team has a few more tools and access to engineers, so they might be the better choice. Either way, let us know if you have any trouble.
-
Thank you for the response. I should have been more clear. It is the weekly SEOMoz crawl that is showing such inconsistent behavior, not Google. Sorry I wasn't more clear.
We have very few (if any) broken links, errors, etc.
Thanks.
Mark
-
Hi there Mark!
We used to have the same issue using Joomla here. It turns out that Google will reduce their crawling if your site has too many errors, broken links, and so on.
We used GWT to look into the 404's then redirected the broken links. Afterwards, we resubmitted the site to be reindexed. A few weeks later -VOILA- all is back to normal and our page freshness stays where it should.
I'd recommend looking at your GWT first, and fixing broken links followed by resubmission to SE's...
Good Luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page authority
Hello, How can my page authority be different across various page built exactly on the same model and none of them having links ? Thank you,
Moz Pro | | seoanalytics0 -
Why would I be ranking for a term when it's not anywhere in my page or pages linking to it?
Hi, I have a keyword that's not anywhere in my page in page-grade tool nor are any pages linking to it with that term, why would that be ranking. The term is "dynamic card solution" This is a company we used to have 5 years ago and merged with our company. This term is ranking #1 for this page- http://www.datacard.com/instant-issuance-solutions . I can't figure why that would be even as this is a newer page in the past year. Thanks for any insight as I'm newer to moz. Laura
Moz Pro | | lauramrobinson320 -
Duplicate Pages
Hello, we have an issue which I'm hoping someone can help with. Our Moz system is saying that this page http://www.indigolittle.com/fees/ Is a duplicate page. We use this page purely for mobiles and we have added code to say This has been on for over a month now however Moz is still picking the page us as a High Priority Issue.
Moz Pro | | popcreativeltd0 -
Pages to Optimise Issues
HI, Can someone tell me why 2 A pages are showing up in red in the F Grade page ? I see Page title missing from the output and just curious if I need to addrss some issue. xpOW4tY.jpg
Moz Pro | | ecrmeuro0 -
Pages Crawled: 1 Why?
I have some campaigns which have only 1 page crawled, while some other campaigns, having completely similar URL (subdomain) and number of keywords and pages, have all pages crawled... Why is that so? It has been also a while I waited and so far no change...
Moz Pro | | BritishCouncil0 -
Open Site Explorer is showing "No Data" for my page titles under the "Top Pages" Tab
This is for my site: www.kibin.com/ Does this mean there is something wrong with how we're titling our pages? We're working on our on-page SEO, titles, meta descriptions this week and getting them up to snuff. I don't understand what is going on here or if it's something to be worried about. Thanks!
Moz Pro | | Kibin0 -
Domain / Page Authority - logarithmic
SEOmoz says their Domain / Page Authority is logarithmic, meaning that lower rankings are easier to get, higher rankings harder to get. Makes sense. But does anyone know what logarithmic equation they use? I'm using the domain and page authority as one metric in amongst other metrics in my keyword analysis. I can't have some metrics linear, others exponential and the SEOmoz one logarithmic.
Moz Pro | | eatyourveggies0 -
How long is a full crawl?
It's been now over 3 days that the dashboard for one of our campaigns shows "Next Crawl in Progress!". I am not complaining about the length... but I have to agree that SEOMoz is quite addictive, and it's quite frustrating to see that everyday 🙂 Thanks
Moz Pro | | jgenesto0