Drop in number of Pages crawled by Moz crawler
-
What would cause a sudden drop in the number of pages crawled/accessed by the Moz crawler? The site has about 600 pages of content. We have multiple campaigns set up in our Pro account to track different keyword campaigns- but all for the same domain. Some show 600+ pages accessed, while others only access 7 pages for the same domain. What could be causing these issues?
-
If that was how our crawler was accessing your pages before, because there aren't links for them else where, then it could be the issue. It does depend there. There are a lot of aspects to how our crawler works so it is really hard to say.
-
Hi Nick,
Thank you very much.. no actually we did not make any changes to robot.txt, but we have made some changes to sitemap.xml
Could it create a temporary problem?
-
AllaO,
Sorry about the odd crawl activity. I pulled up your campaigns and it looks like something blocked rogerbot from crawling. This could be new redirects or changes in your robots.txt file. If you don't think one of those is the issue get a hold of our support team: https://seomoz.zendesk.com/forums You can submit a ticket from here.
Best of Luck
-
AllaO
Do you have some campaigns set to track at the root and some at the sub domain? That would be the only way I could think of when they are essentially being run simultaneously.
Best
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
moz crawl is stopped?
moz stopped indexing the links due to some updates? can some one confirm me thanks
Moz Pro | | 42409300125323700 -
Is this owned by moz?
We continue to get hundreds of spam links per month, and just found a link from here. I do not believe this is owned by moz. https://www.wapz.net/ Maybe I'm wrong? Thanks.
Moz Pro | | plahpoy0 -
Moz Data Issues?
Since the launch of Moz something or other has been wrong with my data. Is everyone having these issues? Or is it just me?
Moz Pro | | EcommerceSite0 -
1 page crawled - again
Just had to let you know that it happend again. So right now we are at 2 out of the last 4 crawls. Uptime here is 99,8% for the last 30 days, with a small downtime due to an update process at the 18/5 from around 2:30 to 4:30 GMT In relation to: http://moz.com/community/q/1-page-crawled-and-other-errors
Moz Pro | | alsvik0 -
Backlink of competitor inner page
How to understand from where the link to a competitor's internal page http://www.exampledomain.com/contest/book are coming? I mean, we want promote the same "event" but we want understanding where the competitor is already present. Open Site Explorer doesn't have any data for the website listed but I'm sure that the page has a lot of backlinks (I've verified the first 50 manually in Google.it) Many Thanks for all your advices.
Moz Pro | | YESdesign0 -
Page Penalization
Hiya, Looking for some advice, i have a page which the on page optimization tool shows as an A grade and Google has indexed it. I have checked vie site: however is not being found in search results even for an exact match on the page title which is very specific. I believe the page may be being penlized for over optimisation? any advice would be great! URL is www.tots-away.com/child-friendly-holidays-spain/
Moz Pro | | iprosoftware0 -
How Do I deal with duplicate page titles for pages on eCommerce site
Hi We have an ecommerce site selling physical products. There are a few areas where the products run into two pages. I have used canonical meta tags and next and prev meta tags too. Despite this SEOMOZ reports are still displaying these as warnings for duplicate page titles. An example would be /brand_name/range/ <link rel="next" href="/brand_name/range/?page=2" /> <link rel="<a class="attribute-value">canonical</a>" href="/band_name/range/"/> /brand_name/range/?page=2 <link rel="<a class="attribute-value">prev</a>" href="/brand_name/range/" /> <link rel="<a class="attribute-value">canonical</a>" href="/band_name/range/?page=2"/> Should I be doing something different?
Moz Pro | | wouldBseoKING0 -
Set crawl frequency
Current crawl frequency is weekly, is it possible for me to set this frequency our-self?
Moz Pro | | bhanu22170