Drop in number of Pages crawled by Moz crawler
-
What would cause a sudden drop in the number of pages crawled/accessed by the Moz crawler? The site has about 600 pages of content. We have multiple campaigns set up in our Pro account to track different keyword campaigns- but all for the same domain. Some show 600+ pages accessed, while others only access 7 pages for the same domain. What could be causing these issues?
-
If that was how our crawler was accessing your pages before, because there aren't links for them else where, then it could be the issue. It does depend there. There are a lot of aspects to how our crawler works so it is really hard to say.
-
Hi Nick,
Thank you very much.. no actually we did not make any changes to robot.txt, but we have made some changes to sitemap.xml
Could it create a temporary problem?
-
AllaO,
Sorry about the odd crawl activity. I pulled up your campaigns and it looks like something blocked rogerbot from crawling. This could be new redirects or changes in your robots.txt file. If you don't think one of those is the issue get a hold of our support team: https://seomoz.zendesk.com/forums You can submit a ticket from here.
Best of Luck
-
AllaO
Do you have some campaigns set to track at the root and some at the sub domain? That would be the only way I could think of when they are essentially being run simultaneously.
Best
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My moz campaing not update completely
I have create new account and campaign last week, till now my moz campaign not update like: keywords ranking, competitor websites links etc. Why it is so please update me.
Moz Pro | | Surabhi_Dewra0 -
SEO Crawl Report Images?
Does SEOMOZ crawl images in the report? Raven tools is showing me about 200 missing alt tags and title tags. I can not seem to find any of this information on the SEOMOZ report. Am I missing something?
Moz Pro | | jasonsixtwo0 -
Seomoz crawler problems
I have had Seomoz for about a month. It has crawled about 1000 pages. I have about 10,000 pages total for the site. Why are these others being a problem? I have contacted help but the guy isn't any help, we have just been going back and forth for the last two weeks. Any suggestions?
Moz Pro | | EcommerceSite0 -
How to Optimize a Home Page?
Can anyone advise on what is the best way to optimise the home page? Many websites I see include the target(focus) keywords from other pages on the home page. Is this self competition? "Avoid Keyword Self-Cannibalization- It's a best practice in SEO to target each keyword with a single page on your site" If an e-commerce website has 4 levels in the SEO architecture. Home Page, Brand Pages(Category), Range pages(sub-category) and individual product pages. It would seem that the main keywords would be within the category(brands) level of a website. However many websites I see include these on the home page as well. Could someone please clarify this? I do not recall seeing the report card saying Self-Cannibalization when these keywords(category level) are listed on the home page. However I could be wrong, Please could someone advise on what SEO MOZs recommendations are for best practise on optimizing a home page so that it does not compete with category pages, or other pages on the website. Thanks
Moz Pro | | WMA0 -
Does the page authority data also considers the on page factors like the presence of keyword in the title,meta text, and keyword frequency ??
The moz difficulty score considers four factors for the top websites. are the on page factors included in the page authority data ?
Moz Pro | | iQuanti0 -
Truncate page URLs
We have some pages (for example a contact us form) for which the URL is modified by the CMS depending on the referring page (this helps to put the form submission in context for the sales reps who get the contact submission). The SEOmoz crawler considers each URL a new page -- and so numbers like in diagnostics are all inflated as the same page is listed multiple times (e.g. for too many links) Is there a setting to change what the crawler considers to be the same page? Here are two URLs for the same page that the reports treat as separate pages: http://www.spirent.com/About-Us/Contact_us.aspx?referurl=0F528F4D703D8BB3523738D6373AA8AD http://www.spirent.com/About-Us/Contact_us.aspx?referurl=10ACDA6055244E369395223437FDCF30 The page is actually: http://www.spirent.com/About-Us/Contact_us.aspx Thanks Ken
Moz Pro | | spirent.marcom0 -
Current on-page best practices
Given all the recent talk about over optimization, when was the last time SEOMoz updated the on-page report card tool? Rand wrote an excellent piece on Perfect On-Page Optimization (which is great, and thanks) in summer 2009. Is that still best practice 3 years later (and post-Penguin/Panda)? If not, has the SEOMoz on-page report card tool been updated to reflect current thinking for on-page best-practices? I know the higher level concept is "write for humans, not for bots" but if you can do both (and not create an unreadable seo-frankenpage) then why not? Does getting an "A" grade wreak of over optimization now? Should I use the key phrase at the start of the title, h1, and strong (or bold) elements on a page? Should have an image with file name and alt text equal to (or containing at the start) the key phrase?
Moz Pro | | scanlin2 -
Sub-domain not crawled
One of our sites was recently re-designed. The home page is a landing page (www.labadieauto.com) and I moved the blog to this domain (labadieauto.com/blog/) and put a link is the bottom left of the home page. Since the change the SEOMOZ campaign overview is showing only 1 page crawled. This is not setup as a sub-domain so why isn't it showing in the crawl? Help!
Moz Pro | | LabadieAuto0