My crawl is only crawling only the 1st page. What’s going wrong?
-
When I crawl the website kmtstore.com usiing the moz tool , it crawls the product urls only the fist page and skips the rest of the products on the remainaing pages .
Here are the produclist page urls which has the products on many pages but on crawling only the 1st page products show up . I have crawled this website kmtstore.com yesterday and all the details being given is as per the latest crawl report generated yesterday./
http://kmtstore.com/pl/11099/H2O-Jet-Parts-Flow-Style.aspx
http://kmtstore.com/pl/11103/Pneumatic-Control-Valves.aspx
http://kmtstore.com/pl/11104/Swivel-Joint-Assemblies.aspx
http://kmtstore.com/pl/11105/Cutting-Head-Components/Cutting-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11106/Cutting-Head-Components/Focusing-Tubes.aspx
http://kmtstore.com/pl/11107/Cutting-Head-Components/Cutting-Heads.aspx
http://kmtstore.com/pl/11108/Cutting-Head-Components/Orifice.aspx
http://kmtstore.com/pl/11109/HP-Valves-Fittings-and-Tubing/Tubing.aspx
http://kmtstore.com/pl/11110/HP-Valves-Fittings-and-Tubing/Inline-Water-Filter.aspx
http://kmtstore.com/pl/11111/HP-Valves-Fittings-and-Tubing/High-Pressure-Fittings.aspx
http://kmtstore.com/pl/11112/HP-Valves-Fittings-and-Tubing/Gauges.aspx
http://kmtstore.com/pl/11113/Intensifier-Parts/Electrical.aspx
http://kmtstore.com/pl/11114/Intensifier-Parts/Low-Pressure-Water-Systems.aspx
http://kmtstore.com/pl/11115/Intensifier-Parts/Topworks-Replacement-Parts.aspx
http://kmtstore.com/pl/11116/Intensifier-Parts/Hydraulic-System.aspx
http://kmtstore.com/pl/11117/Intensifier-Parts/Sealing-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11118/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Coning-and-Threads.aspx
http://kmtstore.com/pl/11119/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Topworks.aspx
http://kmtstore.com/pl/11120/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Lubricants.aspx
http://kmtstore.com/pl/11121/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Pneumatic-Valves.aspx
http://kmtstore.com/pl/11122/Maintenance-Tools-and-Lubricants/Manuals.aspx
http://kmtstore.com/pl/11123/Pneumatic-Control-Valves/Pneumatic-Control-Valve-Replacement-Parts.aspx
http://kmtstore.com/pl/11124/Pneumatic-Control-Valves/Cutting-Head-Nozzle-Tubes.aspx -
Hi there!
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. I appologize for any confusion here. I downloaded your crawl CSV for kmtstore.com, and everything looked fine. From what I can tell Google has 296 pages indexed for your site, and we crawled 241.
Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. Thus, your homepage is the only seed we use to crawl your site. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way. Since we use recursive crawling, if these product pages aren't linked to your homepage, our crawler won't be able to pick them up.
I hope this helps, and if you have any other questions, please reach out to us at help@moz.com! We'll be able to take this conversation offline and answer any other questions for you from there!
Cheers!
Erin
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Even canonical present in my web pages, still moz showing canonical issues.
Even canonical present in my web pages, still moz showing canonical issues.
Moz Local | | prasad.nueve0 -
Trying to connect my moz local to my facebook page - says it isn't found
Trying to connect my moz local to my facebook page - says it isn't found even though it is. I don't want to create another listing for this. Cheers Claire
Moz Local | | blenkiseo0 -
Has anyone found a way to change the Google page linked to a Moz local account AFTER it was purchased?
Before realized that the Moz listing should have the correct GMB account connected to it, a few local listings were purchased. Is there a way to change the GMB page it is pulling data from and sending out to directories? Thanks!
Moz Local | | BlueCorona2 -
What do I do if the business listed in Moz Local Business Listing for Yellow Pages is wrong?
Hey there! I have a business listing set up in Moz Local. The system seems to have accurately found the business on most networks, but on Yellow Pages, it has a completely different business listed (my business is a Pilates studio and it has a soccer club listed). What are the steps I can take to fix this? I can claim my listing on Yellow Pages, but after I do that, how does Moz Local associate the two? Thanks, --eric
Moz Local | | EricOliver0 -
What's Going On with Moz Social
Why can't I add my social info to the Moz campaign that I'm currently running?
Moz Local | | maxcarnage0 -
It's been 1 year, why hasn't our Moz Local completion percent improved?
Our listing is about to renew, but the completion percentage has not improved since the first month we started with Moz twelve months ago. I see in Moz there are many duplicate listings for this business. I was under the impression that Moz was handling this as part of the yearly fee; is that correct? If not, what needs to be done to resolve these issues and make sure our listings are complete and accurate?
Moz Local | | GLabs0 -
Insider Pages, Citysearch, and Bing Not Updating
Hello, I have a campaign for "Teasdale Fenton" that we started back in May, 2015 (5 months ago) and Insider Pages, Citysearch, and Bing have been sitting as "Pending" with no estimated update date I believe since we launched the campaign. Can you guys look into it and let me know if I did anything wrong or if there is something holding it back from updating? Thanks, Chris
Moz Local | | Cincinnati_WebTec0 -
Does the "Unmanaged" number go up immediately after submission?
I just submitted a CSV file with the information about 3 businesses to Moz Local, but I can't tell if it went through properly. The page gave me a message saying "0 are ready to publish. 0 need validation", and didn't list any errors in my listings. However, the number next to "Unmanaged" is still a 0. I know from looking at some of the other questions on here that from the time that Moz sends the information out, it can take a few weeks for the listings to propagate. But I'm not sure whether or not it's supposed to take some amount of time for uploaded listings to show up on here as Unmanaged listings. Should I try resubmitting, or does it just take a few minutes/hours for unmanaged listings to show up?
Moz Local | | BrianAlpert780