My crawl is only crawling only the 1st page. What’s going wrong?
-
When I crawl the website kmtstore.com usiing the moz tool , it crawls the product urls only the fist page and skips the rest of the products on the remainaing pages .
Here are the produclist page urls which has the products on many pages but on crawling only the 1st page products show up . I have crawled this website kmtstore.com yesterday and all the details being given is as per the latest crawl report generated yesterday./
http://kmtstore.com/pl/11099/H2O-Jet-Parts-Flow-Style.aspx
http://kmtstore.com/pl/11103/Pneumatic-Control-Valves.aspx
http://kmtstore.com/pl/11104/Swivel-Joint-Assemblies.aspx
http://kmtstore.com/pl/11105/Cutting-Head-Components/Cutting-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11106/Cutting-Head-Components/Focusing-Tubes.aspx
http://kmtstore.com/pl/11107/Cutting-Head-Components/Cutting-Heads.aspx
http://kmtstore.com/pl/11108/Cutting-Head-Components/Orifice.aspx
http://kmtstore.com/pl/11109/HP-Valves-Fittings-and-Tubing/Tubing.aspx
http://kmtstore.com/pl/11110/HP-Valves-Fittings-and-Tubing/Inline-Water-Filter.aspx
http://kmtstore.com/pl/11111/HP-Valves-Fittings-and-Tubing/High-Pressure-Fittings.aspx
http://kmtstore.com/pl/11112/HP-Valves-Fittings-and-Tubing/Gauges.aspx
http://kmtstore.com/pl/11113/Intensifier-Parts/Electrical.aspx
http://kmtstore.com/pl/11114/Intensifier-Parts/Low-Pressure-Water-Systems.aspx
http://kmtstore.com/pl/11115/Intensifier-Parts/Topworks-Replacement-Parts.aspx
http://kmtstore.com/pl/11116/Intensifier-Parts/Hydraulic-System.aspx
http://kmtstore.com/pl/11117/Intensifier-Parts/Sealing-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11118/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Coning-and-Threads.aspx
http://kmtstore.com/pl/11119/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Topworks.aspx
http://kmtstore.com/pl/11120/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Lubricants.aspx
http://kmtstore.com/pl/11121/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Pneumatic-Valves.aspx
http://kmtstore.com/pl/11122/Maintenance-Tools-and-Lubricants/Manuals.aspx
http://kmtstore.com/pl/11123/Pneumatic-Control-Valves/Pneumatic-Control-Valve-Replacement-Parts.aspx
http://kmtstore.com/pl/11124/Pneumatic-Control-Valves/Cutting-Head-Nozzle-Tubes.aspx -
Hi there!
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. I appologize for any confusion here. I downloaded your crawl CSV for kmtstore.com, and everything looked fine. From what I can tell Google has 296 pages indexed for your site, and we crawled 241.
Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. Thus, your homepage is the only seed we use to crawl your site. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way. Since we use recursive crawling, if these product pages aren't linked to your homepage, our crawler won't be able to pick them up.
I hope this helps, and if you have any other questions, please reach out to us at help@moz.com! We'll be able to take this conversation offline and answer any other questions for you from there!
Cheers!
Erin
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long do Facebook and Google pages have to be published before Moz can verify them?
Hi there. I've checked the forums and have seen similar questions, but couldn't find an answer in them. I have a listing that I imported from my Google My Business account, but it comes up as unverified for both Facebook and Google. Both pages exist, but have only been published for about 3 weeks. Both sites contain the accurate business information and are the appropriate types of pages, so the best I can determine from what I've read in the other posts is that "newly" published Facebook and Google pages may not be able to be verified by Moz Local. So the question is, what is considered "newly published"? The Facebook business page is: https://www.facebook.com/JTBentonConsulting/ The Google+ page is: https://plus.google.com/b/114469171569907148968/+JTBentonConsultingAdamstown?pageId=114469171569907148968&hl=en The name registered for Google My Business is: J T Benton Consulting - Mobile Notary Public Any ideas?
Moz Local | | monicastuckey0 -
3 Locations = 1 Facebook Page or = 3 Facebook Pages
What is the best practice? I have a client with 3 locations with an actual address and not virtual. He has 1 Facebook page with the address and phone number of the main location. Do I add locations in Facebook on that main FB page or do I create 2 more FB pages to have one for each location?
Moz Local | | PSLab0 -
Employees using the Dealership's address and popping up as duplicates in Moz Local?
We are using Moz Local and have a client who's salespeople create individual Facebook Pages for themselves and use the storefront's address. They are noted as duplicates. How much does this really hurt local SEO?
Moz Local | | 9CloudsInc0 -
What do I do if the business listed in Moz Local Business Listing for Yellow Pages is wrong?
Hey there! I have a business listing set up in Moz Local. The system seems to have accurately found the business on most networks, but on Yellow Pages, it has a completely different business listed (my business is a Pilates studio and it has a soccer club listed). What are the steps I can take to fix this? I can claim my listing on Yellow Pages, but after I do that, how does Moz Local associate the two? Thanks, --eric
Moz Local | | EricOliver0 -
What's Going On with Moz Social
Why can't I add my social info to the Moz campaign that I'm currently running?
Moz Local | | maxcarnage0 -
Insider Pages and Citysearch listing issues
On our dashboard, I noticed that our Insider Pages listing (the one with the Insider Pages symbol) is linking to the listing for Citysearch. It is also the same link used for our Citysearch listing. Why is that? I managed to find our Insider Pages listing through a search on Google, but why isn't our Insider Pages listing in the dashboard going to that instead? Secondly, our Citysearch listing is showing a "PowerListings Synced" below the description, which indicates that it is managed by Yext. However, to our knowledge, we have not used Yext for that location. Could that be an issue? If we make an update to our listing, will it still be pushed to Citysearch? Thanks!
Moz Local | | Ree17010 -
Insider Pages & City Search Update Not Going Through
Hello, I updated our company phone number about 2 months ago and every aggregator updated except Insider Pages and City Search. They both have empty circles and are showing as Inconsistent when opening up the campaign because they have the old phone number. I thought for awhile it would just take some time, but it doesn't seem to be updating at all. Is it possible that it's stuck? A lot of the others changed to half circles, then full. But this one is just sitting as an empty circle. Thanks, Chris
Moz Local | | Cincinnati_WebTec0 -
How can I add the county (I'm in the UK) to my Facebook page
I am trying to setup a Moz Local campaign but I am unable to change the county (I am in the UK). In the Moz report, my 'state' is listed as 'Cnwll', but it should be 'Cornwall'. My Google + page has the correct information and is verified. My Facebook page has the correct information, apart from the fact that I can't add add a county. Does anyone have any suggestions on where I could go from here? Thanks.
Moz Local | | Nanuq0