My crawl is only crawling only the 1st page. What’s going wrong?
-
When I crawl the website kmtstore.com usiing the moz tool , it crawls the product urls only the fist page and skips the rest of the products on the remainaing pages .
Here are the produclist page urls which has the products on many pages but on crawling only the 1st page products show up . I have crawled this website kmtstore.com yesterday and all the details being given is as per the latest crawl report generated yesterday./
http://kmtstore.com/pl/11099/H2O-Jet-Parts-Flow-Style.aspx
http://kmtstore.com/pl/11103/Pneumatic-Control-Valves.aspx
http://kmtstore.com/pl/11104/Swivel-Joint-Assemblies.aspx
http://kmtstore.com/pl/11105/Cutting-Head-Components/Cutting-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11106/Cutting-Head-Components/Focusing-Tubes.aspx
http://kmtstore.com/pl/11107/Cutting-Head-Components/Cutting-Heads.aspx
http://kmtstore.com/pl/11108/Cutting-Head-Components/Orifice.aspx
http://kmtstore.com/pl/11109/HP-Valves-Fittings-and-Tubing/Tubing.aspx
http://kmtstore.com/pl/11110/HP-Valves-Fittings-and-Tubing/Inline-Water-Filter.aspx
http://kmtstore.com/pl/11111/HP-Valves-Fittings-and-Tubing/High-Pressure-Fittings.aspx
http://kmtstore.com/pl/11112/HP-Valves-Fittings-and-Tubing/Gauges.aspx
http://kmtstore.com/pl/11113/Intensifier-Parts/Electrical.aspx
http://kmtstore.com/pl/11114/Intensifier-Parts/Low-Pressure-Water-Systems.aspx
http://kmtstore.com/pl/11115/Intensifier-Parts/Topworks-Replacement-Parts.aspx
http://kmtstore.com/pl/11116/Intensifier-Parts/Hydraulic-System.aspx
http://kmtstore.com/pl/11117/Intensifier-Parts/Sealing-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11118/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Coning-and-Threads.aspx
http://kmtstore.com/pl/11119/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Topworks.aspx
http://kmtstore.com/pl/11120/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Lubricants.aspx
http://kmtstore.com/pl/11121/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Pneumatic-Valves.aspx
http://kmtstore.com/pl/11122/Maintenance-Tools-and-Lubricants/Manuals.aspx
http://kmtstore.com/pl/11123/Pneumatic-Control-Valves/Pneumatic-Control-Valve-Replacement-Parts.aspx
http://kmtstore.com/pl/11124/Pneumatic-Control-Valves/Cutting-Head-Nozzle-Tubes.aspx -
Hi there!
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. I appologize for any confusion here. I downloaded your crawl CSV for kmtstore.com, and everything looked fine. From what I can tell Google has 296 pages indexed for your site, and we crawled 241.
Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. Thus, your homepage is the only seed we use to crawl your site. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way. Since we use recursive crawling, if these product pages aren't linked to your homepage, our crawler won't be able to pick them up.
I hope this helps, and if you have any other questions, please reach out to us at help@moz.com! We'll be able to take this conversation offline and answer any other questions for you from there!
Cheers!
Erin
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long do Facebook and Google pages have to be published before Moz can verify them?
Hi there. I've checked the forums and have seen similar questions, but couldn't find an answer in them. I have a listing that I imported from my Google My Business account, but it comes up as unverified for both Facebook and Google. Both pages exist, but have only been published for about 3 weeks. Both sites contain the accurate business information and are the appropriate types of pages, so the best I can determine from what I've read in the other posts is that "newly" published Facebook and Google pages may not be able to be verified by Moz Local. So the question is, what is considered "newly published"? The Facebook business page is: https://www.facebook.com/JTBentonConsulting/ The Google+ page is: https://plus.google.com/b/114469171569907148968/+JTBentonConsultingAdamstown?pageId=114469171569907148968&hl=en The name registered for Google My Business is: J T Benton Consulting - Mobile Notary Public Any ideas?
Moz Local | | monicastuckey0 -
3 Locations = 1 Facebook Page or = 3 Facebook Pages
What is the best practice? I have a client with 3 locations with an actual address and not virtual. He has 1 Facebook page with the address and phone number of the main location. Do I add locations in Facebook on that main FB page or do I create 2 more FB pages to have one for each location?
Moz Local | | PSLab0 -
Moz Local showing wrong Acxiom Listing
Hey there -- The Acxiom listing that is showing up for a client in Moz Local isn't the clients business. It's showing a different law office. The only overlapping data is the city, state and zip code. For reference, the client has claimed their Acxiom business listing. Anything I can do to fix this? I can't even click on the listing from Moz.
Moz Local | | ApogeeResults0 -
Employees using the Dealership's address and popping up as duplicates in Moz Local?
We are using Moz Local and have a client who's salespeople create individual Facebook Pages for themselves and use the storefront's address. They are noted as duplicates. How much does this really hurt local SEO?
Moz Local | | 9CloudsInc0 -
What do I do if the business listed in Moz Local Business Listing for Yellow Pages is wrong?
Hey there! I have a business listing set up in Moz Local. The system seems to have accurately found the business on most networks, but on Yellow Pages, it has a completely different business listed (my business is a Pilates studio and it has a soccer club listed). What are the steps I can take to fix this? I can claim my listing on Yellow Pages, but after I do that, how does Moz Local associate the two? Thanks, --eric
Moz Local | | EricOliver0 -
I updated my Google My Business page weeks ago and the name of my business still has not changed in Moz Local. Of course it won't allow me to change the name. Please help.
I updated my Google My Business page weeks ago and the name of my business still has not changed in Moz Local. Of course it won't allow me to change the name. Please help.
Moz Local | | NicoleDeLeon0 -
You have exceeded your rate limit error while trying to add listing to Mozlocal. What's wrong?
I get the error You have exceeded your rate limit when trying to add my first listing to the MozLocal tool. Can you let me know what's the problem?
Moz Local | | arkenea1 -
What do I do if Moz Local is pulling the wrong business citation?
Hi everyone, I have several business listings that I manage on Moz Local, and a lot of them have similar names. I'm running into an issue where Moz Local is pulling a citation (Ex. Google+, YP, Superpages) for the wrong business, and it is affecting my listing score. Has anyone else ever run into this issue? What did you do about it? Thanks in advance for your help!
Moz Local | | wparlaman0