SEOMOZ Crawling Our Site
-
Hi there,
We get a report from SEOMOZ every week which shows our performance within search. I noticed for our website www.unifor.com.au that it looks through over 10,000 pages, however our website sells less than 500 products so not sure why or how so many pages are trawled? If someone could let me know that would be great. It uses up a lot of bandwidth doing each of these searches so if the amount of pages being trawled reduced it would definitely assist.
Thanks,
Geoff
-
That's a question best answered by a programmer and/or someone who is familiar with the code of your site. Sorry I can't help more!
-
Hi Adam,
Thanks - how do you change the navigation so each page is linked with a single URL?
Thanks
-
If SEOmoz is crawling all those duplicate/unnecessary URLs, the search engines probably are, too.
Best solution: Change the navigation on your site so that each page is linked to using a single URL.
-
Thanks Adam.
Have had a look and it looks like the crawlers are cycling through numerous unnecessarily crawled pages for each product based on the top and bottom nav bar on our website (i.e. it adds FAQ and Shipping Info for each product + all the categories we have created on the website). Attached is an image link of some lines from the CSV file.
I imagine this is what is making it chew up bandwidth. Any insight on how to avoid / change this would be much appreciated.
Thanks
Geoff
-
In addition to what Adam said, you can just search your site on Google and I come up with over 14,000 results:
Both ways should give you a good insight into what's being found.
-
Hi Geoff,
There are multiple reasons your site could have "extra" pages: tag pages, category pages, print this pages, duplicate pages, etc.
To see what pages SEOmoz is crawling:
- Go to http://pro.seomoz.org/campaigns
- Click View This Campaign for your site
- Click Crawl Diagnostics
- At the top right, Export as CSV
- You'll get a spreadsheet listing all the URLs that SEOmoz crawled
Hope that helps!
~Adam
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Major site overhaul
Hi guys and girls After reading all the content It became apparent that our website is a bit of a dog. I'm about to update our site with a new theme and content on moz.com I've found myself becoming so much more confident at all this. I've read all the material I can on Moz re: Site migration but still have one question. How do I find out all the links that I need to 301 before I make the new site live? Please don't worry about patronizing me! I'm really new at this! Ben
Moz Pro | | SussexChef830 -
SEOMoz Langage ?
Hello, Is there any way to change the lagage of the interface ? Especially for reporting tool ? Thanks in advance for your answer, Alex
Moz Pro | | ACORED0 -
Does SEOMOZ use Google Search?
I run a Joomla site and have noticed users having a hard time finding what they are looking for. The default Joomla search is lacking and the latest Joomla search component I added is better but still not great. I've always been able to find what I'm looking for on SeoMoz, do you guys use Google Site Search?
Moz Pro | | mr_w0 -
SEOmoz Rank tracking
I think this tool is a bit off : ) This morning I checked my ranking for a keyword and every time I recheck it I see my results in a different place. I checked on word 5 times and got three different results. Is Google brocken or is the tracker doing its own thing? This is not the first time I am seeing this issue only today its really acting up. IMHO I think it would be best if the tool just says out of order 🙂 rather than 3, 10, 6 for one keyword phrase within ten minutes...
Moz Pro | | SEODinosaur0 -
Crawl Diagnostics Error Spike
With the last crawl update to one of my sites there was a huge spike in errors reported. The errors jumped by 16,659 -- majority of which are under the duplicate title and duplicate content category. When I look at the specific issues it seems that the crawler is crawling a ton of blank pages on the sites blog through pagination. The odd thing is that the site has not been updated in a while and prior to this crawl on Jun 4th there were no reports of these blank pages. Is this something that can be an error on the crawler side of things? Any suggestions on next steps would be greatly appreciated. I'm adding an image of the error spike Xovep.jpg?1 Xovep.jpg?1
Moz Pro | | VanadiumInteractive1 -
Campaign crawl re - schedule
Hello, On the last crawl of a website of mine, seomoz pointed out about 1500 errors (ouch!) on my site. I have made some corrections and i just want to see if they are at the right way but the next crawl is in a week. Is there any way so i can force a crawl before the scheduled date? Thanks!
Moz Pro | | Tz_Seo0 -
Crawl frequency
What is the frequency of crawl. My crawl diagnostics shows data from over a month ago
Moz Pro | | deBreezeInteractive0 -
Site not showing data in Open Site Explorer
I have run open site explorer for a couple of sites but I am getting the "no data available for this URL" message. I have read the reasons listed on the error message but none of these apply to the particular sites I am looking for data from. Any thoughts as to why this might be? Thanks, Kwan
Moz Pro | | SocialKwan0
