Can SEO Moz perform a full site crawl and provide a report showing all URLs within an existing domain?
-
We are conducting a site redesign and need to get an idea of all pages that are out there on our domain (in some report fashion). This would help for discovery and cleanup as we re-work the site and move to a new CMS.
Thanks
-
Hey Stephen,
THanks - I have heard of Xenu many times before. I'll try that out.
I appreciate the input!
-
just going to throw in an alternative for this type of thing Xenu Link sleuth - free, easy and powerful.
Yes that site looks like ass but the tool is great. Dr Pete and others have done "how to use" posts on seomoz - just search xenu on seomoz
-
Yes - that's exactly it. Can I just do an export of the campaign crawl? or is that found somewhere else? Forgive my ignorance...still new to the tool and just can't seem to find.
-
Are you looking at your crawl report? It is available in an Excel file which provides the exact information you desire.
-
Hey Ryan,
Thanks for the response.
It's not a huge site. I would say there's probably around 600 total pages. It definitely looks like the SEO crawler has crawled the entire site, but I can't seem to find a way to just get a one-shot view of all the pages it's accessed.
Any ideas on where this can be found?
Thanks again!
-
How many pages are on your site?
The SEOmoz crawler will do exactly what you want but there is a URL limit. You should also understand it will experience the same limitations of any crawler. It wont list pages which behind a user log-in nor pages which are blocked by robots.txt. I am pretty sure the SEOmoz crawler doesn't list secure pages either but last I heard they were working on including them.
If your site has any island pages they wont be discovered either.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Subdomain or Country Code Top Level Domain
Hello Friends, I am planning to develop my website for other languages. which one is best for SEO? (The main English website is well ranked in google) 1. de.example.com (subdomain may obtain rank faster as it is part of the main website?)
Web Design | | Bold
2. example.de (this is a completely new one; so not easy to get rank?) thank you
Barsbold0 -
Is having a site map page necessary?
Hello all! So I know having a sitemap XML file is important to include in your robots.txt file. I also know it is important to submit your XML sitemap to Google and Bing. However, I am wondering if it is beneficial for your site's SEO value to have a sitemap page displayed on your website? Or is this just a redundant action if you have already done the above two actions with your XML sitemap? Thanks in advance!
Web Design | | Myles920 -
Old Speed Reports
Hey all,
Web Design | | SEOSponge
We just moved our website to WordPress and trying to compare the new reports we are receiving for page speed/user experience to the old ones. Looking at dates before the relaunch, it looks as if old site speed reports are unavailable? Does anyone know of a way to recover older reports? We want to compare the responsive site/new desktop version to the old one (end users would have to "pinch" the screen to look at content on old site). Thanks in advance if you have any info!!!!0 -
Mobile SEO vs. Usability - SinglePlatform
I have a restaurant client that we're setting up with SinglePlatform.com to distribute their menu and make sure all of the restaurant sites have updated information for their business. As part of SinglePlatform's service, they offer a mobile site. Normally we would just create a mobile site and make sure that it's optimized but this client isn't ready to invest in a customized mobile site yet. The mobile site we can get with SinglePlatform is very simple. Call, Menu, Address, View Full Website, Photos, General Info. I know this would make it much easier for mobile users to find information and contact them but it's not mobile best practices. Whatever main page they land on would redirect them to the home page of the mobile site (i.e. not a 1 to 1). We also won't have any Google Analytics information for this site. The question comes down to usability or SEO? I'm leaning toward the mobile site for now and sell them on a customized solution later. I guess I'm just looking for some verification or any insight. ZRVbARv
Web Design | | JaredDetroit0 -
Http://www.domain.com/services/city or http://www.domain.com/city
Hi, I have a website created on a WordPress Platform. My site is in the "Service" industry and we perform the same service for many different "cities." My question pertains to the Navigation Bar and SEO. Is it better to have a "Service" tab on the navigation bar - that has sub navigation that lists all the cities on a drop down menu. When this happens the URL string looks like http://www.domain.com/service/city The other choice would be to create individual TABS on the NAV bar, by doing this the URL string would look - http://www.domain.com/city . I could be wrong, but I am assuming the http://www.domain.com/city is better than http://www.domain.com/services/city for SEO purposes, ....if I am correct is there a way to make SUB Menu URLS appear as http://www.domain.com/city ? Any input as always would be appreciated best regards, Jimmy
Web Design | | jimmy02251 -
Site Ranks on Page 1 - Would launching new site hurt that
Hello, I currently have a website ranking in the top 7 for my main keyword. The website was built in 2004 and is definitely outdated, yet still ranks very high and brings in business. If i launched a new site on this domain, what would happen to my rankings? Would they drop? would they rise? If i don't launch the new site, will this site eventually drop due to being old and outdated? Any advice would be helpful...
Web Design | | Prime850 -
Flash and SEO
When I search the seomoz site for "flash and seo" I only see older articles, 2008. Is there anything newer you can point me to for a current discussion on the impact on SEO of flash sites?
Web Design | | panderson3210 -
Seo for flash home page?
I have a client who insists their home page be a single graphic image of their logo with php menus linking to all of the other pages up top. This appears to me to be an seo nightmare. They seem to be unwilling to have anything to do with changing the appearance and want to rely on seo for "all the other pages" on the site. What's an SEO to do in this situation? Is it possible to have a flash image that lands on a "homepage" for google to crawl, rather than a single image? What's the best seo approach here?
Web Design | | peaceland0