Can SEO Moz perform a full site crawl and provide a report showing all URLs within an existing domain?
-
We are conducting a site redesign and need to get an idea of all pages that are out there on our domain (in some report fashion). This would help for discovery and cleanup as we re-work the site and move to a new CMS.
Thanks
-
Hey Stephen,
THanks - I have heard of Xenu many times before. I'll try that out.
I appreciate the input!
-
just going to throw in an alternative for this type of thing Xenu Link sleuth - free, easy and powerful.
Yes that site looks like ass but the tool is great. Dr Pete and others have done "how to use" posts on seomoz - just search xenu on seomoz
-
Yes - that's exactly it. Can I just do an export of the campaign crawl? or is that found somewhere else? Forgive my ignorance...still new to the tool and just can't seem to find.
-
Are you looking at your crawl report? It is available in an Excel file which provides the exact information you desire.
-
Hey Ryan,
Thanks for the response.
It's not a huge site. I would say there's probably around 600 total pages. It definitely looks like the SEO crawler has crawled the entire site, but I can't seem to find a way to just get a one-shot view of all the pages it's accessed.
Any ideas on where this can be found?
Thanks again!
-
How many pages are on your site?
The SEOmoz crawler will do exactly what you want but there is a URL limit. You should also understand it will experience the same limitations of any crawler. It wont list pages which behind a user log-in nor pages which are blocked by robots.txt. I am pretty sure the SEOmoz crawler doesn't list secure pages either but last I heard they were working on including them.
If your site has any island pages they wont be discovered either.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How can I fix New 4XX Issue on Site Crawl?
Hi all, My recent site crawl shows 27 4xx issues on this website http://www.rrbusinessconsultants.com/ All of them are for 'posts' on this wordpress website. Here is an example of the issue: http://www.rrbusinessconsultants.com/rr-business-consultants-on-the-rise-of-glassdoor-and-how-companies-are-coping/void(null) The blog page seems to be creating links ending in void(null) which are defaulting to 404 pages. I cannot see the links on the site so cannot see how to remove them. Can anyone provide any insight into how to correct his issue? Many thanks in advance.
Web Design | | skehoe0 -
Free websites that are good with SEO?
Dear members, I am looking for a free/ almost free website which is good for SEO. For example i am looking at WIX right now but i keeping reading that they aren't optimal for SEO. Does anybody has some tips which website is can use, example weebly, strato, etc?? Many thanks!
Web Design | | rijwielcashencarry0400 -
Will SASS ruin my SEO?
Hello, I am thinking about using SASS for my website, striping the current CSS style sheets and translating it all to SASS.. will this hurt my SEO?
Web Design | | DanielBernhardt0 -
Tips on website redesign on site with messy URLs?
So I've inherited quite a messy website. It was in drupal and the owner wants it in wordpress. One of the problems is the link paths. Should I try to recreate them exactly? i.e. something/somethingelse/page/ or use redirects (which I'm not confident in doing). Also, some of the pages end in .html, others in a back slash and others without slahes, there's no consistency. Do you have any tips in general? I remember an older seomoz blogpost about successful website relaunches (with press releases and mass emails and stuff being sent out on launch to boot). Thanks!
Web Design | | seonubblet0 -
Switching from HTML Static to WordPress Platform - SEO
Hi All, Hope everyone is doing well. I am currentyl in the process of having a re-design to myu HTML Static Site, I am switching to WordPress. My site is still running now until the new one is prepared. My question is: I do rank pretty well for some of our strong local keywords, we also have a FEW links pointing to inner pages with an extension of .HTML, should I set the the WordPress URL's to reflect .HTML or will the Search Enginesfigure out that http://www.domain/innerpage.html is the same as http://www.domain/innerpage/ or is it NOT the same. Should I switch or not? Thanks for your input Jimmy
Web Design | | jimmy02250 -
Does page speed worth for SEO?
I always broken my head to try to follow all pagespeed guidelines. I increase my pagespeed significantly, but i didnt saw any effect in my SEO performance. In my keywords, my concorrents are crap on it (I have score of 90 and they are at 60-70).Does google gives importance to it?
Web Design | | Naghirniac0 -
Seo for flash home page?
I have a client who insists their home page be a single graphic image of their logo with php menus linking to all of the other pages up top. This appears to me to be an seo nightmare. They seem to be unwilling to have anything to do with changing the appearance and want to rely on seo for "all the other pages" on the site. What's an SEO to do in this situation? Is it possible to have a flash image that lands on a "homepage" for google to crawl, rather than a single image? What's the best seo approach here?
Web Design | | peaceland0