Can SEO Moz perform a full site crawl and provide a report showing all URLs within an existing domain?
-
We are conducting a site redesign and need to get an idea of all pages that are out there on our domain (in some report fashion). This would help for discovery and cleanup as we re-work the site and move to a new CMS.
Thanks
-
Hey Stephen,
THanks - I have heard of Xenu many times before. I'll try that out.
I appreciate the input!
-
just going to throw in an alternative for this type of thing Xenu Link sleuth - free, easy and powerful.
Yes that site looks like ass but the tool is great. Dr Pete and others have done "how to use" posts on seomoz - just search xenu on seomoz
-
Yes - that's exactly it. Can I just do an export of the campaign crawl? or is that found somewhere else? Forgive my ignorance...still new to the tool and just can't seem to find.
-
Are you looking at your crawl report? It is available in an Excel file which provides the exact information you desire.
-
Hey Ryan,
Thanks for the response.
It's not a huge site. I would say there's probably around 600 total pages. It definitely looks like the SEO crawler has crawled the entire site, but I can't seem to find a way to just get a one-shot view of all the pages it's accessed.
Any ideas on where this can be found?
Thanks again!
-
How many pages are on your site?
The SEOmoz crawler will do exactly what you want but there is a URL limit. You should also understand it will experience the same limitations of any crawler. It wont list pages which behind a user log-in nor pages which are blocked by robots.txt. I am pretty sure the SEOmoz crawler doesn't list secure pages either but last I heard they were working on including them.
If your site has any island pages they wont be discovered either.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What things i should do to add more attractiveness in my site?
I have 4 years of experience in feedback and survey of different companies for the people who want to learn some from online education about how to do something? and people want to get ease in daily life skills Kroger feedback provides them with a platform to learn everything about surveys and how to drive some to its highest peaks. It all works for People out there through Education SO I have now my own website ready to go but there is a problem in its on-page ranking factors I shall be thankful to you if some from you answer my question What things I should do to add more attractiveness to my site? take a look at my recent view of my site take a look at this Kroger feedback if somebody from you can answer my request??
Web Design | | SEOTOOLS021 -
Why is my financial services site being flagged as gambling
Watchguard and Websense/Forecepoint are flagging my financial services site gambling...how can I prevent that from happening. https://fwag.com/
Web Design | | AdsposureDev0 -
Why is google still crawling my old website pages?
Why is google still looking at my old indexed pages and not my new index. ? Why are they crawling my old website links when none of them are available? How do I overcome these problems?
Web Design | | optimalspaces0 -
Do Google Fonts Slow Down Your Site?
Hi Guys,
Web Design | | jeeyer
I just did a webpage speed test on http://www.webpagetest.org to see how our site is performing.
I noticed that an exteral URL called fonts.gstatic.com has a "huge" impact on our sites loading time. See a screen here: http://monosnap.com/image/z6drzC2ELoJ48d1rM0Tmtuszl3pFpH#
An overview can be seen here: http://monosnap.com/image/9hofUpr5Ld8D7mi7zyaJmGFIGhpBsY# All our scores are green and A (finally!) but I was a bit concerned when I saw the outcome of the pagespeedtest regarding the fonts.
When I load a page on my pc I indeed notice that the text content is usally quite slow in showing up, pops up afer a few seconds. Is this a know problem and Is this something I need to fix? If so what is the best approach? Looking forward on your thoughts!
Joost1 -
How to Redirect Old Domain to a New Domain?
One of my clients has chosen a new domain for her website so we will redirect the old domain to the new. I understand that once redirected, most of the authority built up over the years for the old domain will pass through to the new domain. Is this correct? Can this be done through the Wordpress dashboard, say with a plugin perhaps? I have a plugin called 404 Redirected I was going to try to use for this purpose. Finally, the client has other domains (5 total) she has used for one reason or another. She wants to consolidate all of them and have everything redirected to her new domain. Do you see any downside to this in terms of crawling, Google trust and authority? I know there are some user experience issues to deal with, but we're addressing those. I just don't want to do anything that may cause Google not to trust the new domain. Thanks, Dino
Web Design | | Dino640 -
How can i embed my video into a table using SEO embed setting?
We use Wistia.com to embed our videos. They have different options for embed settings and we prefer to use the SEO embed setting, however, when we use that setting we aren't able to insert the video in a table side by side with another image or text. When we try, the video jumps out of the table and the table gets (for lack of a better work) out of wack. When we embed the video with the iframe embed setting, the video can be placed in a table with no issues, but then we don't get the SEO credit. We have our site in wordpress. I'm not sure if that has something to do with the tables getting messed up. Check out this link to see an example of how we want the video to show up. http://www.3000doorhangers.com/ Any suggestions as to how we can use the SEO embed setting within a table as shown in the above link?
Web Design | | JimDirectMailCoach0 -
Two sites in same industry and which shopping cart
Right. So I suspect I am going to sound paranoid here - but you'll all forgive me right?? I am sure I saw a reply to a question on the Q&A suggesting that it was a bad idea to have two sites in one industry as Google may see it as trying to get two bites of the SERP cherry... is this accurate? I have an existing asp.net site in the maternity wear industry here in Australia and am wanting to start another site to appeal to a different customer base... the market is quite broad. There will be a core list of products that are the same between the sites, but also some quite different products. Content, product descriptions and categorys will be different. I have another website that I bought with reasonable age and links in the industry that I was going to 301 to the new site to give it a kick in the juice. So, not wanting to deceive my customers in anyway, I was thinking I would call it a "division of" or "sister site to" the existing ecommerce site, with a single link back and forward between the two sites. Would there be anything wrong with this in googles eyes? Even with same contact details? They would be run on totally different platforms and hosted by totally different providers. Or would you keep them totally seperate and only have contact details in images? Or a step further and have totally different phone numbers etc? Then the shopping cart - I would love some suggestions on which opensourse cart to use, preferrably one that I can set up myself, and that has a good framework for seo. I want to use schema.org, authorship, seo friendly urls all of which I am having trouble getting out of the developer of my asp.net site.... I don't want the new site to be asp.net Thanks in advance!!
Web Design | | catfree0 -
Effect of Off-Site Images
I'm getting to start work with a new client, and I've run across something I've never had to deal with before, off-site images. The site I'll be working on is for an appliance retailer, both online and physical. The way they've had their site built (not something I was part of) a third party company maintains the product inventory side of things. They're sourcing from about 35 different manufacturers, and this third party has direct access to the product information streams. They push the weekly updated information to my clients site. What this means, though, is that the product images don't live on the client's site. They're hotlinked from the third party's inventory doohickey. I've never seen something quite like this before. Has anyone else? Any ideas as to what problems I may face when it comes to on-site SEO?
Web Design | | MRCSearch0