Can SEO Moz perform a full site crawl and provide a report showing all URLs within an existing domain?
-
We are conducting a site redesign and need to get an idea of all pages that are out there on our domain (in some report fashion). This would help for discovery and cleanup as we re-work the site and move to a new CMS.
Thanks
-
Hey Stephen,
THanks - I have heard of Xenu many times before. I'll try that out.
I appreciate the input!
-
just going to throw in an alternative for this type of thing Xenu Link sleuth - free, easy and powerful.
Yes that site looks like ass but the tool is great. Dr Pete and others have done "how to use" posts on seomoz - just search xenu on seomoz
-
Yes - that's exactly it. Can I just do an export of the campaign crawl? or is that found somewhere else? Forgive my ignorance...still new to the tool and just can't seem to find.
-
Are you looking at your crawl report? It is available in an Excel file which provides the exact information you desire.
-
Hey Ryan,
Thanks for the response.
It's not a huge site. I would say there's probably around 600 total pages. It definitely looks like the SEO crawler has crawled the entire site, but I can't seem to find a way to just get a one-shot view of all the pages it's accessed.
Any ideas on where this can be found?
Thanks again!
-
How many pages are on your site?
The SEOmoz crawler will do exactly what you want but there is a URL limit. You should also understand it will experience the same limitations of any crawler. It wont list pages which behind a user log-in nor pages which are blocked by robots.txt. I am pretty sure the SEOmoz crawler doesn't list secure pages either but last I heard they were working on including them.
If your site has any island pages they wont be discovered either.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help with 302 Temporary Redirect warning via MOZ crawl
Hi Guys, This is my first post so hopefully I'm using the forum correctly. MOZ crawl tells me that I have 35 pages with a temporary redirect The URL column displays 302 Found along with the http:// URL Redirection Location column shows the corresponding https:// URL This all seems pretty self explanatory. However, I’ve checked my .htaccess file and I can’t see any 302 references in it. I'm trying to figure out where the 302 redirects are from and how I can make them permanent Please can anyone help me out? My .htaccess looks like it needs a little tidy (there are 2 if blocks) <ifmodule mod_rewrite.c="">RewriteEngine On
Web Design | | ianalannash
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://www.mysite.com/$1 [R,L]</ifmodule> BEGIN WordPress <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
RewriteCond %{HTTP_HOST} ^mysite.com$ [NC]
RewriteRule ^(.*)$ http://www.mysite.com/$1 [R=301,L]</ifmodule> END WordPress0 -
Does having too many wordpress portfolio pages with little content hurt a site's SEO?
I have a site that is for a service company, not image based like a photographer or artist. We utilize the Portfolio feature to create a gallery of floor coating finishes (images of all the flooring finish options available) but this solution has created /portfolio/file-name pages for each image. These pages have no other content besides the image. I've run SEMrush audits on this site which shows a high percentage of pages with low text/code ratio and duplicate content (a lot of the finishes have very similar names). This site has been extremely slow to improve any visibility online (more than 9 months) and I'm wondering if this is a factor by possibly having a negative effect on our site. We initially chose the portfolio option because it was the best-looking solution for our users but we can certainly change it to another format if that is better. Thanks!
Web Design | | WillGMG0 -
Best Captcha Recommendations for Magento Site?
I am looking for the best captcha solution for our website which is magento based. Currently our web developer is recommending google captcha. Is this just a spam check list or will it do the job well? I would like any other recommendations that are clear for readers and are professional.
Web Design | | TeguarMarketing0 -
How does Google look at strings added to a URL
For example: http://localhost:3000/en-US/app/a-knsmtrhqrqs/personal where knsmtrhqrqs is a string Can Google tell this is a string and what's their policy? Will it hurt rankings? Thank you.
Web Design | | RoxBrock0 -
For a real estate website, is a different mobile site warranted vs a responsive site?
I researched the major real estate websites: Zillow, Trulia, Remax, Keller Williams, Century 21, etc. They all have a separate mobile site and not a responsive one. The client wants it to be easy to search for properties from a smartphone. Is it possible to get a responsive version of a real estate website on a smartphone?
Web Design | | MassMedia0 -
Internal linking for small site
I have a site with 13 pages, 6 are product pages, 5 are free tips pages (the other 2 are the home page and contact form). Currently I have the navbar at top of site with a "products" dropdown menu for the 6 product pages and a "Tips" dropdown menu for the 5 tip pages. All categories except the contact page are at the bottom as breadcrumbs, the homepage is "home" and the rest are relevant user friendly keyword anchor text. So I have 2 more pages to ad to "Tips" and am wondering whether to have a new 2nd level tips page that links to a 3rd level of 7 different tips pages, or keep it shallow as it is, with only 2 levels from the homepage to the other (now 13) pages, with a potential of 22 pages in the foreseable few years? (and some graphics work to make it user friendly like how Zappo's has categories to the side on each of its drop down navbar menu's and non-link text categories for its bottom of page breadcrumb links) Can those aforementioned pages linking to each other in the footer dilute link equity? (I think that's one of the primary reasons I'm curious). What do you think of this: http://www.dbswebsite.com/blog/2012/08/08/internal-linking-101-5-best-practices/ (I guess I should no follow my contact page), could it be better to have a 2nd level page for "Tips" to get more equity to that page rather than across all 7 tips pages? I have read around about this on here (hence how I found out about Zappo's) and elsewhere and wanted ask to make sure.
Web Design | | Zoolander0 -
Can white text over images hurt your SEO?
Hi everyone, I run a travel website that has about 30 pre-search city landing pages. In a redesign last year we added large "hero" images to the top of the page, and put our h1 headlines on top of them in white. The result is attractive, but I'm wondering if Google could be reading this page as "white text on white page", which is an obvious no-no, especially if it could seem that we're trying to hide text. Here's an example: http://www.eurocheapo.com/paris/ H1: Expert reviews of cheap hotels in Paris I should add that our SERPs for these city pages has dropped (for "Cheap hotels in X"), but it could obviously be related to other issues. Any advice would be appreciated. Many thanks! Tom
Web Design | | TomNYC0 -
Finally have a budget for a great seo ecommerce site but need help choosing wordpress, joomla, modx, magneto or? Thank you in advance for your generosity of time
We finally have a budget and want to dump our intuit/homestead site www.originalartbroker.com Our budget is 5k-10k but could do more if needed. I am slowly catching my competition with this homestead site that I built. But I do realize it is time to step back, figure out what is best, and hire a pro to get the job done. I am green in the seo and web development arena so please go easy on me and please help to point me in the right direction. Just went out on a limb a couple years ago playing with homestead.com site software and built what we have today. Didn't know anything about website development...it sort of just happened. I feel and know that homestead.com solution is hindering what we could be doing due to the bloated nature of the site and inability to perform such task as 301 redirects etc.... I have been able to slowly attain first page seo rankings on keywords based of the artists we carry using this po-dunk homestead platform to build my site after a ton of work education thanks to seomoz and a lot of you. But, have never asked for help and could really use some generosity of time in explaining a solution that would work best for our business. Do we just go with a wordpress site that is similar to our current setup and use their plugins? Do we use a cms software solution like magneto or joomla? We will only have 200-300 pieces at any given time. We are constantly selling and buying new pieces providing us content. We are need of a site that can perform well in terms of seo. I have heard of a lot of people talking about joomla, wordpress, and magneto. Would like to be able to have a product catalogue that ultimately sends whatever inventory we are uploading to our social sites and blogs so I don’t have to pump the product out to all of these sites. We offer free custom framing with our pieces and it would be nice to have a program that could wrap the photos of the pieces with the different frames for our customers. When I add a new piece I would like this software to have a predesigned product page that it plugs the information into. I would like it to create the url extension based of the artists name, medium used, and piece name to create unique and individual urls. I would like it to also create its own H tags throughout that product page according to the artist name description, and medium used. I would like to be able to sink this up to google merchant and other sites to carry our product. Bottom line is we sell art. We sell pieces by specific artists. We are constantly buying and selling. I need something powerful that keeps up with our content
Web Design | | forecastedinvestments1