Xenu Alternative for Large Sites
-
We're launching a new site and we're trying to crawl it to check for any problems. It's millions of pages and Xenu seems to start encountering errors as the numbers mount past 500,000. Does anyone know of an alternative, free or paid, that could handle the size better?
-
There's a post in the last 24 hours about Xenu and Screaming Frog, it's worth a read to see if a) screaming frog my work for you and b) if the 64 bit version of Xenu mentioned in the comments can solve any problems.
http://www.seomoz.org/blog/crawler-faceoff-xenu-vs-screaming-frog
-
WEB CEO and seopowersuite try them if it doesnt work: google webmaster tools but you will have to wait until it crawls your site, and not last thing seomoz crawling it has 10 000 limit minimum but it will be enougth to detect major problems
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site Redesign: 302 Query
Hi there, We'll be redesigning our website www.example.com and as such want to 302 users from www.example.com and all other pages to a new URL www.example.com/landingpage while we go through the redesign. The new landing page will have copy and a sign up form on it and once the redesign is completed, we plan on removing the 302 and sending all traffic back to the original url www.example.com. I'd just like to check that a 302 is the most relevant option here? Obviously, once redesign is completed we'll 301 any old URLs to their new locations once completed.
Technical SEO | | Hemblem0 -
Why is this site ranking better than me
Hi just used the compare tool to try and find out why a site is ranking better than me http://www.opensiteexplorer.org/comparisons?site=www.lifestylemonthly.co.uk%2F my site is www.in2town.co.uk and the site i am comparing with is http://www.lifestylemonthly.co.uk/ Can anyone explain what is going on and how i can achieve better ranking results
Technical SEO | | ClaireH-1848860 -
How is this site doing this?
http://www.meccabingo.com It shows a splash / promotion page yet you check the cache and it's the real homepage, they are doing this so they don't lose rankings but how are they redirecting users to that but Google is caching the real homepage? is it friendly? thanks!!
Technical SEO | | AdiRste0 -
Local business with multiple sites
I'm auditing a local business' sites (a spa) and I wanted to run my recommendations by everyone. There are 3 sites: www.sitename1.com -- main store location, used for Google Places listing #1 www.sitename2.com -- 2nd store location, used for Google Places listing #2 www.sitename3.com -- used for product sales for both locations Sitename1.com has the most ranking power. I'm going to recommend that they move sitename2.com and sitename3.com to sitename1.com as subfolders, 301 redirecting each page to the corresponding page on sitename1.com/subfolder. Google Places listing #2 would be changed from www.sitename2.com to www.sitename.com/location2. Any risks or problems with this strategy anyone can see?
Technical SEO | | 540SEO0 -
Google and QnA sites
My website has a QnA site - a bit like this one except it's not private to premium members. It is a page with a left colomn for category links and it has a list of recently asked questions, each question is a link to view the full question and answers etc. Does google know this is a QnA ? Or will it say - hey, there are far too many links on this page, tut tut. Is there anything I can do to help it understand what the page is.
Technical SEO | | borderbound0 -
My site has vanished from google
Hi my site has vanished from google. We have been for a very long time. for example if you put in gastric band hypnotherapy then we would be first page number two and also lots of other keywords but now we have vanished from google and i do not know why or how to solve this. can anyone please help me and help me understand what i need to do to solve this please My site is http://www.clairehegarty.co.uk I am not sure if i have been banned or why i have dropped out of google
Technical SEO | | ClaireH-1848860 -
Recently revamped site structure - now not even ranking for brand name, but lots of content - what happened? (Yup, the site has been crawled a few times since) Any ideas? Did I make a classic mistake? Any advise appreciated :)
I've completely disappeared off Google - what happened? Even my brand name keyword does not bring up my website - I feel lost, confused and baffled on what my next steps should be. ANY advice would be welcome, since there's no going back to the way the site was set up.
Technical SEO | | JeanieWalker0 -
301 redirect on the root of the site
Due to some historic difficulties with our URL Rewriter, we are in the position of having the root of our site 301 redirected to another page. So the root of our site: http://www.propertylive.co.uk/ has a 301 redirect to: http://www.propertylive.co.uk/home.aspx We're aware that this isn't great and we're working to fix this completely, but what impact will this have on our SEO?
Technical SEO | | LianWard860