Broken Inner Links - Tool Recommendations?
-
Do you have any recommendations for tools that scan an entire website and report broken inner links?
I run several UGC centered websites and broken inner links, and external, is an issue.
Being that these websites are several hundred thousand pages large, I am not really all that excited about running software on my desktop (xenu link sleuth for example). Any online solutions you could recommend would be great!
-
If it happens to be a wordpress site, there is a plugin called something like "Broken Link Checker." If I recall correctly, that checks internal and outbound links. Otherwise, not too sure.
-
Ideric, did any of these suggestions answer your questions, or have you been able to otherwise find a tool for this? I know others would find the information useful.
At a previous company, we had a custom-written solution to check external links, and made it check response headers until a 200 OK showed, or it got five levels deep. What we'd often find is that we'd have a 301 for an external link, and it'd go from non-www to www. Wouldn't necessarily worry about fixing that, but then later realized that from there, the www link was a 404, OR went to a 200 OK category landing page that said "we've reorganized our site, search here for that individual resource".
-
Well you've found the best solution right here at SEOmoz! Instead of wasting time learning new systems to find out if they'll work or not, just solve your problem. Sign with PRO Elite and you can crawl 100,000 pages.
-
I have used this in the past http://www.auditmypc.com/free-sitemap-generator.asp - (Click on the image in the top right of the instructions) a free tool for site map generation that will show broken internal links in the process. I don't think it has any limits to it, although I have not tried it on a site as large as you are suggesting. Just ensure you are not logged into your site when you run it. Although Google webmaster tools is ok, you can't verify changes made very quickly.
-
I think Xenu is your best option here. The size of the site nearly cuts out the chance a web tool could handle it.
Just recently on a site review I had to run Xenu on a site with 160,000 pages. It only took 4 hours running at 30 threads to complete. Any modern PC should handle it fine.
-
WMT is alright, apart from the fact you can't force Google to crawl all your pages. I would doubt that even a majority of the pages were crawled and indexed by Google (though I don't know what the site is).
Plus, as you say, it only deals with internal links and 404s coming in.
Do you know what the upper limit is on how many crawl errors WMT will display?
-
I might be wrong, but I think Google WMT can accomplish this with ease. I'm looking at 1000 right now. Externally you'll probably have to use xenu =/
-
You might be out of luck on a site that size.
I think WebCEO can do this with their online version but to get 100,000 urls crawled I think it'll cost you a bomb (the sort of money that it'd be cheaper to buy a second PC to run Xenu, lol).
Anyway - http://www.webceo.com/ - I think it may also be possible to install the download version to a server and run it that way.
-
I use Google webmaster tools. Go to diagnostics, then crawl errors.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Regarding Internal Links
I analyse my Birthday Page "https://www.giftalove.com/birthday"with comapare link profiles and found that total Internal Link 47,234. How my internal link suddenly increse. Please provide my details about my internal links.
Technical SEO | | Packersmove0 -
Tool to Generate All the URLs on a Domain
Hi all, I've been using xml-sitemaps.com for a while to generate a list of all the URLs that exist on a domain. However, this tool only works for websites with under 500 URLs on a domain. The paid tool doesn't offer what we are looking for either. I'm hoping someone can help with a recommendation. We're looking for a tool that can: Crawl, and list, all the indexed URLs on a domain, including .pdf and .doc files (ideally in a .xls or .txt file) Crawl multiple domains with unlimited URLs (we have 5 websites with 500+ URLs on them) Seems pretty simple, but we haven't been able to find something that isn't tailored toward management of a single domain or that can crawl a huge volume of content.
Technical SEO | | timfrick0 -
Webmaster Tools Verification Problem
Hello, Somehow a website I'm working on has lost it's verification in Webmaster Tools. I have absolutely no clue why... The Google Analytics code is still working, the verification meta tag is in place, both are not working. I get an error message about Google not being able to connect to the server. I asked about any possible changes to server settings or stuff about that, but apparently nothing has changed there. The URL in question is Bivolino.com Does someone has any other ideas what I could be looking for. Thanks, Kind regards, Erik
Technical SEO | | buiserik0 -
Confused on footer links (Which are best practices for footer links on other websites?)
Hello folks, We are eCommerce web design and Development Company and we give do follow links of our website to every projects which we have done with specific keywords. So now the concern is we are seeing huge amount of back-links are being generated from single root domain for particular keyword in webmaster tools. So what should be the best way to practice this? Should we give no follow attribute to it or can use our company logo with link? LtMjHER.png
Technical SEO | | CommercePundit0 -
Is it a problem to have an image + link in your menu
Hi, My menu has a image with links to some of the main pages on the site and text underneath it explaining what the banner is. Will it be beneficial or harmful to have the text hyperlinked to the same pages the images go to?
Technical SEO | | theLotter0 -
Can I reduce link count by no following links?
Hi, A large number of my pages contain over 100 links. This is due to a large drop down navigation which is on every page. To reduce my link count could I just no follow these navigation links or would I have to remove the navigation completely?
Technical SEO | | moesian0 -
External Sitewide Links and SEO
I have one big question about the potential SEO value -- and possibly also dangers? -- of "followed" external sitewide links. Examples of these would be: a link to your site from another site's footer a blogroll link a link to your site from another site's global navigation Aside from the link's position in the HTML file (the higher the better, presumably), are these links essentially the same from an SEO point of view or different (and how)? There used to be an influential view out there that the link juice value of a sitewide link was the same as that of a single link (presumably from the linking site's home page), even though a sitewide link may in fact result a huge number individual links. Is this true or false? What is the math here? Should one worry about having "too many" sitewide links, in the sense that this may raise red flags by way of the algo? I talked to someone a few months ago (before the recent algo updates) who believed that he had got a minus 10 penalty or whatever it was for getting too many sitewide links We offer website design and development as well as SEO, and we put a keyworded link to ourselves in the footer. I think this is a fairly common practice. Is this a good or bad idea SEO-wise? One opinion is that for external sitewide footer links, you should best have a dofollow link on the home page, but nofollow it on all other pages. What is your opinion about that? Is there anything else that is distinct, interesting or important about sitewide links' SEO value and pitfalls? Thank you!
Technical SEO | | Philip-SEO1 -
Internal Linking: Site-wide VS Content Links
I just watched this video in which Matt Cutts talks about the ancient 100 links per page limit. I often encounter websites which have massive navigation (elaborate main menu, side bar, footer, superfooter...etc) in addition to content area based links. My question is do you think Google passes votes (PageRank and anchor text) differently from template links such as navigation to the ones in the content area, if so have you done any testing to confirm?
Technical SEO | | Dan-Petrovic0