Broken Inner Links - Tool Recommendations?
-
Do you have any recommendations for tools that scan an entire website and report broken inner links?
I run several UGC centered websites and broken inner links, and external, is an issue.
Being that these websites are several hundred thousand pages large, I am not really all that excited about running software on my desktop (xenu link sleuth for example). Any online solutions you could recommend would be great!
-
If it happens to be a wordpress site, there is a plugin called something like "Broken Link Checker." If I recall correctly, that checks internal and outbound links. Otherwise, not too sure.
-
Ideric, did any of these suggestions answer your questions, or have you been able to otherwise find a tool for this? I know others would find the information useful.
At a previous company, we had a custom-written solution to check external links, and made it check response headers until a 200 OK showed, or it got five levels deep. What we'd often find is that we'd have a 301 for an external link, and it'd go from non-www to www. Wouldn't necessarily worry about fixing that, but then later realized that from there, the www link was a 404, OR went to a 200 OK category landing page that said "we've reorganized our site, search here for that individual resource".
-
Well you've found the best solution right here at SEOmoz! Instead of wasting time learning new systems to find out if they'll work or not, just solve your problem. Sign with PRO Elite and you can crawl 100,000 pages.
-
I have used this in the past http://www.auditmypc.com/free-sitemap-generator.asp - (Click on the image in the top right of the instructions) a free tool for site map generation that will show broken internal links in the process. I don't think it has any limits to it, although I have not tried it on a site as large as you are suggesting. Just ensure you are not logged into your site when you run it. Although Google webmaster tools is ok, you can't verify changes made very quickly.
-
I think Xenu is your best option here. The size of the site nearly cuts out the chance a web tool could handle it.
Just recently on a site review I had to run Xenu on a site with 160,000 pages. It only took 4 hours running at 30 threads to complete. Any modern PC should handle it fine.
-
WMT is alright, apart from the fact you can't force Google to crawl all your pages. I would doubt that even a majority of the pages were crawled and indexed by Google (though I don't know what the site is).
Plus, as you say, it only deals with internal links and 404s coming in.
Do you know what the upper limit is on how many crawl errors WMT will display?
-
I might be wrong, but I think Google WMT can accomplish this with ease. I'm looking at 1000 right now. Externally you'll probably have to use xenu =/
-
You might be out of luck on a site that size.
I think WebCEO can do this with their online version but to get 100,000 urls crawled I think it'll cost you a bomb (the sort of money that it'd be cheaper to buy a second PC to run Xenu, lol).
Anyway - http://www.webceo.com/ - I think it may also be possible to install the download version to a server and run it that way.
-
I use Google webmaster tools. Go to diagnostics, then crawl errors.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Worried About Broken Links
In Wordpress, I'm using a plugin called Broken Link Checker to check for broken links. Should I be worried about/spend time fixing outbound links that result in: 403 Forbidden -Server Not Found -Timeout -500 Internal Server Error -etc. Thanks for your help! Mike
Technical SEO | | naturalsociety3 -
Competitors link building surely link farming ? but no punishment?
Hi there added a competitors metrics to see what they were doing and to my amazement they seem to have 1000+ links surely this is link farming considering we stay in a very remote area. also why would he be rewarded for this not punished? imgur.com/18dUqNL 18dUqNL 18dUqNL
Technical SEO | | ShauniBROWN2 -
Links disappeared
Hi, I am a wedding photographer based in Liverpool. I have been trying to do my own SEO for the last 6 months. I have been hovering around the top of page two for the main search terms for the past few years. I used an SEO company before christmas who got a lot of spammy links which resulted in my site dropping to page 4 of the SERPS. With the help of this forum I managed to locate them and disavow those links, and have tried to do it myself. I have managed to gain a few "featured weddings" on national wedding blogs and wrote a few articles for another wedding blog and also some forum comments. I have also got a few links for example from a wedding band in exchange for some photographs. I have got onto page 1 about 4 times, the best result was at position 6 on page 1 but every time I have slowly dropped out again. I have methodically (once a month) checked for any of the spammy links and updated the disavow list. My competitors have at best old forum comments and the like and on checking their websites with open site explorer are not actively link building at all. I have just checked my Webmaster tools and google is only recognising 51 links. (none of my good wedding blog links are there) I have an external links csv from the 28th June with 602 links on it. I changed my website around May of this year but it is still on the same domain name www.dwliverpoolphotography.co.uk. Can anybody help? Best wishes. David.
Technical SEO | | WallerD0 -
About google Disavow tool
My website is attacked by spammed link method, so should i use Goolge disavow tool to remove that links? And i have an question that when i use google Disavow to remove backlinks, but i still not remove it on the webpage that placed my links. Does Google index that backlink again? or never?
Technical SEO | | magician0 -
Unnatural links in webmaster tools
Google Webmaster Tools notice of detected unnatural links to my site.I download latest link from google webmaster tool and decided to remove links from last four months.I also got several links from directory sites and i want to remove the links, how to delete links from directory sites ? I f there is no way to delete directory link , please let me know other option to get rid of this issue.
Technical SEO | | Alick3000 -
External Linking & Your sites Link juice
Hey guys, quick question. Does a page lose link juice when it gives link juice? If I link to an outside site, do I lose that same amount of link juice or is it just applied to there site and not removed from mine? I understand that linking to a competitor can in turn help him and hurt me (if he then is seen as more relevant than me to google) but does it have a direct relation to hurting/removing my page link juice? Hope this all makes sense. Thanks
Technical SEO | | SheffieldMarketing0 -
Is link cloaking bad?
I have a couple of affiliate gaming sites and have been cloaking the links, the reason I do this is to stop have so many external links on my sites. In the robot.txt I tell the bots not to index my cloaked links. Is this bad, or doesnt it really matter? Thanks for your help.
Technical SEO | | jwdesign0 -
Is this seen as a Link Exchange
If i give a self serve banner ad to someone on my blog or a image with a link and they give me a text link ad is that in googles eyes a link exchange or a one way link.
Technical SEO | | DavidKonigsberg0