Broken Inner Links - Tool Recommendations?
-
Do you have any recommendations for tools that scan an entire website and report broken inner links?
I run several UGC centered websites and broken inner links, and external, is an issue.
Being that these websites are several hundred thousand pages large, I am not really all that excited about running software on my desktop (xenu link sleuth for example). Any online solutions you could recommend would be great!
-
If it happens to be a wordpress site, there is a plugin called something like "Broken Link Checker." If I recall correctly, that checks internal and outbound links. Otherwise, not too sure.
-
Ideric, did any of these suggestions answer your questions, or have you been able to otherwise find a tool for this? I know others would find the information useful.
At a previous company, we had a custom-written solution to check external links, and made it check response headers until a 200 OK showed, or it got five levels deep. What we'd often find is that we'd have a 301 for an external link, and it'd go from non-www to www. Wouldn't necessarily worry about fixing that, but then later realized that from there, the www link was a 404, OR went to a 200 OK category landing page that said "we've reorganized our site, search here for that individual resource".
-
Well you've found the best solution right here at SEOmoz! Instead of wasting time learning new systems to find out if they'll work or not, just solve your problem. Sign with PRO Elite and you can crawl 100,000 pages.
-
I have used this in the past http://www.auditmypc.com/free-sitemap-generator.asp - (Click on the image in the top right of the instructions) a free tool for site map generation that will show broken internal links in the process. I don't think it has any limits to it, although I have not tried it on a site as large as you are suggesting. Just ensure you are not logged into your site when you run it. Although Google webmaster tools is ok, you can't verify changes made very quickly.
-
I think Xenu is your best option here. The size of the site nearly cuts out the chance a web tool could handle it.
Just recently on a site review I had to run Xenu on a site with 160,000 pages. It only took 4 hours running at 30 threads to complete. Any modern PC should handle it fine.
-
WMT is alright, apart from the fact you can't force Google to crawl all your pages. I would doubt that even a majority of the pages were crawled and indexed by Google (though I don't know what the site is).
Plus, as you say, it only deals with internal links and 404s coming in.
Do you know what the upper limit is on how many crawl errors WMT will display?
-
I might be wrong, but I think Google WMT can accomplish this with ease. I'm looking at 1000 right now. Externally you'll probably have to use xenu =/
-
You might be out of luck on a site that size.
I think WebCEO can do this with their online version but to get 100,000 urls crawled I think it'll cost you a bomb (the sort of money that it'd be cheaper to buy a second PC to run Xenu, lol).
Anyway - http://www.webceo.com/ - I think it may also be possible to install the download version to a server and run it that way.
-
I use Google webmaster tools. Go to diagnostics, then crawl errors.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Massive Influx of Total Links - But External Links are Dropping?
Hey Moz Community, I was checking out the Links on one of my client's sites, as they were hit with spammy external links about a year ago, and noticed a large influx of Total Links to the site. According to Moz, external links have actually dropped over the last few months, so I can only assume they are internal links. But, I don't see how my client could add so many internal links over the past 5 months, as they don't do much besides upload new products (they're an ecommerce clothing retailer) via Shopify. They haven't added much over the past half year either. Total links were about 130K in Oct 2019; today, the site has almost 1 million. I've attached some screenshots for reference via Moz to better illustrate the issue. Appreciate any insights into this. Thank you in advance! hhCCUsk lyGltZD
Technical SEO | | EdenPrez0 -
Are bad links the reason for not ranking?
Hello Moz community. I'm looking here for some input from the experts on what could be wrong with a site I'm working on. The site is in Spanish, but I'm sure you'll get the idea. We want to rank the site first page on Google Mexico (www.google.com.mx) for the keyword "refacciones Audi" and some other brands (refacciones = replacement parts would probably be a good translation, just FYI). Now, our page hasn't been completely optimized, so in my mind it's OK not to be on first page yet. However, our main competitor is ranking first page for all the keywords we want to rank for, but when you check their site, you'll find there is hardly any content, no keywords are being used in their content, all pages have the exact same title and meta description, their catalog is in a completely different domain. In short, no SEO whatsoever. Looking at Moz data, our site has a DA of 26, while our competitor's has a 10. They have no external backlinks at all, while we have a few hundred. This leaves me scratching my head: how can a completely non-optimized site outrank us? I decided to check our backlink profile, and a previous SEO agency seems to have built MANY fake blogs with lots of backlinks with rich anchor text. Quite a big percentage of our backlinks are of this kind, so this is the only thing I can think can be affecting our ranking. Will disavowing be our solution? If you'd like to check, our site is: www.refaccionariaalemana.com.mx Our competitors' is: www.saferefacciones.com ANY help will be extremely appreciated as I feel a bit lost. Thanks!
Technical SEO | | EduardoRuiz1 -
Confused on footer links (Which are best practices for footer links on other websites?)
Hello folks, We are eCommerce web design and Development Company and we give do follow links of our website to every projects which we have done with specific keywords. So now the concern is we are seeing huge amount of back-links are being generated from single root domain for particular keyword in webmaster tools. So what should be the best way to practice this? Should we give no follow attribute to it or can use our company logo with link? LtMjHER.png
Technical SEO | | CommercePundit0 -
Explain me the SEO impact when a website has more internal link compared to less internal links
A website that I am working on has more than 200 internal links (Its because of the design and various kind of service that we offer). I want to know its SEO impact. I also want to know the SEO impact when a website has less internal links compared to more internal links
Technical SEO | | BoniSatani0 -
OUTGOING LINK ISSUES
NO LINK TO HOME MY WEBSIDE PROFITIOK.COM FOR EXAMPLE YOU SEE THE PAGE I HAVE CHANGE TWO DAYS BACK PREVIOUS
Technical SEO | | Luckykhullar
ABOUT US - Profitok.com 2 DAYS BACK I CHANGED THE TITLE title>Commodity tips | Mcx TIPS | Mcx Daily Tips |Mcx Good tips Provider BUT WHEN I CHECKING IT IS NOT UPDATED THIS TITLE DEAR SIR CAN Y GIVE ME THE CORRECT ANSWER0 -
Webmaster Tools Links To Your Site
I logged onto webmaster tools today for my site and the section 'Links to Your Site' is showing no data. Also if I search using link:babskibaby.com it only shows 1 link. My site had been showing 500+ links previously. Does anyone know why this is?
Technical SEO | | babski0 -
No inbound links. Should I link-build or create new content?
I have a PR4 site with good traffic but the blog is not very popular--the posts do not generate any backlinks and hardly get any traffic. Yet, I continue to kick out a new post every week. Site: http://www.stadriemblems.com/
Technical SEO | | UnderRugSwept
Blog: http://www.stadriemblems.com/blog/ I keep posting content so that Google keeps crawling the site and viewing it as fresh (and yes, I'm posting for my human visitors' benefit too!), but I'm wondering if eventually this will hurt more than help if Google detects all these new pages are not being linked to, and therefore starts viewing the site as low quality and devalues it. So should I: Keep posting Stop posting and build links to the posts Try to promote my blog to get more traffic and hope people link to it Something else or some combination of the above0 -
Diagnostic says too many links on a page and most of the pages are from blog entries. Are tags considered links? How do I decrease links?
I just ran my first diagnostic on my site and the results came back were negative in the area of too many links one a page. There were also quite a few 404 errors. What is the best way to fix these problems? Most of the pages with too many links are from blog posts, are the tags counted as well and is this the reason for too many links?
Technical SEO | | Newport10300