Automated checking for broken links within content pieces
-
Hi, I am wondering if anyone can send me in the right direction on a system suggestion.
We have currently grown out amount of content pieces on our website and our manual checking if the links in the content pieces are still 200 status is becoming extremely time consuming. Does anyone have a recommendation of a system that will crawl your pages and check both the internal and external links within the content for a status code (404,200,etc)? Preferably something server side so it can just run on a schedule but really anything would be fine.
I have tried things like Screaming frog, etc and it just doesn't seem to be the right tool.
-
Try ScreamingFrog again Jonathan, it works great for these kind of things and should also be able to solve your use case.
-
Jonathan, I'm not sure why you're saying that Screaming Frog isn't the right tool--we use it with great success to check the internal links on the site. There are other tools that you can use, such as Integrity (on a Mac), or Xenu, which is an older link checker but still works.
-
Have you tried http://www.link-assistant.com/website-auditor/ as it checks for broken links and can be scheduled to run automatically. You can sit it on your own server or something like AWS. We ran it on a free instance of AWS for quite a while before upgrading and never had issues. We upgraded as we run quite a bit of software on there - still isn't huge costs involved.
Hope this helps!
Matt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ratings pages are Duplicate Content
This brought up another question. should the review page (which now has a canonical to the item page) be Index,follow? My item review pages are showing up with Duplicate Content errors in MOZ. Here are two examples http://www.americanmusical.com/ItemReview--i-HAM-SK1-LIST http://www.americanmusical.com/ItemReview--i-MAC-203680902-LIST is the problem that the pages contain the same code and questions with very little customer created info?
On-Page Optimization | | dianeb1520 -
Use External Links
Hey 🙂 I noticed when analysing my pages that Moz gives the following advice about adding external links to my articles; "On any page specifically targeting a keyword, link externally to at least one (and possibly more than one) relevant, trusted resources as a best practice." As a small business I work pretty damn hard to get visitors to my website, so why on earth would I want to go to all that trouble just to send them away again to a trusted resouce? Secondly, what exactly is a "trusted resource"? Can I simply use search and use the top competitor, for example Moz or Wikipedia and does the anchor need to be an exact match or will a partial suffice. I say this because I already have the top spot for my longtail, so an exact match would be pointless. And lastly, I notice that pretty much all quality sites use external links to open in the same window i.e. not target=_blank, I never thought of it before today, but now that I'm considering using external linking in my articles I guess it's important to know the answer - i.e. Is this a best practice and does this give any seo benefit? Cheers, Lee :)
On-Page Optimization | | LeeC0 -
No followed links, what happens to the PR?
Hi, I have read a few times, on here and other places that when a website applies a no follow tag to a link the PR is not retained but instead disappears (evaporates) thus neither website benefiting. Is that true? If so what is the actual benefit of no following a link?
On-Page Optimization | | Bondara0 -
Related Items Links
Hi, If an item page has 'related items' links and 'you may also like' links - and both those are essentially the same links, doesn't it make sense to only have one set?
On-Page Optimization | | Freelancer130 -
How to handle lots of outbound links
I decided to create a page on my website where I would list all of my favorite resources and 3rd party tools. There are now 35 links in the main content section of the page, all with anchor text, pointing to websites in my industry. My question is this: what is the best practice here? Should I add nofollow tags to the links? Should I do something else to indicate that these links shouldn't be crawled? Frankly, I don't mind passing some link juice to these tools (in this case, and from this particular website), but I might make a different decision with a client's website. Does anyone have any thoughts on this? Here is the page I'm referring to, in case anyone wants to look: http://willmarlow.com/resources-2/the-digital-marketing-toolbox/. Thanks!
On-Page Optimization | | williammarlow0 -
Content by Country
Currently we have a news website aimed at several countries. We want to filter the content of some url (home, category pages, ...) using the country of origin of the visitor. For example in the home we've heard of global character, and a column with news of the country of origin of the visitor. This may affect the position or cause a Google penalty? thank you very much
On-Page Optimization | | promonet0 -
Footer link to home page?
Quick question - is it a best practice to add a footer link on each page of a website that points back to your home page, with the anchor text being your official brand name?
On-Page Optimization | | Bandicoot0 -
Content within JavaSccript code
I know that it is not a good practice to inlcude SEO content within JavaScript, but are there exceptions to what Google can spider or is it best to just avoid completely?
On-Page Optimization | | mjmorse0