Automated checking for broken links within content pieces
-
Hi, I am wondering if anyone can send me in the right direction on a system suggestion.
We have currently grown out amount of content pieces on our website and our manual checking if the links in the content pieces are still 200 status is becoming extremely time consuming. Does anyone have a recommendation of a system that will crawl your pages and check both the internal and external links within the content for a status code (404,200,etc)? Preferably something server side so it can just run on a schedule but really anything would be fine.
I have tried things like Screaming frog, etc and it just doesn't seem to be the right tool.
-
Try ScreamingFrog again Jonathan, it works great for these kind of things and should also be able to solve your use case.
-
Jonathan, I'm not sure why you're saying that Screaming Frog isn't the right tool--we use it with great success to check the internal links on the site. There are other tools that you can use, such as Integrity (on a Mac), or Xenu, which is an older link checker but still works.
-
Have you tried http://www.link-assistant.com/website-auditor/ as it checks for broken links and can be scheduled to run automatically. You can sit it on your own server or something like AWS. We ran it on a free instance of AWS for quite a while before upgrading and never had issues. We upgraded as we run quite a bit of software on there - still isn't huge costs involved.
Hope this helps!
Matt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Content above the Fold, or Below
Hi, I have an ecommerce site with several categories that I consider good landing pages. In order to get better search results I add content to these pages, usually above the fold, then after the content products are listed. Example:https://www.carburetor-parts.com/Carburetor-Kits_c_568.html I worry that customers get to the page and since they don't see the products above the fold, they move on. Should I be putting content in the footer instead of the header and if so how does that effect SEO? This has been bugging me for a long time. Thanks
On-Page Optimization | | MikeCarbs
Mike0 -
How to deal with this duplicate content
Hello our websites offers prayer times in the US and UK. The problem is that we have nearby towns where the prayer times are the same and the pages (exp : https://prayer-times.us/prayer-times-lake-michigan-12258-en and https://prayer-times.us/prayer-times-lake-12147-en) are in duplicate . Same issue for this page https://prayer-time.uk/prayer-times-wallsend-411-en How can we solve this problem
On-Page Optimization | | Zakirou0 -
Should I nofollow certain outbound links?
Hi, All of our outbound links are currently follow.I've read that the only case where nofollow should be used, according to Google, is for paid links, crawl prioritization & untrustworthy sites.The kinds of websites we are linking to from our blog include:- Websites with great content & high authority that is relevant to the topic we've written about that would enhance the user experience- Partners/companies we have a strong relationship with who also have decent authority- Social media profiles of industry people not within our organisation - Websites with definitions/wikipedia/soundcloud/published statistics/news sites and other large well known sites. I am wondering if we should put the nofollow attribute on the last 2 points (social media profiles of people we've referenced, websites with definitions/wikipedia/soundcloud/published statistics/news sites and other large well known sites.)Or do you think it's totally fine that all our links are follow provided the websites are legitimate, trustworthy and have some authority?Thanks in advance!
On-Page Optimization | | PGAUE0 -
Navigation Links Causing Too Many Links Help?
Hello, I have read some SEOMOZ search results for this, but am still concerned that Google may see 4,500 Too Many Link warnings as a problem. This is caused primarily due to our header navigation, which is not intended to be keyword stuffing, but to provide all avenues for our breadth of content. site: crazymikesapps.com. Most answers seem to advise if there is no keyword stuffing at hand don't worry about it. Any help appreciated. thank you Mike
On-Page Optimization | | crazymikesapps0 -
Duplicate content from category pages?
I have an ecommerce store with different categories for my products. Some products do appear in more than one category, is that an issue even if you end up on the same product page/link? Also, I have a "show all products" category, which I believe creates duplicate content too? What is your take on this? What can I do to solve this? Is it even an issue of duplicate content? All answers are very much appreciated!
On-Page Optimization | | danielpett0 -
Footer-links (navigation)
I want to put navigation links in my footer. I watched the (distilled) whiteboard friday video saying it can be a penalty. How are links in the footer different from links in the navigation. Why don't you get "hit" for putting the word "Real Estate Listings" in the navigation menu? It appears on every one of the 30k pages. I don't get why the links in the footer would have a negative effect if the links in the navigation don't. My example is a real estate website. When people are browsing properties (30k), it is convenient that at the end of the page there is a navigation of areas. I could put the navigation on the sidebar but that would be in essence, the same thing. A wordpress site would dynamically include that sidebar on every page.
On-Page Optimization | | JML11790 -
Feedback on Our Mainsite - Rip it to Pieces
Hi Guys We run a marketing company based in the Uk and rank well for internet marketing on google but conversions and bounce rates are quite high. The site is one of those constantly in progress due to Time and budgets so we are well aware of the short comings. I wanted to however get some other peoples opinions on why our site doesnt convert very well and is there anything you can suggest. You never stop learning I have been in internet marketing for 10 years so willing to listen to any suggestions. Thanks here is the site - http://www.onlinemediadirect.co.uk
On-Page Optimization | | onlinemediadirect0