Automated checking for broken links within content pieces
-
Hi, I am wondering if anyone can send me in the right direction on a system suggestion.
We have currently grown out amount of content pieces on our website and our manual checking if the links in the content pieces are still 200 status is becoming extremely time consuming. Does anyone have a recommendation of a system that will crawl your pages and check both the internal and external links within the content for a status code (404,200,etc)? Preferably something server side so it can just run on a schedule but really anything would be fine.
I have tried things like Screaming frog, etc and it just doesn't seem to be the right tool.
-
Try ScreamingFrog again Jonathan, it works great for these kind of things and should also be able to solve your use case.
-
Jonathan, I'm not sure why you're saying that Screaming Frog isn't the right tool--we use it with great success to check the internal links on the site. There are other tools that you can use, such as Integrity (on a Mac), or Xenu, which is an older link checker but still works.
-
Have you tried http://www.link-assistant.com/website-auditor/ as it checks for broken links and can be scheduled to run automatically. You can sit it on your own server or something like AWS. We ran it on a free instance of AWS for quite a while before upgrading and never had issues. We upgraded as we run quite a bit of software on there - still isn't huge costs involved.
Hope this helps!
Matt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I change PDF content?
Hi everybody, My Website is ranking well for several keywords and long-tail keywords. However, all these visits are going directly to some .PDF guides that exist on our products and information on industry sectors the company is based around. I feel the PDF's are bad simply because they dont offer easy interaction with the rest of the website. I am considering making each PDF into a webpage but am not 100% sure of the pro's and cons of doing so. I will still need to the PDF's accessible for user to download but don't want my new webpages to get tagged as duplicate content. Is it possible to,
On-Page Optimization | | ATP
1 - change the PDF's so they send any link authority to the new webpage
2 - make google aware that I want the webpage not the PDF to be the "ranking" page What is the likely hood of destroying my rank for these keywords on the PDF by making these changes and then not being able to rank the webpage for the same keywords? It would be pointless if I just lost all the traffic lol.0 -
Webmaster Tools - How your data is linked?
This may be an easy questions, but I can't seem to find the answer anywhere and I never really looked into it before. In google webmaster tools, in the dashboard there is the section that says "How Your Data Is Linked". What does that refer to? Is that just using internal link anchor text, external link anchor text or a combination of both? I am pretty sure that it is a combination of both, but I just want to make sure before making some internal link changes so that the most common anchor text is no longer "Prices" and "Sign up". Thanks.
On-Page Optimization | | rayvensoft0 -
Cannibalizing link
this is what the reports states :Cannibalizing link"Red & Color Roses", "Red roses", "Black Baccara Roses\rBlack And Red Roses\rStarting at: $ 0.72 per Rose", "Black Magic Rose\rDark Red Black Rose\rStarting at: $ 0.96 per Rose", "Forever Young Roses\rRed Rose Flowers\rVelvety Deep Red\rStarting at: $ 1.16 per Rose", "Freedom Roses\rRed Rose\rStarting at: $ 0.66 ¢ per Rose", "Madame Delbard\rRed Wedding Rose\rStarting at: $ 0.66 per Rose", "Night Fever Roses\rTraditional Roses\rRomantic Red Roses\rStarting at: $ 0.66 per Rose", and "Red Paris Roses\rDark Red Rose\rStarting at: $ 0.72 per Rose"ExplanationIt's a best practice in SEO to target each keyword with a single page on your site (sometimes two if you've already achieved high rankings and are seeking a second, indented listing). To prevent engines from potentially seeing a signal that this page is not the intended ranking target and creating additional competition for your page, we suggest staying away from linking internally to another page with the target keyword(s) as the exact anchor text. Note that using modified versions is sometimes fine (for example, if this page targeted the word 'elephants', using 'baby elephants' in anchor text would be just fine).RecommendationUnless there is intent to rank multiple pages for the target keyword, it may be wise to modify the anchor text of this link so it is not an exact match. the questions is then why having those links on that page hurts, when we need the links to take the costumer to the color of red roses they want.
On-Page Optimization | | globalrose.com0 -
Thin content and tabs on page
I am reviewing a site, and the web designer used tabs to impart information. I think the tabs idea looks great, but it leaves the page looking thin. Here is a link to a product page, could anyone chime in please? http://www.aireindustrial.net/spill-berms/foam-berm-drive-over-berms.asp Thanks in advance for your opinion!
On-Page Optimization | | drufast10 -
Content Update
Hello, If I update the existing content i.e.I added some content to the already existing indexed content in a post,how will it effect SEO wise? Venkee
On-Page Optimization | | Venkee0 -
Do we have too many links in our footer?
Hi guys, we have 41 links on our holiday(vacation) rental website, this seems too many when looking at best practice. 24 of these are links to community pages while 8 link to activities pages. The community and activity pages are also accessible from links on the top menu so they are not strictly necessary but do get 10% of site clickthroughs according to Google in-page analytics. I therefore do not want to remove the links if there is no good evidence that google will penalize us for this. What do you think would be best for our site? Thanks, John Tulley. footer.jpg
On-Page Optimization | | JohnTulley0 -
Canonical links
My website is relatively new, January. We climbed steadily to 6th for our search term then overnight rocketed to 1st. This only lasted a week and have been stuck at 9th ever since. When I use the SEO Moz tools our site should theoretically be top...I only joined today btw. Anyway in Google webmaster tools I noticed it said I had duplicate title tags, when I checked to see what the pages were- it was my home page! Google also seems to have cached two versions of our homepage, the root domain and the Default.aspx page. Now I have fixed this canonical linking issue today (using canonical link tag and 301s) so time will tell but has anyone got any first hand experience of this issue? Was it a big factor? Thanks!
On-Page Optimization | | SplashBacksNI0 -
Where does link juice flow on a cloaked link?
Hello, I use a wordpress plug in that allows me to display tot he user any link I want from my domain, so it might be like: www.domain.com/gift-card, but the actual link is www.someaffiliatelink/w09fjai;owfoienw <--- and then a bunch of crap after the domain for the affiliate link. It uses the common technique of an iframe to hide the actual url from the user and show the one that I want them to see. What I am wondering is, does link juice in this case flow to my site, or to their site? And also, do you have any comments regarding this type of link cloaking? Thanks. Thanks
On-Page Optimization | | BigJohnson0