Automated checking for broken links within content pieces
-
Hi, I am wondering if anyone can send me in the right direction on a system suggestion.
We have currently grown out amount of content pieces on our website and our manual checking if the links in the content pieces are still 200 status is becoming extremely time consuming. Does anyone have a recommendation of a system that will crawl your pages and check both the internal and external links within the content for a status code (404,200,etc)? Preferably something server side so it can just run on a schedule but really anything would be fine.
I have tried things like Screaming frog, etc and it just doesn't seem to be the right tool.
-
Try ScreamingFrog again Jonathan, it works great for these kind of things and should also be able to solve your use case.
-
Jonathan, I'm not sure why you're saying that Screaming Frog isn't the right tool--we use it with great success to check the internal links on the site. There are other tools that you can use, such as Integrity (on a Mac), or Xenu, which is an older link checker but still works.
-
Have you tried http://www.link-assistant.com/website-auditor/ as it checks for broken links and can be scheduled to run automatically. You can sit it on your own server or something like AWS. We ran it on a free instance of AWS for quite a while before upgrading and never had issues. We upgraded as we run quite a bit of software on there - still isn't huge costs involved.
Hope this helps!
Matt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content query
I'm currently reauthoring all of the product pages on our site. Within the redesign of all the pages is a set of "why choose us?" bullet points and "what our customers say" bullet points. On every page these bullet points are the same. We currently have 18% duplicate content sitewide and I'm reluctant to push this. The products are similar but targeted at different professions, so I'm not sure whether to alter the text slightly for the bullet points on each page, remove the bullet points entirely or implement some form of canonicalisation that won't impact the profession-specific pages' ability to rank well.
On-Page Optimization | | EdLongley0 -
On hover my links are with additional Parameters while links that are indexed are without additional parameters
On hover my links are with additional Parameters while links that are indexed are without additional parameters does it impact in a negative way. For ex: i have a site http://www.yoursite.com and Its internal pages that are linked to the site are in pattern of http://www.yoursite.com/jobs-in-india?xz=3_0_5 and these are the pages which are interlinked through out the site. When any user click the link they will land to the similar pages with additional parameter even on mouse hover any one can see the same link. while we have used Canonical, so pages that are getting indexed are http://www.yoursite.com/jobs-in-india. But my concern is: - To showing two different link as when Google crawler follow the site they will get the links with additional parameter while in its index its a URL without additional parameter so is there problem that we can encounter or is there any negative impact on ranking?
On-Page Optimization | | vivekrathore0 -
Internal Linking From Blog to Website
Hi all, I'm just seeking opinions on something an external company have told us about linking from our blog to our website... Our website is; www.XYZ.com and our blog is www.XYZ.com/blog I add content to the blog on an almost daily basis and generally link on average 3 times from the blog (internally) to a various relevant pages on our website. Today I was told that by doing this I am 'diluting' the link juice which I understand but don't agree with... All I am doing is a form of internal linking which as far as I am aware is a good on-page technique? Just curious to learn other people's view on this... Many thanks Andy
On-Page Optimization | | TomKing0 -
Locating broken links on site?
Hey guys, I'm using Screaming Frog to help locate some broken links on a client's site and I've managed to pick up two. However, I can't seem to find whereabouts they're located on the site in order to fix them! Is there a way I can do this? Cheers!
On-Page Optimization | | Webrevolve0 -
In-page Optimization check list....
Hey, Mozzers I have a question about optimizing a web-page. I was just having a conversation with our web developer about optimizing our website and which changes would make the most difference in affecting the SERPs as far as on-page optimization goes. I was explaining to her we wanted to optimize our pages in the following levels (ranked most important to least important): <title>: key word laden </p> <p>2) <H1> : key word in <strong>bold</strong></p> <p>3) page content: keyword laden as well</p> <p>The idea was to have three or four layers of keywords on the page. I work for a real estate brokerage and the context was the actual page of the property listing. So in this case, the keywords would be the address of the property. She explained to me that the the H1 tag is more important than the title tag.</p> <p>Does this ring true with you guys?</p></title>
On-Page Optimization | | AubbiefromAubenRealty0 -
Duplicat contents on wordpress
I ran a crawl error and found that I have many pages with "tag" i.e. http://www.soobumimphotography.com/tag/70-200-2-8-is/ What's the best way to deal with this problems? Is it worth to visit all of them and fix? Delete? Could you give me some suggestions?
On-Page Optimization | | BistosAmerica0 -
Silo and content
I'm about to launch my site but I have a question regarding content and silo structure. If I don't have enough content to fill 4 subpages, could it be better to have only a content-keyword-rich landing page for a silo instead of multiple pages with poor content? Thank you!
On-Page Optimization | | mediodigital0 -
Too many links
Hello, sorry question 2! I run the report it says we have 120 links roughly per page. I know this is bad however I have question about an oscommerce site. I identified some links that can be deleted for example Product picture is a link then the title and then its says details (so I can delete the details and that should save about 15 links) Now I really can only delete the breadcrumb for example car>>ford>>spareparts>>engine If I delete the breadcrumb I would lose four links would you say thats a good idea? or leave the breadcrumb due to the seo benefits. Many thanks for your help. Sonja
On-Page Optimization | | reallyitsme0