Automated checking for broken links within content pieces
-
Hi, I am wondering if anyone can send me in the right direction on a system suggestion.
We have currently grown out amount of content pieces on our website and our manual checking if the links in the content pieces are still 200 status is becoming extremely time consuming. Does anyone have a recommendation of a system that will crawl your pages and check both the internal and external links within the content for a status code (404,200,etc)? Preferably something server side so it can just run on a schedule but really anything would be fine.
I have tried things like Screaming frog, etc and it just doesn't seem to be the right tool.
-
Try ScreamingFrog again Jonathan, it works great for these kind of things and should also be able to solve your use case.
-
Jonathan, I'm not sure why you're saying that Screaming Frog isn't the right tool--we use it with great success to check the internal links on the site. There are other tools that you can use, such as Integrity (on a Mac), or Xenu, which is an older link checker but still works.
-
Have you tried http://www.link-assistant.com/website-auditor/ as it checks for broken links and can be scheduled to run automatically. You can sit it on your own server or something like AWS. We ran it on a free instance of AWS for quite a while before upgrading and never had issues. We upgraded as we run quite a bit of software on there - still isn't huge costs involved.
Hope this helps!
Matt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does javascript generated content consider as regular content?
The website mentioned below, the content is generated using javascript, and content is something to do with Unicode char. The Unicode content creates as you scroll down. Will this content affect SEO https://www.myweirdtext.com/
On-Page Optimization | | teenmass423230 -
Dynamically populated content
We are developing a website for a school that has 19 campuses divided into 8 districts. Ideally, we would like to have one search page that dynamically populates when people search WHILE on the site. The question is what happens when someone does an organic search, will the search engine populate with the schools in that district. For instance, if i search on Google "Austin Schools", will the Austin district-that does not have a unique URL- show up in a Google search? What the generated page looks like is on this link http://imgur.com/stCQcP6. If yes, any special type of coding we need to add to the backend?
On-Page Optimization | | jgodwin0 -
Duplicate content: Form labels and field content
I have a site that has 500 pages, each with unique content, the only content that could be deemed the same is the 'Make Contact' form, which has the same labels and placeholder text on each page. Is this likely to cause any duplicate content penalties?
On-Page Optimization | | deployseo0 -
Link to social network
I have very powerfull social network but i do not link to them from my website to improve page speed and avoid external links on main page. Althought i do link from my twitter acounts , facebook and google to my site. Should i link to my twitter or social on main page. I do not want to send people to there. I want to send people from there to my page.
On-Page Optimization | | maestrosonrisas0 -
23000 Links are not found- Should I redirect them?
Hi I have been deleting product links from my website but never redirect them. On my google webmaster, it shows there is total 23000 products are not found. Should I redirect them all back to the home page? For the pages with soft 404 response.. should I also redirect those original URL back to home page ? Thanks
On-Page Optimization | | ilovebodykits0 -
Help: my WordPress Blog generates too many onpage links and duplicate content
I have a WordPress Blog since November last year (so I'm pretty new to WordPress) and the effects on ranking for some keywords are really good. So I thought tag clouds are good. Crawl Diagnostics tell me now that I have too many onpage links for example my author page breaks the record: 256
On-Page Optimization | | inlinear
http://inlinear.com/blog/author/inlinear/ I think thats because there are links for each word in the tag cloud generated ... On this page (and many other pages) WordPress displays (teasers) the beginning of each post (read more ...) producing duplicate content and even new canonical tags.... The page titles are also too long because I installed "All in One SEO Pack" and now this plugin and wordpress itself mixes titles together ... But what can I do to avoid all this. Is there a PlugIn that can help... I think millions of blogs will have the same problems... I my blog yet has very few content. Thanks for your answers :))0 -
Do NoFollow links still split link equity?
So I realize that Google will split link equity between all links on any given page. Example, if a landing page has 10 links then the authority from the landing page is split into 10 and each link given its own smaller amount of equity from that landing page. My question is if I were to turn 9 of the 10 links on this page to NoFollow links would the equity still remain split 10 ways or would it simply pass all of it to the one DoFollow link left on the page?
On-Page Optimization | | PageOnePowerGang0 -
Linking Back to the Same Page
What are the other seo's opinions on linking the same keyword you are targeting lets take an example like Trampolines. So we have a Online shop selling trampolines would you feel it a good or bad thing to link the keyword trampolines from the homepage to the homepage almost creating a loop. Some SEO's say yes some say no ?
On-Page Optimization | | onlinemediadirect0