Automated checking for broken links within content pieces
-
Hi, I am wondering if anyone can send me in the right direction on a system suggestion.
We have currently grown out amount of content pieces on our website and our manual checking if the links in the content pieces are still 200 status is becoming extremely time consuming. Does anyone have a recommendation of a system that will crawl your pages and check both the internal and external links within the content for a status code (404,200,etc)? Preferably something server side so it can just run on a schedule but really anything would be fine.
I have tried things like Screaming frog, etc and it just doesn't seem to be the right tool.
-
Try ScreamingFrog again Jonathan, it works great for these kind of things and should also be able to solve your use case.
-
Jonathan, I'm not sure why you're saying that Screaming Frog isn't the right tool--we use it with great success to check the internal links on the site. There are other tools that you can use, such as Integrity (on a Mac), or Xenu, which is an older link checker but still works.
-
Have you tried http://www.link-assistant.com/website-auditor/ as it checks for broken links and can be scheduled to run automatically. You can sit it on your own server or something like AWS. We ran it on a free instance of AWS for quite a while before upgrading and never had issues. We upgraded as we run quite a bit of software on there - still isn't huge costs involved.
Hope this helps!
Matt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Creating Tables with Multiple Links
Hello, Is Creating Tables with Multiple Links and using schema markup bad SEO? For example I have a real estate website which I have a hierarchy of home page -> County pages -> city pages-> blog posts.. However at the bottom of the county pages I list the city pages i serve under a table with links..
On-Page Optimization | | sqc0 -
Cross-linking for mobile SEO
Hi everyone! I am having a hard time finding information about weather to/how to apply internal seo linking to mobile versions of sites. We decided to go with dynamic serving with user agent detection. Our desktop site has a quite heavy seo-internal-cross-linking. As I understand, for mobile we should simplify and focus on usability, so get rid of unnecessary links. But I have a doubt about weather removing this part of the web structure can hurt our SEO. Do Google mobile bots look at and rank mobile versions of pages from scratch or do they use what they know about the site and the site's structure from its desktop version?
On-Page Optimization | | ofertia0 -
Is there SEO benefit of automated content through Narrative Science or Automated Insights?
I'm considering working with a group like Narrative Science or Automated Insights to create content for 10k cities around the country. Each article they would create (3-5 per city) would be completely original, based on data we either own or license, written to our editorial tone, voice and direction, and consist of 300-500 words per page. If you are familiar with these groups, you'll know that it is not spun content or spammy crap that we know Google kills off in droves. It will be well written, accurate, articulate original content on topics like health, demographics, population growth, schools and education and weather pertaining to a city or metro area. My question - assuming the answer is actually known - is how well (or if) this content will perform in Google. It is a significant investment for my group (well into size figures) and we don't want to take this decision lightly. We are looking to challenge sites like city-data.org and bestplaces.net, who largely just regurgitate aggregated data.
On-Page Optimization | | barberm0 -
Help With Duplicated Content
Hi Moz Community, I am having some issue's with duplicated content, i recently removed the .html from all of our links and moz has reported it as being duplicated. I have been reading up about Canonicalization and would to verify some details, when using the canonical tag would it be placed in the /mywebpage.html or /mywebpage file? I am having a hard time to sort this out so any help from you SEO experts would be great 🙂 I have also updated my htaccess file with the following Thanks in advance
On-Page Optimization | | finelinewebsolutions0 -
Too Many On-Page Links
Ok so I am very new to MOZ, and we have just set up our PRO account and campaign. When we got our crawl results we had 4 pages with "too many on-page links". All of these pages correspond with our blog, which is a wordpress hosted blog that is integrated into our site. Our site is www.moxicopy.com and the blog is www.moxicopy.com/blog I am confused on how we have over 100 on page links on these pages, as we have very few links on our blog.
On-Page Optimization | | Moxicopy.com0 -
In-page Optimization check list....
Hey, Mozzers I have a question about optimizing a web-page. I was just having a conversation with our web developer about optimizing our website and which changes would make the most difference in affecting the SERPs as far as on-page optimization goes. I was explaining to her we wanted to optimize our pages in the following levels (ranked most important to least important): <title>: key word laden </p> <p>2) <H1> : key word in <strong>bold</strong></p> <p>3) page content: keyword laden as well</p> <p>The idea was to have three or four layers of keywords on the page. I work for a real estate brokerage and the context was the actual page of the property listing. So in this case, the keywords would be the address of the property. She explained to me that the the H1 tag is more important than the title tag.</p> <p>Does this ring true with you guys?</p></title>
On-Page Optimization | | AubbiefromAubenRealty0 -
Duplicate page content errors
Site just crawled and report shows many duplicate pages but doesn't tell me which ones are dups of each other. For you experienced duplicate page experts, do you have a subscription with copyscape and pay $.05 per test? What is the best way to clear these? Thanks in advance
On-Page Optimization | | joemas990 -
Link Product Thumb & Product Name with same anchor link?
We have an issue on one of our sites we're monitoring a campaign for that seems to have TOO many links on each page. I think the biggest reason is that each product listing on each category page has two separate anchor links into that page. One for the thumb and one for the name. So even though there should only be 60-70 links on each category page, that amount is being inflated because each product listing technically is being split into two separate links. Question is, should I place the thumbnail and name within the same anchor link? We do this on a lot of other sites we operate, but I'm not sure what's a better strategy. It would seem to me that it would be better to have a single anchor link that shares the thumb and product name.
On-Page Optimization | | AarcMediaGroup0
