Tool for scanning the content of the canonical tag
-
Hey All,
question for you. What is your favorite tool/method for scanning a website for specific tags? Specifically (as my situation dictates now) for canonical tags?
I am looking for a tool that is flexible, hopefully free, and highly customizable (for instance, you can specify the tag to look for). I like the concept of using google docs with the import xml feature but as you can only use 50 of those commands at a time it is very limiting (http://www.distilled.co.uk/blog/seo/how-to-build-agile-seo-tools-using-google-docs/).
I do have a campaign set up using the tools which is great! but I need something that returns a response faster and can get data from more than 10,000 links. Our cms unfortunately puts out some odd canonical tags depending on how a page is rendered and I am trying to catch them quickly before it gets indexed and causes problems. Eventually I would also like to be able to scan for other specific tags, hence the customizable concern. If we have to write a vb script to get it into excel I suppose we can do that.
Cheers,
Josh
-
No idea on that one - it's still pretty new. The developers actually chimed in on the post, so you could ask them in the comments.
-
Thanks Dr. Pete and Marcus.
I just finished reading the post. I have looked at Screaming Frog before but was hoping to be able to find a way to do it myself. Just didn't want to plop money down on something that seemed like it should be able to be done using tools I already had. But the software does look good. Any thought on if they will come out with a one time purchase instead of a yearly subscription?
Cheers!
Josh
-
Hey Dr. Pete, Joshua
I was just coming here to say that I had read the Dr. Pete post and this may do the job. It's a paid bit of a software but I will be picking it up later. I have my guys knocking up a canonical checker that will be free for all but that may take a day or so to get perfect.
Let me know if you have a play with Screaming Frog!
Marcus
-
I'm pretty sure that Screaming Frog SEO Spider will do it, but you need the paid version to custom-filter on the canonical tag. I've got a post going up about it tomorrow.
-
Great, really appreciate it! Many thumbs up
-
Hey Josh,
Right, cool. I have got a few jobs to sort out but I am going to have a bash at knocking this up this afternoon. Should be easy enough (he said, damning himself to hours of problems).
Leave it with me for 24 hours.
Marcus
-
Hey Marcus,
thanks for the quick response. That is exactly what I would be looking for. I do have a list of url's and that is also simple enough to get from something like xenu. Would love to work with you on this.
Thanks.
Josh
-
Hey, I am not aware of any such tool, but it should not be too hard to put one together, maybe a useful little tool as well.
If you have all of your pages in spreadsheet or database, it should be easy enough to write a little script that cycles through them.
Start Loop
-
request page
-
parse code to get canonical URL
-
compare page to canonical
-
output problem URLs
End Loop
Slightly over simplified and requires a list of all your URLs but would be willing to help put something like this together, could be useful for all of us, especially for those (like me) that work with a lot of CMS sites.
Cheers
Marcus
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
API for On Page tool
I'm looking for a tool similar to On Page Grader (Moz) or Focus Keyword (Yoast) with API. We are building out or internal CRM system. Even though none of these tools can replace manual on page analysis, it will be used as a metric and to catch human mistakes.
Moz Pro | | OscarSE0 -
How are you using Moz Content?
Hey people, I just subscribed for Moz Content and I am wondering how are other professionals using it as a strategy tool. Today I just released a blog post talking about how larger content impacts PA. Not a big deal. I would appreciate some ideas and insights.
Moz Pro | | amirfariabr0 -
Interested in testing a new Moz product for Content Strategists?
We're building a new product for content professionals that helps with website auditing, content performance measurement and content discovery/curation. If you're a content strategist, blogger, editor, copywriter, SEO with a content focus or social media manager that curates content we'd love to hear your thoughts. Interested in a sneak peak of what's cooking? Either respond to this post or send me an email at jay@moz.com. Be sure to include your job function and the type of company your work for. Thanks all!
Moz Pro | | JayLeary5 -
How to handle subdirectories in Moz and Webmaster Tools?
I have 3 websites: http://www.abc.com (English pages) http://www.abc.com/fr (French) http://www.abc.com/de (German) Webmaster Tools In Google Analytics: I created an Account called ABC and created 4 properties with relative filters ABC Default (No filter) ABC English (filter to exclude /fr, /de) ABC French (filter to include /fr) ABC German (filter to include /de) In webmaster tools, I wish to add 3 sites http://www.abc.com http://www.abc.com/fr http://www.abc.com/de How do I validate each url with Google analytics, considering I just want the specific language for folder. How do I validate the domain name http://www.abc.com to always ensure the inclusion of the default english data? MOZ Do I need to create 3 campaigns (one per language) and if yes, again how do I handle http://www.abc.com which should exclude /fr and /de? Thank you
Moz Pro | | seo12120 -
Why does Crawl Diagnostics report this as duplicate content?
Hi guys, we've been addressing a duplicate content problem on our site over the past few weeks. Lately, we've implemented rel canonical tags in various parts of our ecommerce store, over time, and observing the effects by both tracking changes in SEOMoz and Websmater tools. Although our duplicate content errors are definitely decreasing, I can't help but wonder why some URLs are still being flagged with duplicate content by our SEOmoz crawler. Here's an example, taken directly from our Crawl Diagnostics Report: URL with 4 Duplicate Content errors:
Moz Pro | | yacpro13
/safety-lights.html Duplicate content URLs:
/safety-lights.html ?cat=78&price=-100
/safety-lights.html?cat=78&dir=desc&order=position /safety-lights.html?cat=78 /safety-lights.html?manufacturer=514 What I don't understand, is all of the URLS with URL parameters have a rel canonical tag pointing to the 'real' URL
/safety-lights.html So why is SEOMoz crawler still flagging this as duplicate content?0 -
Problem with the keyword difficulty tool
sorry if this has been asked/been written about previously, I had a search but couldn't find anything. I am using the keyword difficulty tool to narrow down on some keywords and whilst the % score for each keyword is being displayed, the Google Adwords data is blank. It just shows the animation as if it was pulling the data but after some ten mins or so not data is pulled, just the animation still running. Is there a problem with the tool or does the problem lay at my end? Thanks, Carl
Moz Pro | | Grumpy_Carl0 -
Duplicate content & canonicals
Hi, Working on a website for a company that works in different european countries. The setup is like this: www.website.eu/nl
Moz Pro | | nvs.nim
www.website.eu/be
www.website.eu/fr
... You see that every country has it's own subdir, but NL & BE share the same language, dutch... The copywriter wrote some unique content for NL and for BE, but it isn't possible to write unique for every product detail page because it's pretty technical stuff that goes into those pages. Now we want to add canonical tags to those identical product pages. Do we point the canonical on the /be products to /nl products or visa versa? Other question regarding SEOmoz: If we add canonical tags to x-pages, do they still appear in the Crawl Errors "duplicate page content", or do we have to do our own math and just do "duplicate page content" minus "Rel canonical" ?0 -
Onpage Optimization Tool - Optimize it? :)
I've been using your on-page optimization tool allot lately, I must say it simply is a great checklist to run a page through and it forces you to think about it a bit more and you notice stuff that you might otherwise have overlooked (read forgot) . But! ('cause there's always a but..) I have noticed a few issues: it could be optimized to recognize plural endings, given this is a hard one since there's allot of languages.. but It would be awesome if it did.. right? (just like the SE's do) I would be more then willing to help with the Danish. Since you already specify what version of google you wanna target eg: ".DK" it might not be all that hard to implement plural? now that we are mentioning other languages.. It would be equally sweet if it would see the correlation between Ø and OE, Å and ÅÅ and so on.. (scandinavian chars for those of you who don't know) again the SE's do For German letters I expect (not sure though, I'm not German, I speak it a bit though, so if your German feel free to correct me 🙂 that the SE's would see the letter Ö as an OE.. Ü as UE, Ä as an AE and so on. Why is the above important? well because that many still use those versions of the letters in URL's. Even though all browsers/mail clients now a'days are able to understand punycode.
Moz Pro | | ReneReinholdt1