Tool for scanning the content of the canonical tag
-
Hey All,
question for you. What is your favorite tool/method for scanning a website for specific tags? Specifically (as my situation dictates now) for canonical tags?
I am looking for a tool that is flexible, hopefully free, and highly customizable (for instance, you can specify the tag to look for). I like the concept of using google docs with the import xml feature but as you can only use 50 of those commands at a time it is very limiting (http://www.distilled.co.uk/blog/seo/how-to-build-agile-seo-tools-using-google-docs/).
I do have a campaign set up using the tools which is great! but I need something that returns a response faster and can get data from more than 10,000 links. Our cms unfortunately puts out some odd canonical tags depending on how a page is rendered and I am trying to catch them quickly before it gets indexed and causes problems. Eventually I would also like to be able to scan for other specific tags, hence the customizable concern. If we have to write a vb script to get it into excel I suppose we can do that.
Cheers,
Josh
-
No idea on that one - it's still pretty new. The developers actually chimed in on the post, so you could ask them in the comments.
-
Thanks Dr. Pete and Marcus.
I just finished reading the post. I have looked at Screaming Frog before but was hoping to be able to find a way to do it myself. Just didn't want to plop money down on something that seemed like it should be able to be done using tools I already had. But the software does look good. Any thought on if they will come out with a one time purchase instead of a yearly subscription?
Cheers!
Josh
-
Hey Dr. Pete, Joshua
I was just coming here to say that I had read the Dr. Pete post and this may do the job. It's a paid bit of a software but I will be picking it up later. I have my guys knocking up a canonical checker that will be free for all but that may take a day or so to get perfect.
Let me know if you have a play with Screaming Frog!
Marcus
-
I'm pretty sure that Screaming Frog SEO Spider will do it, but you need the paid version to custom-filter on the canonical tag. I've got a post going up about it tomorrow.
-
Great, really appreciate it! Many thumbs up
-
Hey Josh,
Right, cool. I have got a few jobs to sort out but I am going to have a bash at knocking this up this afternoon. Should be easy enough (he said, damning himself to hours of problems).
Leave it with me for 24 hours.
Marcus
-
Hey Marcus,
thanks for the quick response. That is exactly what I would be looking for. I do have a list of url's and that is also simple enough to get from something like xenu. Would love to work with you on this.
Thanks.
Josh
-
Hey, I am not aware of any such tool, but it should not be too hard to put one together, maybe a useful little tool as well.
If you have all of your pages in spreadsheet or database, it should be easy enough to write a little script that cycles through them.
Start Loop
-
request page
-
parse code to get canonical URL
-
compare page to canonical
-
output problem URLs
End Loop
Slightly over simplified and requires a list of all your URLs but would be willing to help put something like this together, could be useful for all of us, especially for those (like me) that work with a lot of CMS sites.
Cheers
Marcus
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content for Default Document Domains
I've noticed recently that within the Moz Crawl Report I keep seeing duplicate content for one of our pages that pulls from a default document. The pages are product pages, one ending in releases/ and the other ending in releases/index and are both identical pages. Normally in these situations I would prefer to make sure that every link is being sent to the releases/ page, however according to Moz, the releases/index page is actually ranking better and has a higher internal link count. Can someone advise me on the best way to deal with this situation? Hopefully I've explained myself well enough! Thanks Sam
Moz Pro | | BlueLinkERP0 -
Problems with duplicate contents...
Hi folks, how's going? I started using Seo Moz and from the first crawling appears that I have 11 pages with duplicate contents... but this is not true, are different pages with different url, contents, tags... any idea to solve he problem? Alessandro, MusicaNueva.es
Moz Pro | | musicanueva0 -
Crawl test from tools
Hi, I notice that the crawl test which is from the Research Tools doesn't really get a new crawl even though there is 2 crawl per day. It will only provide the data which was acquire from the crawl diagnostics in my pro account. There is no point for me to get the data which I get from my crawl diagnostic isn't it? Even seomoz provided with more than 2 crawl per day also useless in this case. This whole thing doesn't make sense as the crawl diagnostics will only perform a full crawl test once every week. but even the crawl test also not helping any thing out for me.
Moz Pro | | hanzoz0 -
Keyword ranking tool
Hello, What is best practice for knowing if a keyword is too difficult to try to rank for. i understand the keyword difficulty tool in seoMOZ, but am unsure of how it actually relates to if I should attempt to rank for the keyword. Is there a differential people use such as if a site has a Page Authority of 60 we will not try to rank for a keyword that has a keyword difficulty ranking of 40?
Moz Pro | | digitalops0 -
Crawl Diagnostics and missing meta tags on noindex blog pages
Hi Guys/Gals We do love the Crawl Diagnostics, but do find the missing meta tags ("Missing Meta Description" Tag in this case) somewhat spammy. We use the "All in One SEO Pack" for our blog and it does stick in noindex,follow (as it should) on the pages that is of no use to us. "2008/04/page/2/" and the likes. Maybe I'm wrong but should the Diagnostics tool not respect the noindex tag and just ignore any warnings, since it should really mean that these pages are NOT included in the search index. Meaning that the other meta tags are really useless. Any thoughts?
Moz Pro | | sfseo0 -
Convince me to stay! How should I best use SEOMoz tools.
Hey folks. I'm here on a free trial and I'm really loving the Q&A forum. I've learned a lot in the last couple of days. I've been playing around with the SEOMoz tools and campaigns and I'm trying to decide if it will be worth it for me to pay the monthly fee to continue with these once my trial is up. I direct two main websites. One is an informational site and the other is a real estate site. I've started a campaign for each and so far we're still waiting for a crawl. (I'm assuming that will happen on Sunday). I've played around with some of the tools. They keyword difficulty tool is cool but I can't see me using it a whole lot. The rest is just a little overwhelming...I'm not sure where to start. So what should I be checking out next? Have at 'er and convince me to stay!
Moz Pro | | MarieHaynes3 -
Any tools for scraping blogroll URLs from sites?
This question is entirely in the whitehat realm... Let's say you've encountered a great blog - with a strong blogroll of 40 sites. The 40-site blogroll is interesting to you for any number of reasons, from link building targets to simply subscribing in your feedreader. Right now, it's tedious to extract the URLs from the site. There are some "save all links" tools, but they are also messy. Are there any good tools that will a) allow you to grab the blogroll (only) of any site into a list of URLs (yeah, ok, it might not be perfect since some sites call it "sites I like" etc.) b) same, but export as OPML so you can subscribe. Thanks! Scott
Moz Pro | | scottclark0 -
A tool for seeing all the keywords a website ranks for
As title, is there a tool out there that will allow you to see all the keywords a website ranks for in the first few pages of a particlur search engine? I would be particularly interested for keywords in the first 10 pages of Google. Thanks
Moz Pro | | blagger0