Is there a tool that I can use to scrape and see metatags?
-
Looking for a tool that allows me to scrape the websites off a page listing of Google and output a spreadsheet with the websites and their related meta-tag details (mainly title tag). Is there a tool out there that can conveniently allow me to do this?
-
Thanks for everyone's responses! Matt-Antonino, I'll look to implement that solution. Appreciate it!
-
Thanks for sharing Matt-Antonino.
That's good to know!
-
You can do this fairly easily with ImportXML in Google docs or xpath in Excel.
This is the guide that got me started:
https://www.distilled.net/blog/distilled/guide-to-google-docs-importxml/
If you use Chrome, finding the xpath is fairly simple and then you can just add it to your list.
The other way to do this is use Seer's Top 30 Results Scraper alongside the SEO Tools Excel plugin:
http://www.seerinteractive.com/blog/google-scraper-in-google-docs-update
In Excel tools simply use =htmltitle() and get the info you need.
-
Also keep in mind that search engines will change the title displayed in the SERP based on your query, so if you just get the info presented in the SERP, you may not be seeing the actual title tag.
-
Agreed Donna! It would be nice if Screaming Frog did that but it doesn't. I'm not sure of any existing tools that would do that.
You can run each site from the SERP through Screaming Frog to get all that data exported but that would take a good amount of time!
-
I don't think you can use Screaming Frog for that. It will certainly give you a ton of information about a single website, but it doesn't allow you to scrape Google search results and derive meta data for the top X results. Unless I'm missing something....
-
Screaming Frog is a great tool for this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO Implications of using Images for Article Titles
Hi guys! New to Moz Pro. I just recently completed an online course with Moz... I have a client who is writing some new content for their site, and we are approaching it with SEO in mind. I was wondering about using an image with text on it as the article title, instead of an actual "text on the page" title. Wondering if that's going to "cost" us anything, SEO wise. I guess we could use alt-text/title/description fields to make sure the keywords are crawlable for our article title but do they have less "weight" than a standard title? How does that work? Hope my question makes sense. Article header attached mB0PXsA.jpg
On-Page Optimization | | JakeWarren1 -
New to MOZ and so far love tools but need some quick tips on title, keywords and descriptions
First of all:
On-Page Optimization | | nickcargill
I am excited about using all the MOZ tools! I just got back my webcrawl and have found
a lot of issues that I am working on. I am a Vacation Rental Property Manager with 150 properties, all different. 90% of my pages have the same keywords and descriptions and a lot of same page titles
too. I can change all of these by adding a few fields in my database and then
populating the meta information dynamically. For Example: <title>Big Bear Cabins | Big Bear Cabin Rentals - Chateau Alta Vista Cabin</title>
Chateau Alta Vista, Big Bear Cabins, Big Bear Cabin Rentals
Chateau ALta vista is a Big Bear Cabin with four bedrooms, three
bathrooms, it is Big Bear Luxury Property. If you notice the Title, keyword are dynamically populated from the database using the
property name of Chateau Alta Vista. The description is an extra field in the database that I just implemented that I can customize per property. I have a few questions, but here is more information: I also have 5 or 6 related pages
to each property. Pages Like general information, photos, calendars, book
Question 1): Is the best way of doing my titles, keywords and descriptions. Any concerns or recommendations. I have read, do not even use keywords anymore. Question 2) I have other pages that show maybe 8 or 10 photos. So is it ok to do the following; Title <photos of="" big="" bear="" cabin="" chateu="" alta="" vista<="" title="">Meta Keywords <photos of="" big="" bear="" cabins="" chateu="" alta="" vista="">Meta Description <check out="" photos="" of="" big="" bear="" cabin="" rentals,="" chateua="" <br="">Alta Vista, etc Currently, I have some archaic page naming with database driven url parameters such as</check></photos></photos> http://destinationbigbear.com/property_Detail_v.aspx?propid=669 and at worse http://destinationbigbear.com/Property_detail_v.aspx?propid=669&rate= $1,127.10&checkoutdate=07/13/2014&beds=4&firstnight=07/11/2014&nights=2 I do not have he ability to to full url encoding such as http://destinationbigbear.com/cabins/chateau-alta-vista but I can do http:/www.destinationbigbear.com/big-bear-cabin-info.aspx?cabin=chateau%alta%vista Question is what do I do... If I do change the page names I will lose the history of property_detail_V.aspx which only has a Page Authority of 21, but if I change the page name and dynamic navigation of my website will I not lose all my authorities, if so, is it worth doing it? My highest keyword ranking is 30 which is terrible. Nick0 -
Can't get authorship to show in the rich snippets tool...
Can't get authorship to show in the rich snippets tool... am I doing something wrong? spokanepaintingcontractor.org
On-Page Optimization | | Superflys0 -
Can't rank for a target key word "penalized?"
I've been trying to rank for the key word "kayak fishing" for my site www.yakangler.com. Last year when I started working on the SEO for my site I was on page 30 for Google search results so like the 300th result. After tweaking things on my site I managed to get to the second page but have since fallen all the way back to page 25-26 in the search results. I'm wondering if I'm penalized for this key word. I can't figure out why my site is ranking soooo badly for "kayak fishing" Any help or ideas would be greatly appreciated!
On-Page Optimization | | mr_w0 -
Can Page Authority of a site be higher than its Domain Authority?
I own a website called Takeyourtips.com. While doing a search no Google, I found that the page authority of the home page (31) is higher than the domain authority (23). I was wondering if it's really possible because my understanding was page authority of a page is determined by its domain authority. Therefore, it the domain authority of a website is 23, none of its page could have a higher page authority. Plus, upon consulting an SEO expert, I was told that neither Domain Authority or Page Authority of a page carries any importance as far as higher ranking of a website is concerned. Is this true? Thanks in advance for the answers. Cheers, Sushant large
On-Page Optimization | | suskanchan0 -
Should I use this Facebook comment content on my related blog post?
I have a blog post that ranks pretty high for the term "justin bieber tickets". We are running a ticket giveaway and have received tons of responses on Facebook and G+. The responses are often poorly written in they sense that they are from younger fans, but it is a bunch of related content that I thought could be a "good "addition of unique content to the post. Is this a good idea in general? Is it still a good idea if the comments are poorly written and contain lots of slang an exclamation points? Is it bad form to put people's Facebook comments live on the web, even though it is a public page. Here is the post Example of what this would look like in the post >http://cl.ly/1Q3N0t091V0w3m2r442G Source of comments >http://www.facebook.com/SeatGeek Another less aggressive option would be to curate some of my favorite comments... Thanks for any thoughts.
On-Page Optimization | | chadburgess0 -
Which method should I use for my URL structure?
I have an existing site that is currently utilizing a structure that is like this: http://www.mysite.com/Ohio/City-of-Cleveland-PRODUCT-NAME Should I restructure it like: http://www.mysite.com/Ohio/City-of-Cleveland/Product-Name We are doing very well with very specific searches already but are sometimes coming in 2nd and 3rd place. For example: If I search for CLEVELAND PRODUCT NAME I always come up in the top three and about 60% of the time I am #1. I want to make it better. We have only launched in 4 states but plan on launching an additional 4 states over the next few weeks and I want to make sure we are building things properly. Any feedback would be wonderful. As usual, thanks everyone!! -Alex
On-Page Optimization | | dbuckles0 -
Page speed tools
Working on reducing page load time, since that is one of the ranking factors that Google uses. I've been using Page Speed FireFox plugin (requires FireBug), which is free. Pretty happy with it but wondering if others have pointers to good tools for this task. Thanks...
On-Page Optimization | | scanlin0