Is there a tool that I can use to scrape and see metatags?
-
Looking for a tool that allows me to scrape the websites off a page listing of Google and output a spreadsheet with the websites and their related meta-tag details (mainly title tag). Is there a tool out there that can conveniently allow me to do this?
-
Thanks for everyone's responses! Matt-Antonino, I'll look to implement that solution. Appreciate it!
-
Thanks for sharing Matt-Antonino.
That's good to know!
-
You can do this fairly easily with ImportXML in Google docs or xpath in Excel.
This is the guide that got me started:
https://www.distilled.net/blog/distilled/guide-to-google-docs-importxml/
If you use Chrome, finding the xpath is fairly simple and then you can just add it to your list.
The other way to do this is use Seer's Top 30 Results Scraper alongside the SEO Tools Excel plugin:
http://www.seerinteractive.com/blog/google-scraper-in-google-docs-update
In Excel tools simply use =htmltitle() and get the info you need.
-
Also keep in mind that search engines will change the title displayed in the SERP based on your query, so if you just get the info presented in the SERP, you may not be seeing the actual title tag.
-
Agreed Donna! It would be nice if Screaming Frog did that but it doesn't. I'm not sure of any existing tools that would do that.
You can run each site from the SERP through Screaming Frog to get all that data exported but that would take a good amount of time!
-
I don't think you can use Screaming Frog for that. It will certainly give you a ton of information about a single website, but it doesn't allow you to scrape Google search results and derive meta data for the top X results. Unless I'm missing something....
-
Screaming Frog is a great tool for this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should you aim for Google to use your meta tags?
When updating meta titles and descriptions, I'm taking note of whether Google is displaying the set tag or changing it to copy from the page. Does this affect the ranking position if Google is having to change the tag? How much should I worry if Google is choosing to change every other page? Thanks!
On-Page Optimization | | Omar_aw0 -
Can bots crawl this homepage's content?
The website is https://ashleydouglas.com.au/ I tried using http://www.seo-browser.com/ to see if bots could see the content on the site, but the tool was unable to retrieve the page. I used mobile-friendly test and it just rendered some menu links - no content and images. I also used Fetch and Render on Search Console. The result for 'how google sees the page' and 'how a visitor sees the page' are the same and only showing the main header image. Anything below isn't shown. Does this mean that bots can't actually read all content on the page past the header image? I'm not well versed with what's going on with the code. Why are the elements below the header not rendering? Is it the theme? Plugins? Thank you.
On-Page Optimization | | nhhernandez0 -
What Are other people using in replacement for sliders?
Hello, Moz Community! I am currently trying to replace a slider on our client's site. Sliders, in my opinion, are awful, they slow load times and just don't convey a solid message. I am using Wordpress and the visual composer plugin. Any ideas are really appreciated even they may seem a bit much, if I don't know how to do it I will figure it out. I apologize as well if this isn't the appropriate place for this type of question.
On-Page Optimization | | Striventa0 -
Google ranking is HORRIBLE. Following SEOMoz suggestions and just can't climb.
First of all, the URL is stores.dhsequipment.com. In January, this online store switched from Homestead to Big Commerce. Since the store updated, we decided now is the time to update our product descriptions, URL's, title tags and meta descriptions. (For the first time, we had the ability to customize our URL's.) Product Description: I went through 2,500 products and updated the product description. I added an H1 & H2 to each description, and included pertinent information such as part numbers. Each product also received a new page title, meta description (which is usually the first line of the product description, don't know if this is bad or not) and a new URL, (which did redirect). Once I would complete a section, I would submit a new sitemap to Webmaster Tools. After a month and nothing happening, I started using SEOMoz which helped me rebuild some of my more important pages, such as the home page and main category pages such as: http://stores.dhsequipmentparts.com/stihl-ts420-parts/
On-Page Optimization | | pearldesign
http://stores.dhsequipmentparts.com/stihl-ts700-parts-stihl-ts800-parts/ I fetched these pages in Webmaster Tools after completion. However, it's been several weeks since and I'm still on page 4 or 5 in the SERPs. Just a little history on the store; this store has been in operation for more than 6 years. Previously, we ranked on page one for 75%+ of our products. My belief is because our URL's had history, probably more so than our competitors. I'm not sure what I should do. Business is super slow and we can't afford to wait much longer.0 -
Same titles in Webmasters' tools
Hello, In webmasters' tools I get a message saying I have 505 pages html improvements that are possible because 505 of my titles are used on two pages. Actually, webmasters' tools is tripping since the doubles they find are: www.example.com and www.example.com/ Anyways, I have indexation problems and I was wondering if this could be the case and how to solve that. Thanks for your help;
On-Page Optimization | | EndeR-0 -
Webmaster Tools
Webmaster tools is throwing up an error for this page http://www.itwasntmyfault.com.au/testimonial/sharon-m/ HTTP Error: 404 - When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to an HTTP status error. All accessible URLs will still be submitted.The page is fine.... any advice?Thanks
On-Page Optimization | | RGOnline0 -
Best Way to Use Date in Title
Hi, I do most of the current copy for our blog which you can find here http://appointedd.com/blog/ I believe having a regular blog structure with a mix of irregular ad hoc posts to go in around these. So, for this blog, I write an article on "Beauty Industry News" every week. Now, I don't want to use the same title for each post, so I've peen butting in the date after each one i.e. "Beauty Industry News - 24/04/13". Is this best practice or is there a better way of naming regular posts? Thanks in advance!
On-Page Optimization | | LeahHutcheon0 -
On page report card instances of key terms in the body - I can't find them all
Hi guys, I'm using the on-page report card for a particular page which is returning 22 instances of my term, however when I check the source of the page, I can only find 6 instances across the whole page, let alone the body. The site is www.sportsbet.com.au, and the term "horse racing". I'm sure I'm missing something, would appreciate any explanation for this apparent discrepancy. Cheers, Jez
On-Page Optimization | | jez0000