Is there a tool that I can use to scrape and see metatags?
-
Looking for a tool that allows me to scrape the websites off a page listing of Google and output a spreadsheet with the websites and their related meta-tag details (mainly title tag). Is there a tool out there that can conveniently allow me to do this?
-
Thanks for everyone's responses! Matt-Antonino, I'll look to implement that solution. Appreciate it!
-
Thanks for sharing Matt-Antonino.
That's good to know!
-
You can do this fairly easily with ImportXML in Google docs or xpath in Excel.
This is the guide that got me started:
https://www.distilled.net/blog/distilled/guide-to-google-docs-importxml/
If you use Chrome, finding the xpath is fairly simple and then you can just add it to your list.
The other way to do this is use Seer's Top 30 Results Scraper alongside the SEO Tools Excel plugin:
http://www.seerinteractive.com/blog/google-scraper-in-google-docs-update
In Excel tools simply use =htmltitle() and get the info you need.
-
Also keep in mind that search engines will change the title displayed in the SERP based on your query, so if you just get the info presented in the SERP, you may not be seeing the actual title tag.
-
Agreed Donna! It would be nice if Screaming Frog did that but it doesn't. I'm not sure of any existing tools that would do that.
You can run each site from the SERP through Screaming Frog to get all that data exported but that would take a good amount of time!
-
I don't think you can use Screaming Frog for that. It will certainly give you a ton of information about a single website, but it doesn't allow you to scrape Google search results and derive meta data for the top X results. Unless I'm missing something....
-
Screaming Frog is a great tool for this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can I use nofollow to limit the number of links on a page?
My website is an ecommerce and we have on homepage about 470 links ! 1. We have a top bar with my account, login, faq, home, contact us and link to a content page. 2 . Then we have multistore selection 3. Then we have the departament menu, with several parants + child category links 4. Then we have a banner 5. Then we have a list of the recently sold and new products. 6. then we have an image grid with the most important cms/content pages (like faq, about us, etc) 7. then we have footer, with all info pages, contact us, about us, my account etc. There are some links that are repeted 2, 3 times. For a user it is easier to find the informations but I'm not sure how search bots (google) deal with that. So I was thinking on how can I have around 150 links to be followed. To remove the links from the page is not possible. What about to add nofollow to repeted links and some child category, as the spider will crawl the father and will access child on the next page? Is this a good strategy?
On-Page Optimization | | qgairsoft0 -
Can I have schema.org links as relative on my site? Getting an html validation error.
I'm getting an html validation error on relative schema.org links "Bad value //schema.org/Organization for attribute itemtype on element div: The string //schema.org/Organization is not an absolute URL." This is my code for https site: <code class="input">e itemtype="//schema.org/Organization"><a itemprop="url" class="navbar-brand" …<="" code=""></a></code>
On-Page Optimization | | RoxBrock0 -
Can you use multiple keywords for on page for ranking?
I understand using a keyword (or phrase) and correctly building that into the site structure (URL, Title Tag, body, etc). So, this question is going to be elementary, but I am starting to question myself as I write content. I have a client, for example, that has a new site and a page for Chocolate cakes. Now the other pages they built out are for Cheesecakes, Cupcakes, etc. So we optimized the Chocolate cakes page with our keyword throughout (Getting an A+ on page content grade). But now they are asking me why they can't be found for chocolate eclairs, chocolate fudge cake, devils chocolate cake, double chocolate cake, etc. My first quick answer is that they should build more pages. But am I doing this wrong?
On-Page Optimization | | cschwartzel0 -
Can you find the "problem" metric or metrics?
I just dropped from #2 for my main keyword to #5 and am not sure why. My companies Ranking Metrics compared to top 5 Page Authority:#2, MozRank:#1, MozTrust:#1, MT/MR:#1, Total Links: #1, Internal Links:#1, External Links: #1, Followed Links:#1, No Follow Links: #1, Linking Root Domains:#1, OnPageAnalysis Grade: "A", Broad Keyword usage: Yes, Broad keyword in document:
On-Page Optimization | | Boodreaux0 -
Plagiarism or duplicate checker tool?
Do you know a plagiarism or duplicate checker tool where I can receive an email alert if someone copies my content? I know there's a tool like this (similar to http://www.tynt.com/ though people can still remove the link from the original source) but I forgot the name or site. It's like a source code that you must insert in each of your webpage. Thanks in advanced!
On-Page Optimization | | esiow20131 -
How do I get rid of duplicate page titles when using a php site?
Hi. I have an e-commerce site that sells a list of products. The list is divided into categories and then those categories for the various pages on the site. An example of a page title. would be given root/products.php?c=40 another page would be given root/products.php?c=41 Is there a way to structure the site with SEO in mind?
On-Page Optimization | | curtisgibbsiii0 -
If I have too many on-page links can I reduce it with nofollow tags or do the links have to be removed?
On my site I have a top nav drop down menu but once visitors go to one particularly large subsection, that menu is repeated on the left for easier viewing. As a result, I shoot over 100 links on page. Can I put nofollow or noindex tags on the left side links and reduce my "official" on-page links count or do I have to actually eliminate some of the links? Thanks, Oak
On-Page Optimization | | CSA-2316710 -
Re-optimizing onsite SEO, can it hurt?
We finished the re-design of our website a few months ago. We have hired a few freelance SEO guys that were horrible. We then decided to pull the SEO work in-house. I got nominated to do the SEO work. I started with what I thought was pretty good on-site SEO. At that time, with no experience, I was pretty proud of myself. I managed to get a bunch of our pages at top SERPs for long-tail keywords. Good enough for then. Now when I go look at the pages, I'm embarrassed to admit that it's my work. Please be kind. 🙂 Since then I have been trying to learn as much as possible about SEO. I'm certainly far ahead of where I was a few months ago. For the past few weeks I've been trying to focus my efforts on creating original keyword rich original content. Our competitors all have tried this, but their content is hardly readable by humans. Anyhow, we finished our fist article, it got indexed by G almost immediately and started to push our keyword SERPs up within just a few days. Now for my question. I have a much better understanding of on-page SEO and realize that I could make many improvements to ALL of our other pages. However, these pages are already doing fairly well with the SERPS and are moving up a few ranks a week. I'm very tempted to throw caution to the wind and completely redo all of our on-page SEO for our entire site. Is this a good strategy?
On-Page Optimization | | dmac
Should I expect our SERPs to drop a little, a lot, or not at all? I look forward to your response. DMac0