Text Scraping Tool
-
Can anyone recommend an easy to use website or tool (ideally free) to scrape all the visible text from a website to use for SEO/SEM keyword research?
Thanks as always MOZ friends!
-
Thanks for the info - and of course always for good and not evil with these types of tools. What I'm hoping to do is just enter a URL, have it scrape the site, and then dump all of the text on the site into a document to put into a keyword density tool to gain perspective on new partner websites for industries where we are entering for the first time. All for SEO, content ideas, learning about value propositions and services as well as for keyword ideas for SEM.
Looks like doing some programming might be the best way to go here - was hoping someone may have already built this kind of functionality!
Thanks again!
-
If you know a bit of programming you can go to GitHub and find some basic python code to build a custom web scrapper and format it to grab html. Or Screaming frog and Scrapebox might do the trick. But use with care and don't abuse them.
Hope that helps some.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Impressions data: Google Webmaster Tools VS Google AdWords
Dear community, I was doing some research within Google Webmaster Tools (WMT) keyword data when I noticed that the impressions within this tool are quite different compared to the keyword impression data provided by Google AdWords (ADW). Some specs about the situation: Date: December 2014 (31 days). Keyword: brand + main keyword* (e.g. amazon shoes, if Amazon would have been the client) Visitors come from: the Netherlands (>97%). Search volume: 3,8 K for this branded keyword last year (December 2013), of which >99% came from the Netherlands (source: Google Keyword Planner). Search query data: "impressions" for Google WMT (organic keyword impressions) and “impressions” for Google ADW (paid keyword impressions). Of course the ADW keyword is [exact]. So, what's wrong? The data doesn’t match, while I expected approximately the same amount of impressions assuming the keywords are: both [exact] keywords; (check! ✓) within the same period of time; (check! ✓) the Ad is being displayed "all the time" (check! ✓) the domain/page is being indexed "all the time" (check! ✓) How much of a difference is there? Organic impressions: 7,4 K (source: Google WMT) Paid impressions: 2,1 K (source: Google ADW) Note that the search volume according to Google Keyword Planner of last year is somewhere in between: 3,8K. The search volume from this tool of last December is not available just yet (but I don’t expect much difference here since Google Trends shows a steady search volume). If the difference would have been 10-20%, I wouldn't be surprised at all, but this is huge. **What could explain the differences? ** If a lot of people were using AdBlockers (they do, but not nearly as massively (around 10%)). If we would have made mistakes in AdWords: budget, bidding, targeting etc. - This is not the case, got this confirmed by the manager who double-checked the data and settings. Also: since it’s a branded keyword it’s really cheap for us and easy to get high quality scores. If we would have made mistakes regarding indexing/crawling that would have caused an extreme loss in domain visibility in the SERP's: possibly caused by robots.txt, a noindex-tag, server problems etc. This was not the case and Google WebmasterTools says the average position was 1,0 during the complete month. "Something else" went wrong during that specific period of time with this specific domain. I don’t think so because I checked multiple months and multiple other domains of other clients. These gave me the same relative results (okay, some were a bit closer: 30 K paid impressions vs 62 K organic impressions for instance, still a big difference). What other possibilities are left? The impressions from Google WMT and Google ADW are not the same, even though they are called the same and therefore suggest they should be (about) the same. AdWords just randomly fails to display and/or measure some branded ads (even though there is plenty of budget, bidding is fine and QS is 10/10). Definitions of "impressions" for both tools according Google: AdWords definition of "impressions" WebmasterTools definition of "impressions" Hope someone has some more suggestions or useful links! Thanks in advance! Ektor Tsolodimos
Paid Search Marketing | | BlueMango0 -
Will Google penalize a text ad if the display URL goes nowhere?
Today I had a colleague ask me if Google Adwords will penalize text ads that have display URLs that land on 404 pages. I fully understand the policy that the domains in display URLs and destination URLs have to match up or Google will disapprove the text ad. However if the display url contains the proper domain (and a target keyword phrase for a file folder) will Google penalize the text ad quality score if the display URL lands on a 404? I had trouble finding specific evidence for this.
Paid Search Marketing | | VanguardCommunications0 -
Is there a tool that can use measure and improve the quality score of my PPC landing pages?
Hey guys, I'm trying to improve the quality score for a few of my PPC landing pages. Does any one know a Tool that can help me in the process. Something kind of like the "on page grader" but for quality score. Thanks
Paid Search Marketing | | Felip30 -
Why doesn't exact match appear to be working for me in Google AdWords tool?
Hi guys, I recently read Rand's article here and tried out the exact match symbols on my keywords. However, these don't appear to be working for me as the results aren't showing up as they do in the above SEOmoz article (attached screenshot). What am I doing wrong? Google_AdWords__Keyword_Tool-20130511-115528.jpg.jpg?resizeSmall&width=832
Paid Search Marketing | | featherseo0 -
Adspert and bid management tools.
Hi, Just a quickie. Has anyone on here used Adspert, or a similar bid management tool? We are currently trialing it, but I have mixed feelings about it. Most of our campaigns have the option of using conversion optimiser and I wonder if I would be better using the Google tools. What are your thoughts? Thanks Steve
Paid Search Marketing | | Steve250 -
Best keyword traffic analysis tool for long tail search terms?
Please bare with me, this might turn into a bit of a waffle, but I'll get to my question... I promise! I've just been looking at our CPC traffic for April and 2 search terms jumped out at me. I recognised them from previous keyword research because they are search terms that I expected to be high traffic (from past experience), but Google Adwords keyword tools showed them to have no potential traffic, and next to no potential traffic (literally 0 local searches and 12 local searches per month). Last month search term A had 46 visits, with 19:25 average time on site and 8.70% bounce rate and search term B had 10 visits with 14:47 average time on site and 0% bounce rate. For very boring reasons we are not currently able to measure conversions on these terms since (they are related to consumer finance and when a customer applies for finance it is all done on our finance providers website) but despite the low volume, these are pretty good figures for on site behaviour and so it got me thinking... Is there a more accurate tool to estimate traffic volume that we should be using rather than the Adwords tools? I appreciate that the estimates are probably made based on historic search behaviour and April's traffic could just be a one off, but these particular terms used to be insanely popular 4-5 years ago when I worked at a competing company.
Paid Search Marketing | | DWJames1 -
Adwords how to's and tools
Hi mozzers. I have questions. 1. Can you Share tools for google and msn adwords,example templates,etc.. 2. Any project Management Software or Template for ppc 3. The Best Book from A to Z about ppc advertisement 4. Share your MOS Importanz Stepps to Build really competitive campaign and to Be Number 1 on pair Search results. Thanks
Paid Search Marketing | | leadsprofi0 -
What's a good CTR for text ads in GMail?
I'm running text ads in Google's Display/Content network, and these ads are targeted only to display in GMail. I'm getting a rather low CTR, but this is to be expected? Then again, how low is too low? What's a good CTR for GMail ads?
Paid Search Marketing | | monetize-2660060
