Thoughts on scraping SERPs and APIs
-
I'm perplexed about a couple of things.
Google says that scraping keyword rankings is against their policy from what I've read. Bummer. We comprise a lot of reports and manual finding and entry was a pain. Enter Moz! We still manually check and compare, but it's nice having that tool. I'm confused now though about practices and getting SERPs in an automated way. Here are my questions
- Is it against policy to get SERPs from an automated method? If that is the case, isn't Moz breaking this policy with it's awesome keyword tracker?
- If it's not, and we wanted to grab that kind of data, how would we do it? Right now, Moz's API doesn't offer this data. I thought Raven Tools at one point offered this, but they don't now from what I've read. Are there any APIs out there that we can grab this data and do what we want with it? (let's day build our own dashboard)?
Thanks for any clarification and input!
-
You're much too kind Happy New Year!
-
Hi Rand. Anyone ever tell you you're awesome?
I just did. Thanks bud
-
Hi Boogily,
A) In terms of ToS, I'm not 100% clear. I know that Google has been strict about automated rank checking programs installed on a user's local machine, and they've enforced that a bit through technical and guidelines means. However, they've been unclear about companies like Moz, Searchmetrics, GetStat, AuthorityLabs, Conductor, Brightedge, Yield, RankAbove, and hundreds of others which provide this data in their software. I suspect they don't love it, but tolerate it and haven't taken action for over a decade now.
B) On the API front, check with GetStat and AuthorityLabs. I believe they both have rank data offerings from an API standpoint. Moz doesn't offer that and probably won't in the future, either.
Cheers!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can I batch export top 5 or 10 SERPS for each keyword in my campaign
I'm a newby. I have about 70 keywords and I want to create a list of all of the websites ranking in the top say 10 positions. This is to create a list of competitors in my client's space. How do I do this? I poked around but can only see single-keyword search capability to get the full serp listing (top 50). thanks.
Moz Pro | | empiricalSEO0 -
Domain_to_domain api call syntax
I am trying to construct an api call that lists the top authority link from each domain. This page_to_page call works http://lsapi.seomoz.com/linkscape/links/www.jbwebanalytics.com?SourceCols=4&TargetCols=4&Scope=page_to_page&Limit=400&AccessID=membexxx&Expires=xxx&Signature=xxx This domain_to_domain does not http://lsapi.seomoz.com/linkscape/links/www.jbwebanalytics.com?&Scope=domain_to_domain&Limit=400&AccessID=member-xxx1&Expires=xxx&Signature=xxx I'm plum out of ideas why? Anyone know? The Idea output is the single highest authority link for all linking domains thanks! Brian
Moz Pro | | VISISEEKINC0 -
SERP Changes depending on ISP
If I search Google for my site using my mobile internet provider I get different results from my main internet. Before anyone suggests it, it's nothing to do with saved searches as I'm clearing cookies and I've tried on multiple different platforms. I can plug my iPad and use my wireless, it's my normal rankings, but as soon as I switch to my mobile provider for internet, I'm 3rd instead of 2nd on the SERPs. Any ideas?
Moz Pro | | awdgareth0 -
Mozscape API call returning no json
Hi - I'm having issues getting started with the Mozscape API. I'm putting the following into my browser: http://[member-ID]:[secret key]@lsapi.seomoz.com/linkscape/url-metrics/greatbritishchefs.com According to the blog posts I'm reading I should get loads of json script returned but all I'm getting is {} Does anybody have any pointers as to what I might be doing wrong? Thanks
Moz Pro | | GBC0 -
What does the "Internal Links" data in the Keyword Difficulty SERP report represent?
What does the "Internal Links" data in the Keyword Difficulty SERP report represent? Thank you! QocBS
Moz Pro | | richpalpine0 -
Rank Tracker via API
Hello SEOMoz, The functionality of the Rank Tracker Tool is excellent - I was wondering if it might be coming to the API so we can do some deeper analysis? Thanks!
Moz Pro | | igzebedze0 -
Report: keyword difficulty verses SERPS ranking
Hi there, I've been using the PRO tools for a few months now, and find them great for revealing information for my clients, and planning our SEO approach. One report I've put together has been really helpful for a client, but I built it manually and it took ages. It lists, in a spreadsheet, keywords side-by-side with keyword difficulty, target URL, and SERPs result. Sorting by keyword difficulty or SERPS position has helped us to prioritise which pages to hit first. But it took a while to generate the report, and I'm wondering if anyone has a better approach, or if this kind of report can be generated automatically: http://www.cicada-online.com/app/images/documents/product-page-SEO-blank.pdf Thanks Mozzers!
Moz Pro | | nedwells0 -
Any tools for scraping blogroll URLs from sites?
This question is entirely in the whitehat realm... Let's say you've encountered a great blog - with a strong blogroll of 40 sites. The 40-site blogroll is interesting to you for any number of reasons, from link building targets to simply subscribing in your feedreader. Right now, it's tedious to extract the URLs from the site. There are some "save all links" tools, but they are also messy. Are there any good tools that will a) allow you to grab the blogroll (only) of any site into a list of URLs (yeah, ok, it might not be perfect since some sites call it "sites I like" etc.) b) same, but export as OPML so you can subscribe. Thanks! Scott
Moz Pro | | scottclark0