Thoughts on scraping SERPs and APIs
-
I'm perplexed about a couple of things.
Google says that scraping keyword rankings is against their policy from what I've read. Bummer. We comprise a lot of reports and manual finding and entry was a pain. Enter Moz! We still manually check and compare, but it's nice having that tool. I'm confused now though about practices and getting SERPs in an automated way. Here are my questions
- Is it against policy to get SERPs from an automated method? If that is the case, isn't Moz breaking this policy with it's awesome keyword tracker?
- If it's not, and we wanted to grab that kind of data, how would we do it? Right now, Moz's API doesn't offer this data. I thought Raven Tools at one point offered this, but they don't now from what I've read. Are there any APIs out there that we can grab this data and do what we want with it? (let's day build our own dashboard)?
Thanks for any clarification and input!
-
You're much too kind
Happy New Year!
-
Hi Rand. Anyone ever tell you you're awesome?
I just did. Thanks bud
-
Hi Boogily,
A) In terms of ToS, I'm not 100% clear. I know that Google has been strict about automated rank checking programs installed on a user's local machine, and they've enforced that a bit through technical and guidelines means. However, they've been unclear about companies like Moz, Searchmetrics, GetStat, AuthorityLabs, Conductor, Brightedge, Yield, RankAbove, and hundreds of others which provide this data in their software. I suspect they don't love it, but tolerate it and haven't taken action for over a decade now.
B) On the API front, check with GetStat and AuthorityLabs. I believe they both have rank data offerings from an API standpoint. Moz doesn't offer that and probably won't in the future, either.
Cheers!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEOMoz API not working for Scrapebox
I want to import SEOMoz data to list of URLs I have using scrapbox. I added in my credentials according to the API but am getting error 401 as the status of all my links. Any idea why and what I should be doing?
Moz Pro | | theLotter0 -
What search engines does SEOMoz pull the SERP data from on the keyword analysis report?
I did a Google search and I noticed that some of the rankings that pulled up did not match the SEOmoz top 1-10 competitors. I was sure not to include personal results. I was completely signed out of my Google account. Another one of the SEO tools showed the same result as when I did the Google search, completely different from SEOmoz's report. I am curious to know where the data comes from, so I can accurately compare/ contrast the varying results.
Moz Pro | | JMFieldMarketing0 -
Why my seomoz keyword ranking are different to google SERPs
I use Cute Rank to check my keywords ranking in Google, but after last update seomoz is showing some of keyword ranked much higher than Cute Rank. I checked them manually and they are on positions showed by Cute Rank and not by Seomoz. Can someone explain me why is that?
Moz Pro | | misterdam0 -
Learn SEOmoz API (and more)?
I'm sorry for this very basic question. The reason why I'm asking is that I wan't to start to use SEOmoz API (and later on Majestic SEO etc.) I have search on the web about API in general, SEOmoz specifically, in Q&A and asked around but:
Moz Pro | | OscarSE
I really can't get my head around what "set of skills" I need to know to be able use the SEOmoz API? And other APIs for that reason. Do I need a programming skills, or just knowledge how each specific API works(+ common sense)? Or both? I'm sorry if this is off topic but the main reason why I'm asking is that I wan't to start to use the SEOmoz API. If you could provide a link or two about where I can start to learn about how this works, I would really appreciate that!0 -
How to get the total external links to a page and total external links to the domain using Mozscape API? I could not see an option in the bit flags
Hi, I was trying to get the data for total external links to a page and total external links to the domain using Mozscape API but I can't see a bit flag which can do that. There are bit flags for external followed links to a page and external followed links to a domain but I wanted the total external links data, is there a way to do that using Mozscape API else I would end up copying the data manually from OSE which would be cumbersome and time consuming. Your help is highly appreciated.
Moz Pro | | HQP0 -
Some thoughts on MozTrust based on OSE Findings X ref'd with SERPS
I've been doing a bit of competitor analysis for a client using OSE. There are a group of about 4 websites (our clients website included) that all dominate the sector with none of the 4 clearly out in front (call this GROUP A). Then there are another group of about 5 websites, which come lower in the SE's consistently than the top 4 (Call this GROUP B) **I've been doing some analysis in OSE: ** ALL GROUP B Websites outrank all of GROUP A websites in the OSE Metrics (Including Trust Rank). I did some analysis on the backlinks in Group A VS Group B Group A - Generally a mixture of ok links from blog posts, sponsorship, and ok directories. Group B - As A, but with fewer numbers of links from quality blogs PLUS A high level of spammy links ( .edu and .gov spam filled pages), very low quality, almost non legible blog posts on MFA sites (think Digital Point sellers). From the above it is clear that the OSE metrics are out of whack with the real SE results. Clearly OSE has a few problems with working out what are spammy links and what are decent. Obviously google also has issues with working this out, so I am not surprised that OSE also does - but that doesn't solve the issue. This is a general discussion - so I would just throw in a few thoughts on how OSE may possibly try are overcome some of these issues : 1/. % Trust Links vs Non trust Links:
Moz Pro | | James77
Add in a metric to Trust Rank where the number of links close to trusted sites are also compared to the number of links not close to trusted sites. If you see a very high ratio of links from sites that are not close to trusted sites, it is a strong indicator of spammy links. 2/. Use seed "Non Trusted" sites to create a negative Trust Rank
Use something like a reverse of the "trusted sites" theory, but taking a load of very clearly spammy / link manipulative sites and work out in terms of links connections how far the site is away from these sites. Thoughts???0 -
Need assistance with tool development using SEOMoz & Google APIs
I don't know where the right place to ask this is, but I work for an SEO agency, and we are looking for someone to help us with development of some tools utilizing the SEOMoz API and probably the Google API as well. Does anyone know where I could find a person with some previous experience with development using these APIs? We've had poor luck just trying to use a developer without specific experience in this area or at least some SEO knowledge. If you're a developer and would like to talk with us, you can contact me directly if you prefer, rather than post your contact info publicly, but I welcome any helpful thoughts or ideas regarding development of SEO tools. Thanks,
Moz Pro | | BandLeader
John
jmaher [at] mcdia.com0 -
Related Backlinks by Keyword SERP results
Is there no tool to search a keyword and find related backlinks among the top10,20,30 SERP results?
Moz Pro | | FirePowered0