In lue of the canceled Moz Index update
-
Hey Moz, Overall we love your product and are using it daily to help us grow, part of that has been to rely on the Moz Index for DA and PA as well as places where we are doing positive linking through genuine partnerships and reviews of clients. We were really excited to see any the results for this month as we have been partner linked from lots of high reputation sites and google seems to agree as our rankings are moving up weekly.
The question from our marketing team is, since a significant part of Moz will not be available to us this month, will there be any compensation handed out to the paying community.
PS: I am an engineer and I know how you have probably lost a very large set of data which cant simply be re-crawled over night but Moz Pro is not a cheap product and we do expect it to work.
Source: https://moz.com/products/api/updates
Kind Regards.
-
Thanks, I wont be giving up my subscription just yet as I think the work Moz does in terms of whiteboard fridays and everything else is worth it. I have spoken to your live help and reached a more than satisfiable outcome. Just a fruit for thought, the crawlers should be running 24/7 365 days and doing less work over a longer period, my job is a big data Analytics engineer / DevOps and this helped us solve a ton of issues at the petabyte scale.
-
I can't help but agree with you. Over the last few years, we've consistently had terrible delays releasing indexes, and despite a team of people way smarter and more talented than me working their tails off to get it right, we haven't had success making it work regularly yet.
This latest index cancellation is embarassing and it sucks. We produced an index, but when we looked at it, a huge problem had arisen that we couldn't see until processing was complete (another problem with our indices is our inability to get a good sense of what they'll look like until they're done which takes 20+ days of processing after a crawl). I'll detail that in a Q+A thread soon (once I get the full rundown and plan from our Big Data team) and then share around.
In any case, you have my sincere apologies and deep regrets. We'll keep trying to get this right, but just FYI - we've simultaneously been building a new index system that's more real-time (like Google's, Ahrefs, Majestic, etc) that can still calculate metrics like MozRank and Page Authority. We've made a lot of progress on it, but it's still probably 6+ months away from launching, so we'll have to deal with the old Mozscape system until then.
-
It happens almost every time they set a date to release an Index Update. Usually they just push it back a week or two, then do it again, finally releasing the updates weeks after it was initially scheduled. This time they took a different approach and cancelled it all together.
I love MOZ for keyword rank tracking, keyword difficulty, etc, but have almost given up on their Indexes. There seems to be no consistency with the bank link data from one index to the next anyway, so you're really only able to measure relative changes in backlink profiles against competitors. Getting index updates when they're scheduled seems to be a rare occurrence these days, so you may want to get used to not getting data when expected, and I highly doubt there will be any compensation.
I understand we are talking about huge amounts of data... a huge undertaking that can't be easy. My advice to MOZ on the index updates, under promise and over deliver, not the other way around. Just tell us the next Index Update will be in two months, then release it in a month and a half (or on time) and everyone will be happy you came out with it early. Much better than stringing everyone along with expectations of an update then hearing another excuse the day after the update is cancelled/postponed.
Hopefully they can get this figured out and prevent future issues with index updates and customer frustration with this part of the product.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemaps and Indexed Pages
Hi guys, I created an XML sitemap and submitted it for my client last month. Now the developer of the site has also been messing around with a few things. I've noticed on my Moz site crawl that indexed pages have dropped significantly. Before I put my foot in it, I need to figure out if submitting the sitemap has caused this.. can a sitemap reduce the pages indexed? Thanks David.
API | | Slumberjac0 -
Sitemaps and Indexed Pages
Hi guys, I created an XML sitemap and submitted it for my client last month. Now the developer of the site has also been messing around with a few things. I've noticed on my Moz site crawl that indexed pages have dropped significantly. Before I put my foot in it, I need to figure out if submitting the sitemap has caused this.. can a sitemap reduce the pages indexed? Thanks David. TInSM
API | | Slumberjac0 -
MOZ API - Search Visibility
Hello there, We are looking to see if we can recreate the Search Visibility report through the API. We are wondering which API parameters and metrics we should use to recreate the report? Also, we want to distinguish between traffic coming from desktops and mobile traffic. Is this possible with the API? Which type of account do we need to use for these functionalities: the free account or paid account? We are currently using a free account. Also: we are wondering if can access historical data with the API? For instance: we want to determine the domain authority from October 2016 off on. Is this possible? Thanks in advance! Kind regards, Bart Minten
API | | thomas.deruiter0 -
Frequency of Moz page authority updates?
I have some new pages on my site, and Moz gives them a very low PA ranking. I am wondering if these scores are updated monthly or quarterly? I'm not sure how frequently to check back for updated scoring.
API | | AndrewMicek0 -
August 3rd Mozscape Index Update (our largest index, but nearly a monthly late)
Update 5:27pm 8/4 - the data in Open Site Explorer is up-to-date, as is the API and Mozbar. Moz Analytics campaigns are currently loading in the new data, and all campaigns should be fully up-to-date by 4-10pm tomorrow (8/5). However, your campaign may have the new data much earlier as it depends on where that campaign falls in the update ordering. Hey gang, I wanted to provide some transparency into the latest index update, as well as give some information about our plans going forward with future indices. The Good News: This index, now that it's delivered, is pretty impressive. Mozscape's August index is 407 Billion URLs in size, nearly 100 Billion (~25%) bigger than our last record index size. We indexed 2.18 trillion links for the first time ever (prior record was 1.54 trillion). Correlations for Page Authority have gone up from 0.319 to 0.333 in the latest index, suggesting that we're getting a slightly more accurate representation of Google's use of links in rankings from this data (DA correlations remain constant at 0.185) Our hit ratio for URLs in Google's SERPs has gone up considerably, from 69.97% in our previous index to 78.66% in the August update. This indicates we are crawling and indexing more of what Google shows in the search results (a good benchmark for us). Note that a large portion of what's missing will be things published in the last 30-60 days while we were processing the index (after crawling had stopped). The Bad News: August's index was late by ~25 days. We know that reliable, consistent, on-time Mozscape updates are critically important to everyone who uses Moz's products. We've been working hard for years to get these to a better place, but have struggled mightily. Our latest string of failures was completely new to the team - a bunch of problems and issues we've never seen before (some due to the index size, but many due to odd things like a massive group of what appear to be spam domains using the Palau TLD extension clogging up crawl/processing, large chunks of pages we crawled with 10s of thousands of links which slow down the MozRank calculations, etc). While there's no excuse for delays, and we don't want to pass these off as such, we do want to be transparent about why we were so late. Our future plans include scaling back the index sizes a bit, dealing with the issues around spam domains, large link-list pages, some of the odd patterns we see in .pl and .cn domains, and taking one extra person from the Big Data team off of work on the new index system (which will be much larger and real-time rather than updated every 30 days) to help with Mozscape indices. We believe these efforts, and the new monitoring systems we've got will help us get better at producing high quality, consistent indices. Question everyone always asks: Why did my PA/DA change?! There are tons of reasons why these can change, and they don't necessarily mean anything bad about your site, your SEO efforts, or whether your links are helping you rank. PA and DA are predictive, correlated metrics that say nothing about how you're actually performing. They merely map better than most metrics to Google's global rankings across large SERP sets (but not necessarily your SERPs, which is what you should care about). That said, here's some of the reasons PA/DA do shift: The domains/pages with the highest PA/DA scores gain even faster than most of the domains below them, making it harder each index to get higher scores (since PA/DA are on a logarithmic scale, this is smoothed out somewhat - it would be much worse on a conventional scale, e.g. Facebook.com 100, everyone else 0.0003). Google's ranking algorithm introduces new elements, changes, modifies what they care about, etc. Moz crawls a set of the web that does or doesn't include the pages that are more likely to point to a given domain than another. Although our crawl tends to be representative, if you've got lots of links from deep pages on less popular domains in a part of the web far from the mainstream, we may not consistently crawl those well (or, we could overcrawl your sector because it recently received powerful links from the center of the web). My advice, as always, is to use PA/DA as relative scores. If your scores are falling, but your competitors' are falling more, that's not a bad thing. If your scores are rising, but your competitors' are rising faster, they're probably gaining ground on you. And, if you're talking about score changes in the 1-4 points range, that's not necessarily anything but noise. PA/DA scores often shift 1-4 points up or down in a new index so don't sweat it! Let me know if you've got more questions and I'll do my best to answer. You can also refer to the API update page here: https://moz.com/products/api/updates
API | | randfish8 -
About Expiry of MOZ API authentication and MOZ API call in .NET application
MOZ API signed authentication is getting expire after particular time, and coming up with the error "Authentication has expired , Re-send your authentication". **right now we are getting new authentication details using ** https://moz.com/products/api/keys after doing login. but, how do we get new credentials (authentication details) , by doing MOZ API call using c# or any other programming language, we are using this API in .NET. Also, How we can increase the session expiry time. i think it is concerned with the Sample Expires of Signed authentication , but we are unable to change it , as it is coming with the authentication failed.
API | | BenuAggarwal0 -
Does any one know if there is a tool out there built with the Moz API where you can feed in a large list of URLs and get back Domain Authority?
Does any one know if there is a tool out there built with the Moz API where you can feed in a large list of URLs and get back Domain Authority? Has anyone used the API for something like this before? Thanks! Max
API | | Porch0 -
Are EC2 servers blocked from using the Moz API?
Hi I've created a script that I'd like to use to check a list of domains using the Moz API. It works totally fine on my local machine. However, when I run it from my EC2 instance, it fails every time. To be specific, the response is always empty (the response is an empty json array) when the request is sent from EC2. Is, for some reason, EC2 blocked by the Moz API? Many thanks for your help, Andrew
API | | csandrew0