Keyword Difficulty Tool
-
Hi,
When can we expect the keyword difficulty tool to be operational again?
Thanks
-
I love reading these in-depth answers from the CEO of Moz. It's so nice to see a hands-on CEO taking time to deal with even more mundane things like discussions around support issues.
-
Danny - see my responses above to your questions.
-
Permission is different than an API, IMO. Does Google have "permission" to crawl all of the world's websites and serve advertising alongside the results? No. But it's also not illegal or explicitly against those sites' terms (and they can opt out by keeping info behind a registration wall or robots.txt).
Everyone who collects rankings data - us, Raven, AuthorityLabs, Searchmetrics, Conductor, Covario, etc, etc. - does so in the same fashion. There's no special deals or APIs with search engines (Google, Bing, Yandex, Baidu - any of 'em). Anyone showing rankings data is transparent about this simply be virtue of having it.
Sadly, I do agree that nothing we do here will ever be 100% reliable - it's a tough problem, but we believe it's one worth fighting for. This information deserves to be in the hands of marketers, and we're willing to work hard on their behalf to get it (just like our competitors and colleagues in the space do).
In terms of the bet - I'll be happy to just take the losing side now and donate to Movember Will send $99 when I'm back online later today!
In terms of "The degree to which my use account usage has been affected is an irrelevance. A chunk of the $99 you receive every month from me includes the use of tools that are reliant on ranking data to function. By not issuing automatic refunds to all pro users you are simply charging money for a service that was not available."We're more than happy to issue you a full refund, but you and I disagree on the issue of automatically refunding users who didn't use the tools or for whom the service didn't break. I'm happy to chat about that more offline - you can feel free to email me at rand@seomoz.org
-
Hi Rand
It's good to finally have an honest explanation for why these tools have been so unreliable. To the best of my knowledge it has never been made clear anywhere on this website that ranking data is reproduced without the permission of Google. With this in mind, it's little wonder that these tools are so flaky.
The fact remains that you are charging for a service that you are often unable to deliver. I shouldn't have to spend my time drafting emails to your support department requesting a refund every time these tools go down.
If your aim was to be truly transparent, you would make a full declaration on the site along the lines of, "SEOmoz Tools that use ranking data do so without the permission of the search engines concerned. As a result these tools are liable to frequent downtime. Please consider this fact when signing up for a pro account as the availability of these services cannot be guaranteed". At least then users could make an informed decision as to whether to use SEOmoz tools over a competitors'.
The degree to which my use account usage has been affected is an irrelevance. A chunk of the $99 you receive every month from me includes the use of tools that are reliant on ranking data to function. By not issuing automatic refunds to all pro users you are simply charging money for a service that was not available.
Until such a time that you have the full co-operation of Google to collect this data through a licensed API, I really can't see that there is ANYTHING your tech guys could do to make these features 100% reliable. The fact is, that SEOmoz is a service that is, and probably always will be, built on technically shaky foundations.
How about a $99 bet on whether the Keyword Difficulty tool stays fully functional to the end of 2012? The loser donates the money to your Movember charity. You game?
Danny
-
Hi Danny - sorry for my delay. Been a very busy week. You asked:
"Can you please confirm for me the exact reason why the Google ranking data and the Google search volume data in KDT has always been so unreliable."
Yes, I can. What we do is adversarial analytics data. The source of data (the search engines) are not stable providers on information with SLAs and promises and deals. In fact, they are constantly changing and in some cases working actively to prevent the collection of this data. Hence, our attempts to gather this information are sometimes thwarted, as they were this past week. You can see rankings data instability in virtually every other product/software/tool that tries to collect this type of data as well (a continuing problem for all of us in this field over the past 9 years - since Google stopped providing a rankings API).
We currently have a primary system, an early warning alert system, and a backup system. Last week, we lost our primary system, our early warning system alerted us, and we moved to the backup, which had some instability and some trouble catching up to the volumes needed, but eventually did so. I believe 95% of campaign rankings were caught up as of yesterday, and it should be 100% today.
I certainly apologize for the downtime - it sucks and I want to do better. We're building a third and fourth backup system (they were actually already in the sprint for the production engineering team this month, but had to be pushed back to deal with this emergency), and we hope that will help make things more stable in the future. However, until and unless there's an API with a true guarantee, this data's reliability will always be in question.
You also asked about refunds. We always want to be very generous with our refund policy - if your account and usage has been severely impacted by this outage, please write to our help team and we will issue a refund. However, we treat refunds individually, not on the overall level, for temporary outages of individual features.
For many folks, rankings data was on time (as I noted, only a percentage of rankings collections fell behind thanks to the backup). And ~25% of our users will use the keyword difficulty tool in a month (that tool was back up and operational earlier this week, and had less downtime that some of the campaign rankings).
If PRO went down entirely, or if a tool like Open Site Explorer (used by 70%+ of members monthly) was down for a long period, I think we'd need to re-examine the individual-based refund policy, but I'm hoping it never comes to that. The worst we've had so far was a period in September where several services were throwing frequent errors and issues.
I can tell you that over the next 3 months, uptime and reliability is a HUGE focus. The production engineering team has 4 fulltime engineers, 2 contractors, and 2 open full time positions. These folks do nothing but worry about our backups and how to make sure customers get data.
Thanks for sticking by us in tough times, and if you'd like a refund, please do email help@seomoz.org.
-
Hi Peter
I appreciate you providing a full and detailed response but I cannot accept your explanation.
Users being automatically compensated should have nothing to do with how much they are affected by an issue. A big chunk of the $99 fee that you charge every month is for the provision of ranking data. That data is crucial to the both campaign ranking reports and the Keyword Difficulty Tool. If those services were never available, SEOmoz would look pretty poor value and would struggle to charge $99 per month. Charging users a fee, for features they cannot use, regardless of whether or not they actually need that feature, is far from being fair and ethical.
Imagine a situation in which your cell phone operator was unable to provide you with a mobile data service for several months. You could still make calls and send text messages but you couldn't browse the web or send emails. Whether or not you personally needed to use mobile data in those 3 months has nothing to do with the fact that you will have been paying for a service that you could not have used. After all, you could have been on a phone package that included no data and saved yourself several dollars every month!
It is for this reason that this issue should not be handled on a case by case basis. Every pro user has been getting less than they have paid for and should be compensated accordingly. This should be irrespective of whether they use ranking data or have made a complaint.
Can you please confirm for me the exact reason why the Google ranking data and the Google search volume data in KDT has always been so unreliable. This is not a temporary glitch; it has been going on for years! If you are pulling this data from Google's API; is it that the API doesn't function correctly? Or is SEOmoz scraping data from Google web pages without a contractual agreement with Google to do so?
Danny
From: Peter [mailto:notifications-support@seomoz.zendesk.com]
Sent: 01 December 2012 02:19
To: Danny
Subject: [SEOmoz Help] Re: RE: [SEOmoz Help] Re: Credit|
| |
Peter (SEOmoz Help)
Nov 30 06:19 pm (PST)
Danny,
I am happy our tools are back up too! I'm sorry we have dampened your confidence in our services, we hope to gain that back in the future with more service uptime, better and more robust services, and of course better customer experience!
Regarding mass crediting to customers, I think I can answer your questions in a short way that doesn't cause any confusions. The reason we look at customer issue on a case by case basis is because that is exactly how it sounds, the only way to solve customer issues is to physically talk to and solve every unique problem that our customers are facing. While the last week has been tough as a fair number number of problem have been about KWD and Rankings. I think you would be surprised to know that not everyone have the same problems, as SEOmoz PRO have a pretty diverse set of tools that fits a wide audience of SEO consultants which utilizes the suite rather than just one tool (or two in last week's case). The only way we know which specific problem they have is by our users reporting it on the Help@SEOmoz.org, we have tools that allows us to effectively answer and keep track of the issues and come up with unique resolutions, which can include credits/refund which we happily provide if it warrants the situation. While mass refund strategy might be one way to handle issues in your opinion, I believe it is generally not the way we should do business because it is neither generous or authentic. When our users come to us with an issue (which we have no way to know unless they report it), they want more than a credit/refund. I feel like finding ways to provide solutions that are unique to their situation is often time more generous (value-added), transparant and authentic because we admit our limitations to our customers quite freely if that is something we did wrong, we provide solutions that are Unique to their issues and we in turn take those feedback and improve our processess which pays dividends in the future of the product.
I hope this is helpful in letting you know some more of my thoughts, I think it is healthy to disagree on something, I am sorry we lost some of your confidence in our service with last week's outage, we will work hard to try to earn back not only your confidence.
Regards,
Peter|
|
-
I'm glad to see that KDT is up and running but this tool has never been stable. It has got to the point where we cannot ever rely on it to work. I have very little confidence that these problems will not re-occur.
I have repeatedly asked the question why users are not automatically compensated when such a vital part of the service is not available for extended periods of time. All I get are glib replies such as this one from Peter, "I think when it comes to finding a to compensation plans for our customers in a time like this, it is hard to just come up to a standard convention when it comes to compensating our customers for their time. We usually take care of all of our user issues on a case by case basis".
The only "hard" thing about giving compensation to users is that losing some of your profits hurts your bottom line. There is nothing technically difficult about saying "We will rebate users for every day that our tools are down". If you can do it for me, you can it for everyone!
The truth is that compensating only users who complain costs SEOmoz less money. By insisting that this is handled on a case by case basis, what you are really saying is "We are happy to charge users for a service that they have not received".
Which part of TAGFEE does this sound like? It is neither Transparent, Authentic or Generous. Perhaps your famous acronym should be changed to just FEE. This would more accurately reflect SEOmoz's current attitude to its users. Do the right thing here guys!
-
Hi all. Here's a quick update. The good news is that we tracked down the core issues and have made progress toward fixing them. Keyword difficulty is up and running again, although we are still testing and tracking down a few imperfect results. Campaign rankings are in good shape. If you had any missing rankings, they should be retrieved by late tonight, though there a very small set of locales for which rankings are not yet available. And we are still on target to have rank tracker working tomorrow.
I also wanted to note that the engineering team has been investing in building up redundancy to ensure greater resiliency in the future. Unfortunately these systems were not yet complete when the latest problems arose. Fortunately, a lot of the work that the team has put in over the last week has helped us better test and prepare those systems.
Thanks to everyone for your patience during this outage. We don't take these issues lightly and have put every available resource toward solving them. We’re truly sorry for any inconvenience this has caused you.
-
Hi everyone,
We've just updated the Known Issues page and I wanted to reiterate what Nick said over there:
Hey everyone! Some good news from our engineers.
- Keyword Difficulty is back up for Google.
- Rankings are in good condition. We have identified and are in the process of fixing the calendar icon issue. Most campaigns should be fixed late tonight. However, some locales are not available yet.
- Rank Tracker is still a little sick, but we hope to have it up soon.
Thanks for the patience everyone.- Nick
-
I have been told it will be fixed by Friday which is tomorrow.
I asked seomoz on twitter and included the CEO in it.
-
Hi Peter
Thanks for your response
You say, "we believe by providing our users with credits (often multiple months), we think that is the best way to make up for the time that our service is not working".
Your statement seems disingenuous as you a) make me no offer of compensation of multiple months and b) make no offer to other pro users unless they complain and demand compensation. Why would you not compensate all users who are not getting the service that they have paid for?
If your tools are simply scraping content from Google SERPS, then they are bound to go down every time Google decides to change the format of the page. If you don't have access to a Google API these issues are going to keep cropping up with no long term solution in sight.
So, I will ask again...
-
How will I be compensated?
-
How will you compensate all pro users? (Including those that don't complain)
-
How will this issue be fixed in the long term so that it does not keep reoccurring
I don't need your platitudes about how great I am. I just need simple, straight answers from an organisation that claims to be transparent and ethical
Danny
From: Peter [mailto:notifications-support@seomoz.zendesk.com]
Sent: 29 November 2012 17:59
To: Danny
Subject: [SEOmoz Help] Re: RE: [SEOmoz Help] Re: Credit|
| |
Peter (SEOmoz Help)
Nov 29 09:59 am (PST)
Hello Danny,
Thank you for your honest and critical feedback, I want to thank you for being forward on your feelings about our service. It is passionate user like you that keeps us to constantly improve ourselves. Regarding the recently rankings issue, we can understand that when you rely on a service as much as we rely on google for our service, sometimes when problem arises how that can impede or sometimes stop your operations. In terms of our upfront we are with our customers, we consistently update our users our known problems forum, our blog, and twitter @SEOmoz.
Regarding the compensating our users, we believe by providing our users with credits (often multiple months), we think that is the best way to make up for the time that our service is not working. The other form of compensation is obviously our commitment to properly predict and create fire drills for this kind of release from Google in the future (which we learned our lesson from this time). You can read about it here in a great Public post by our CTO: http://www.seomoz.org/blog/biting-the-bullet-of-technical-debt.
I want to both thank you for being a valued user/customer/mozzer and I do apologize for the troubles that you and the community have experience this holiday season.
Warm regards,
Peter Li
SEOmoz Help team.P,S. I will keep this ticket in my queue so I can keep you updated.
|
|
-
-
**The constant stream of excuses may be more believable if this tool has ever been reliable!
If the the ongoing issues with this tool are due to Google's API then you have got to either find a workable LONG TERM solution or charge separately for this tool on the rare occasions on which it does work.
A tool that works half the time is worse than no tool at all
When will SEOmoz be compensating all Pro users? **
-
If your customer services team was truly being "generous", you would be automatically compensating all pro users for a service that consistently fails to work. Instead you ask people to contact you privately and refund only those that shout the loudest.
I thought SEOmoz was supposed to be better than this? What happened to to the TA in TAGFEE? You may need to change this to FEG!
-
Yes, please soon. I'm in my trial period and I like what I see but this hasn't worked once for me since I signed up. I'm hoping this is not the norm--doesn't seem that way b/c the seomoz reputation is great.
-
Hi everyone,
I just wanted to chime in and let you know we are hard at work trying to get this going.
The change in Google's processes took us as much by surprise as it did everyone else. I really appreciate your understand and patience while we get this fixed.
Cheers,
Joel. -
Thank you for the update but it doesn't bode well considering its one of the core features of SEoMoz Pro.
-
Hi! This link will give you information about the Keyword Difficulty tool. We'll keep it updated, as well as our Twitter feed. https://seomoz.zendesk.com/entries/22457872-keyword-difficulty-and-rank-tracker-issues
-
In my experience, there isn't anything that remotely compares to it. SEOMoz, you have a great tool that we want to use. When will this be fixed?
-
I agree this needs to be sorted once and for all as this is getting ridiculous.
Does anyone know a decent alternative to this tool?
-
This needs to be fixed asap. I haven't been able to use it in a week and my clients are not happy!
-
The Keyword Difficulty tool is currently unresponsive due to difficulties with real-time rankings retrievals. We apologize for the inconvenience and are working to fix it.
I have been getting this message for the last week.
This tool is the most important tool i use with seomoz subscription.
Roger fix the problem!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why did our highly ranked keyword drop to 51+ for just one week?
One of our most important keywords (ranked 5) dropped to 51+ one week and then went back to 5 around the time we launched a new site. Why did that happen?
Technical SEO | | virtuance_photography0 -
How to rank for several keywords
Hi there Mozers, So far I have been building my webshop ouibonjour.com over the course of several weeks while studying the awesome educational content here on Moz and other SEO-related sites. I think I have done an OK job so far, and I have managed to rank #1 on several long tail keywords after a couple of weeks. Problem is - there are more long tail keywords I want to rank #1 for as well! **What I have done so far: ** I have taken the strongest keywords that can generate conversion and added them in page titles, title heads, main anvigation titles, in my alt text in images and so forth. The page is not screaming "SEO!" and I am happy with that. Nevertheless, I think the SEO measures here are done in an effective way. If you bothered to click on my page, you can see that the home page has only 5 posts (posts, not pages). They redirect visitors to other important parts of my site, and I change the image and/or the title once every 3 weeks to not keep the page too static. In a /magazine subfolder, I have a blog that ranks pretty good for several keywords, but the content here is not directly related to my products, but rather articles that fits the interests of my potential buyers (lifestyle, culture, etc). **What I need - but not sure how - to do: ** I basically want to rank for way more long tail keywords that relate to my products, but I don't want to continue setting up posts that have these keywords, because the content on the main page will be too heavy on the SEO, and therefore lose value for my visitors. How can I rank for more keywords without having to pepper my site with posts full of these keywords? I do not want to write articles on my blog that focuses on different combinations of long tail keywords neither, because it will get too spammy and make my visitors not return. I need help! I hope I have managed to explain my problem clearly. If there are any Whiteboard Fridays or posts on this subject that I have missed, please link me directly. If not, I would be tremendously happy and appreciative for any tips and tricks 🙂
Technical SEO | | Fernando_0 -
Remove Directory In Webmaster Tools
Hey Moz'erz, I'm removing some URLS from the index and want to confirm the use of the "remove directory" request. If my structure is this: /blogs/customer-success-stories/tagged/ --- all pages that are /tagged/abc, /tagged/dce etc. will be removed correct? First time trying a directory removal as their are 100 plus of these tagged pages. Comments, suggestions and past experiences welcome!
Technical SEO | | paul-bold0 -
Google dance tools - do they still give an indication of rankings to come?
I can't remember the tool I used in the past but it searched a number of data centres which gave a pretty good indication of what your new rankings would be once full updated, is this still the case today? If so, can anyone recommend any tools?
Technical SEO | | designguy760 -
Bogus Crawl Errors in Webmaster Tools?
I am suddenly seeing a ton of crawl errors in webmaster tools. Almost all of them are URL links coming from scraper sites.that I do not own. Do you see these in your Webmaster Tools account? Do you mark them as "fixed" if they are on a scraper site? There are waaaay too many of these to make redirects. Thanks!
Technical SEO | | EGOL0 -
Someone is building anchor text links with child porn keywords..help :-(
Hey Everyone; I recently found that there are tons of links to my url (30+) from several sites with the anchor text "download child porn, kiddie porn, and the likes". Although the sites linking to us where the person built the links do not have this content (thank god) it is just my url + the horrible anchor text someone is putting. [added image] Obviously I would not do that to my own site especially since I was ranked for over 15 keywords in the top 5 spots on Google and Bing. I am guessing its a competitor and I am wondering that is there any way I can remove them? Is this negative seo? Also could this be the reason my site has dropped off google page 1 last month for all my positions? I am consistently trying to rebuild but my positions are flying all over the place. Im going from ranking #12 one week to #198 the next; then to #60 and finally not even on google or bing anymore; then back to #50, etc for the past month and a half... Its very aggravating and it has hurt a few of my websites; im at a total loss on how to recover now if this is the case...It seems they have started to build these links in January from what I can find on the postings.. Thank you SEOMOZ im so happy im a member now. Any suggestions on how i can get these links off or if it is why my site has lost its original rankings.. Should I just can my website and start all over with a different URL? How can I watch out for this in the future? Any help would be so so so greatly appreciated and valued 🙂 xpVjMw8
Technical SEO | | Circa0 -
Webmaster Tools - Clarification of what the top directory is in a calender url
Hi all, I had an issue where it turned out a calender was used on my site historically (a couple of years ago) but the pages were still present, crawled and indexed by google to this day. I want to remove them now from the index as it really clouds my analysis and as I have been trying to clean things up e.g. by turning modules off, webmaster tools is throwing up more and more errors due to these pages. Below is an example of the url of one of the pages: http://www.example.co.uk/index.php?mact=Calendar,m1a033,default,1&m1a033year=2084&m1a033month=3&m1a033returnid=59&page=59?phpMyAdmin=xxyyzz The closest question I have found on the topic in Seomoz is: http://www.seomoz.org/q/duplicate-content-issue-6 I want to remove all these pages from the index by targeting their top level folder. From the historic question above would I be right in saying that it is: http://www.example.co.uk/index.php?mact=Calendar I want to be certain before I do a directory level removal request in case it actually targets index.php instead and deindexes my whole site (or homepage at the very least). Thanks
Technical SEO | | Mitty0 -
Strange Webmaster Tools Crawl Report
Up until recently I had robots.txt blocking the indexing of my pdf files which are all manuals for products we sell. I changed this last week to allow indexing of those files and now my webmaster tools crawl report is listing all my pdfs as not founds. What is really strange is that Webmaster Tools is listing an incorrect link structure: "domain.com/file.pdf" instead of "domain.com/manuals/file.pdf" Why is google indexing these particular pages incorrectly? My robots.txt has nothing else in it besides a disallow for an entirely different folder on my server and my htaccess is not redirecting anything in regards to my manuals folder either. Even in the case of outside links present in the crawl report supposedly linking to this 404 file when I visit these 3rd party pages they have the correct link structure. Hope someone can help because right now my not founds are up in the 500s and that can't be good 🙂 Thanks is advance!
Technical SEO | | Virage0