Also, notably, Moz Analytics is way, way better than the old Pro app (so many more features, way better charts, new landing pages report that helps get back <not provided="">data, and more) :-)</not>
Best posts made by randfish
-
RE: SEOmoz PRO app not offered
-
RE: In lue of the canceled Moz Index update
I can't help but agree with you. Over the last few years, we've consistently had terrible delays releasing indexes, and despite a team of people way smarter and more talented than me working their tails off to get it right, we haven't had success making it work regularly yet.
This latest index cancellation is embarassing and it sucks. We produced an index, but when we looked at it, a huge problem had arisen that we couldn't see until processing was complete (another problem with our indices is our inability to get a good sense of what they'll look like until they're done which takes 20+ days of processing after a crawl). I'll detail that in a Q+A thread soon (once I get the full rundown and plan from our Big Data team) and then share around.
In any case, you have my sincere apologies and deep regrets. We'll keep trying to get this right, but just FYI - we've simultaneously been building a new index system that's more real-time (like Google's, Ahrefs, Majestic, etc) that can still calculate metrics like MozRank and Page Authority. We've made a lot of progress on it, but it's still probably 6+ months away from launching, so we'll have to deal with the old Mozscape system until then.
-
RE: Keyword Explorer is Now Live; Ask Me Anything About It!
It will likely be a few months at least before we can broaden outside the US, and maybe some more before we get to non-English languages. Sorry about that! We will get there; just need to do a lot of work to make it happen.
-
10/14 Mozscape Index Update Details
Howdy gang,
As you might have seen, we've finally been able to update the Mozscape index after many challenging technical problems in the last 40 days. However, this index has some unique qualities (most of them not ideal) that I should describe.
First, this index still contains data crawled up to 100 days ago. We try to make sure that what we've crawled recently is stuff that we believe has been updated/changed, but there may be sites and pages that have changed significantly in that period that we didn't update (due to issues I've described here previously with our crawlers & schedulers).
Second, many PA/DA and other metric scores will look very similar to the last index because we lost and had problems with some metrics in processing (and believe that much of what we calculated may have been erroneous). We're using metrics from the prior index (which had good correlations with Google, etc) until we can feel confident that the new ones we're calculating are correct. That should be finished by the next index, which, also, should be out much faster than this one (more on that below). Long story short on this one - if your link counts went up and you're seeing much better/new links pointing to you, but DA/PA remain unchanged, don't panic - that's due to problems on our end with calculations and will be remedied in the next index.
Third - the good news is that we've found and fixed a vast array of issues (many of them hiding behind false problems we thought we had), and we now believe we'll be able to ship the next index with greater quality, greater speed, and better coverage. One thing we're now doing is taking every URL we've ever seen in Google's SERPs (via all our rank tracking, SERPscape, the corpus for the upcoming KW Explorer product, etc) and prioritizing them in Mozscape's crawl, so we expect to be matching what Google sees a bit more closely in future indices.
My apologies for the delay in getting this post up - I was on a plane to London for Searchlove - should have got it up before I left.
-
RE: Bye Bye Keyword Difficulty Tool :(
Hi Greg - totally hear you, but strongly, strongly disagree
I worked personally on the scores for both and I can promise that the old KW Difficulty tool's numbers just aren't right. The old tool frequently over or understates the difficulty of ranking, and it relies on metrics that are outdated (age of domain? yech). I would strongly advise you to switch to using the metrics from KW Explorer. They're more accurate, the spread is better (the old KW Difficulty tool scrunches up scores so almost everything is between 30-70, when it really should be a wider spread), the other metrics are way more useful (CTR % and volume), and the accuracy of the metrics fetches is solid too (sometimes, old KW Difficulty doesn't even grab data correctly).
A few examples:
- "Harry Potter" - old tool says 80, KW Explorer says 89 (not a huge difference, but you can see what I mean about the scrunching effect -- clearly this KW should be one of the highest difficulties possible)
- "calendar app" - old tool says 62, KW Explorer says 76 (on a hand review, I think we'd all agree 76 is far more accurate; this is a very tough keyword)
- "northwest moss garden examples" - old tool says 47, KW Explorer says 33 (this is one of the easier keywords out there, with lots of low DA sites in the results; I think 33 is far more indicative of reality)
I know it's tough to make a switch or trust something new, but having studied these both closely and worked on the design of the metrics and data for both, I can assure you KW Explorer's Difficulty metrics are head and shoulders above the old tool.
-
RE: Mozscape Index update frequency problems?
Hi Kevin - the index update should be live as of right now (probably only a few hours after you posted this message). We aim to have one index update per month, so 12 per year. We had a catastrophic failure on our early January index, so it had to be abandoned (noted here: https://moz.com/products/api/updates), but the team has been working hard to fix issues and prevent others from arising. Unfortunately, it's often the case that we encounter new/unexpected/never-before-seen issues that need to be addressed. Frustrating, but unavoidable as best we can tell. Obviously, we will continue to do our best to get these indices out on time.
As far as the future goes, it's hard to say. We have a bigger team now than we last year -- 4 folks work full time on the Mozscape index and 4 are working on the next-generation version of the index (which will update in near-real-time), and we certainly have much better monitoring and operational structures in place. But, as I noted above, it seems that the errors/issues we face are always new and unique - things we've never seen before in the 100+ index runs we've had over the last 8 years. I can tell you we're building processes to identify problems before they happen, and that we're better staffed, and that we have engineers on-call 24/7 to fix issues if they crop up, but processing full-graph metrics on a full-scale web index whose shape and composition can vary wildly means there's still uncertainty and probably always will be. Our job is to keep reducing that uncertainty and finding optimizations, while we rebuild the full system in the background to eventually replace the old, batch-processing system that's at the core of so many of our challenges.
Hope that helps.
-
RE: Lost 50% google traffic in one day - panic?
Hi Dean - not sure we could (or would want to) segment Q+A or the blog/YOUmoz to create a sub-section specifically about sites being penalized by Google spam updates. However, it sounds like some folks in this thread, like Georg below, may take this upon themselves (which is awesome).
Little known story - SEOmoz's more prominent focus on community began in earnest after a large exodus of members from another forum way back in 2004.
-
RE: Best Location to find High Page Authority/ Domain Authority Expired Domains?
You might also check out http://flippa.com. Just be careful with buying expired domains, presuming that PA/DA may still be preserved. Google is actually taking a lot of steps to reset a site's authority/link profile on change of ownership to prevent manipulation through expired/sold domains.
-
RE: Subdomains vs. Subfolders Inheriting Authority/Ranking Value
Hi Kristina - I actually wrote an in-depth post about this and did a whiteboard Friday on the topic:
- http://www.seomoz.org/blog/understanding-root-domains-subdomains-vs-subfolders-microsites
- http://www.seomoz.org/blog/whiteboard-friday-the-microsite-mistake
Basically, the answer is "sometimes" subdomains will be connected with the root domain and other subdomains in the eyes of the engines, but other times, this isn't the case.
In the specific one you're describing, my guess is that much of the root domain authority and ranking power will pass to the subdomain, so long as there are good links passing between the two (indicating a relationship, rather than a new, separate site).
best,
Rand
-
RE: When is OSE updated?
Yeah- - we're targeting April 30th right now. Our index is taking much longer to run than expected. We think there may be a hardware or software issues in processing that's having problems. We'll be doing maintenance so we can get back to faster indices ASAP.
-
RE: Is Google stupid?
We've seen a pattern with this stuff - it works temporarily, then gets cut in value. TLA is the one I tested last year - rankings went way up, but as soon as we dropped the links they fell (indicating that TLA stock did work). However, when we observed and talked to folks who'd turned it on and kept it on, they reported that rankings, then dropped (within ~90 days with TLA).
Recently, we've seen Google start banning many private blog networks (warning, links below contain a LOT of black hat stuff and some none-too-friendly posters as well, unfortunately):
http://www.internetmarketingtoolsupdate.com/internet-marketing-tips/seo-link-monster-warning/
Those are all in just the past week.
Google's not stupid, but I do think their webspam team has been swamped. Recently though, it feels like they're making a bit more progress, and I wouldn't be surprised if they continue kicking some tail in the link spam/black hat world.
-
RE: Very basic hands-on type of question about SEO
Three things I'd recommend checking out:
- This video on SEO basics - http://www.seomoz.org/learn-seo (at the top)
- The Beginner's Guide to SEO - http://guides.seomoz.org/beginners-guide-to-search-engine-optimization
- This guide to on-page optimization, which can help answer a lot of the specific questions you've got here - http://www.seomoz.org/blog/perfecting-keyword-targeting-on-page-optimization
As Joel pointed out, the on-page optimization tool that's part of PRO can also be helpful, but it's good to have an understanding of the what and why, too.
This powerpoint deck might also be helpful if you like PPT format as a way to run through things - http://www.seomoz.org/blog/a-comprehensive-intro-to-seo-powerpoint-slide-deck-
Wish you the best of luck!
-
RE: What's the Story on Mozscape Updates?
Unfortunately (or maybe fortunately), those issues are unrelated. The campaigns getting behind on updating was due to some server configuration challenges as I understand it - somehow our monitoring falsely told us things were fine, and thus, even after we fixed the issue, we've been trying to play catch up on thousands of weekly/monthly reports. I'd seen an email from the team lead last night that we should be nearly clear - maybe a day or two away.
Re: memberlist - that's a separate team, and they're aware of it, so should get a fix out soon. I think one of Moz's challenges is that we've bitten off a bit more than we can chew (that's my opinion, not necessarily representative of the whole company) and with 6+ engineering teams handling 100s of operations across a giant software suite, it feels like we're always playing whack-a-mole. We fix one thing, and the next week, 3 more things we never imagined could go wrong do. It's a frustrating problem, to be sure, and I honestly think the only way we'll break out of the cycle is to cut back to fewer projects and staff up teams, which will take time, discipline, dollars, and some good luck.
-
RE: Open Site Explorer.. Bit of a let down?
Hi Lee - thanks for bringing these up! I'll try to answer with regards to each of the items you've mentioned:
#1 - This is very tough at our scale (literally 1 trillion+ links, and in the latest index about to launch, 150 billion+ pages across nearly 200 million domains). However, you're correct that we're working towards a classification system that will lean on some seeding + machine learning + user input. I'd suspect 12-18 months before we can launch it, though.
#2 - This is not currently in the plans, at least not at full web scale. We will have Blogscape back up and running soon, and that crawls ~10mm+ fresh sources daily, so if you have a link/mention on a blog/forum/news site with any notoriety, we should be catching those and updating hopefully every 6-8 hours. Mozscape (aka Linkscape, our full web index), will continue to be at least 2-4 weeks between updates and will require 3-4 weeks of processing. We're trying some new things on the technology side, but it's a huge challenge to get to Google's scale and keep our metrics, sorting, views, etc. Majestic can do this with their index because they simply export links directly into the consummable portion of the app, but it limits the sorting/views/filters and ability to generate high quality metrics like PA/DA/mozRank/etc.
#3 - Yup. Definitely agree and working on it. We're trying some new hardware stuff, new parallelization of processing and everything under the sun to make it work.
#4 - Our next index will probably have a lot more of this (and we'll be watching carefully to see how this performs for our customers). We're also working to build a spam score that maps against what we've seen Google penalize/ban over time (currently doing a bunch of research there), so you can get a good sense of what things are likely to be hit over time.
Mozscape/OSE is a huge priority for us and we have 7 extremely talented folks working night and day (and a lot of weekends) to improve this. In the next 3-6 months, it will get massively better than it is today - the next new index is only a few days away now, and after that, we shouldn't ever have such a long period without an update (this round was an impossibly hard-to-fathom confluence of problems that we've taken many steps to prevent from ever happening again).
Thanks for your patience and the good questions.
-
RE: Are moz analytics now redundant ?
Couple thoughts:
-
Google's always done this, they've just been more aggressive recently. Moz metrics should still be useful to help separate the wheat from the chaff, but any non-editorially-acquired links will always be risky.
-
We are working on a MozSpam score. Starting some data collection for a training model next week actually. I'd expect 4-6 months before it rolls out, but hopefully will be useful for exactly this kind of thing.
-
-
RE: Source? Google says having an author photo increases CTR by 15% on average.
Hi Project Labs! Glad you liked the interview. There are a few sources, but http://searchengineland.com/is-google-authorship-affecting-rankings-today-168230 is one of the best (note the 15% average across the sites). Unfortunately, I can't seem to find the reference to Google's formal statement, but I believe it was also via a SELand post (or possibly from an SMX conference stage discussion).
-
RE: Why is OSE showing a higher Domain Authority than Moz?
Hi Maureen - sorry about that! The issue is with the switchover from the old index to the new one we rolled out. For the first 24 hours, sometimes the old one will show because of a caching issue. They should both be the same very soon.
More details in this Q+A: https://moz.com/community/q/august-3rd-mozscape-index-update-our-largest-index-but-nearly-a-monthly-late
-
RE: Convince me to stay! How should I best use SEOMoz tools.
Next time you're in Seattle, just drop me a line
-
RE: Subtle line of asking links for money/service/benefits
Hi Fabrizo - as Jake noted, this can cause penalties and problems nowadays, so I'd recommend against a direct offer of discounts or remuneration in exchange for links. I went ahead and updated my reply to that 8-year-old blog post, too.
-
RE: DA/PA Fluctuations: How to Interpret, Apply, & Understand These ML-Based Scores
Hi Donna - yes, that fluctuation should be much larger on average in the tail of the web (sites with DA 0-40) than in the middle or head. This makes sense because with a relative metric, all of the factors I describe above are going to be magnified in the tail, particularly because Google's rankings change so much there and because just a few links can have such a huge impact. For metrics-savvy clients, they should be best poised to understand that since DA/PA are exponential, a few links here or there and a few valuation shifts on those links can have big swings in the 10-40 point ranges of DA/PA, whereas in the 50/60+ ranges, small shifts in link discovery or in link valuation (from us or Google) won't have as much change.
As far as a ceiling - no, we don't have a recommendation there. The idea is that as DA/PA fluctuate, especially as they get more accurate in predicting rankings (correlations & coverage), the fluctuations are generally happening because Google's changing and we're getting closer to tracking how and where (with exceptions I noted above around issues with our crawl/indexing). My biggest recommendation is to keep track of similar-sized competitors (and larger/smaller ones) so you've got a set of benchmarks for comparison.
Thanks for the note and apologies for the frustrations this causes.
-
RE: Loss of Google AdWords API
We could probably do something that allows folks to upload their own AdWords data, but the issue, unfortunately, isn't related to account stuff, but to keyword research and search volume numbers (which are on-demand). For that, we'll need to find a different solution.
Way to be creative though!
-
RE: Another tool like opensitexplorer?
I suspect you heard that Yahoo! Site Explorer would close its doors (which it did). Open Site Explorer's definitely sticking around!
-
Just Discovered Links Are Down for ~2-3 Days
UPDATE 12/01/15: This issue is now resolved and Just Discovered Link data is back.
Sadly, I've got bad news. Due to an error we made (literally a typo on a curl command), Just Discovered Links in Open Site Explorer will be unavailable for 2-3 days. The data was actually lost, but we're re-indexing all those links now and they should be back in working order by Monday.
NOTE: this only affects link counts and links listed in the "Just Discovered" sections of OSE. No links in the main index nor any metrics (PA/DA/etc) are affected.
Our sincere apologies - we're going to build mechanisms to prevent this for the future.
-
RE: Can I add another user to my SEOMoz Pro campaign?
Good news - we started working on it this quarter with the intention of having it launched by end of March. Bad news - a problem came up - we've been having serious troubles with billing and credit cards and found a ton of bugs, errors, and not applying best practices in our billing system. As a result, all the effort for this team was diverted on Sarah's instruction (I think she made the right call - we can't survive as a company if we don't get billing right). Sadly, this means the multi-seat project is now delayed until Q2. I'm hopeful they'll get it done by June.
-
RE: Houston Company Needs Help (Will Our SEO Work Be Destroyed While Site is Down?, Can Anything be Done?)
You might want to check out https://moz.com/blog/how-to-handle-downtime-during-site-maintenance
Depending on the length of the expected downtime, a 503 is probably the way to go (and will tell Google it's a temporary issue).
Congrats on the rankings gains BTW! Nice work!
-
RE: The New Link Explorer (which will replace Open Site Explorer) is Now in Beta
Thanks Ajaz - outside of this beta product, all the other spots in Moz Pro will still use the DA scores (and link counts) from OSE (including Mozbar). Once we launch publicly, that'll change to the new one though.
-
RE: Detailed Revisions of Articles coexisting with Automated Description Articles
Hi Antonio - a lot of sites, particularly in the e-commerce field, face precisely this issue. What I've seen be most effective is what Amazon, BestBuy and many others do, which is to create a single page for any product and include editorial/user reviews and more detailed information when it's available and when it's not, leave that area open for future additions of content. This way, you have a single version of any given page and you create a positive association with the crawlers and humans that some/much/most of your content/products will eventually get a good, rich description.
You can also use Saibose's suggestion in combination if you'd prefer having this content in separate, embedded "tabs" on the page that all resolve to the same URL. Check out a code sample and example of this in action here - http://dhtmlkitchen.com/scripts/tabs/tutorial/navigation.jsp
Best of luck!
Rand -
RE: SEO and Squarespace? Is this Really an Option?
Yeah - I'd say the inability to customize the titles and meta descriptions is a dealbreaker by itself. That said, it shouldn't be impacting indexing -- could be that Google's crawled those pages but determined they don't have enough unique content, enough link equity, enough positive user/usage signals, or a combination of these, to keep in the main index.
At this point, though, I would probably consider migrating. Wordpress is still my first choice for the customization abilities, but Hubspot's a good one, too.
-
RE: Why should your title and H1 tag be different?
I don't know... There's a surprising number of people who've reported hearing Matt say things. Yet, somehow, whenever there's video of him, he magically says next to nothing. I'd be skeptical at best.
-
RE: Should UK websites expect any benefit from using Google+?
Hi Mick - I'd say that if you're not finding value from Google+, I wouldn't worry about it. Google is investing less in it and unless you're in a community/niche where G+ is heavily used (travel, photography, online marketing are a few), probably best to put your effort elsewhere. The only real SEO benefit to G+ left is the personalization of SERPs, but that will only work if you have a large group following you who can be biased in their SERPs by their connection with you. Don't think it's geographically specific, but it's certainly niche-specific.
Best of luck!
-
RE: Disavow Links & Paid Link Removal (discussion)
I'm in agreement with William. If you proactively submit the disavow file, you should be protected. I'd also think about sending a note via Webmaster Tools to let Google know about the network and that this person is extorting you/your site by forcing payment to remove links. That may help others whom Google might penalize for this in the future if they refuse to pay (and paying it forward like that is a great way to serve the web community and discourage future spam extortionists).
-
RE: Is this Directory Guide by SEOmoz still accurate?
Since the full-fledged version is very significant, I've asked the team to come up with a mimimum solution; possibly just another list without comment/submissions functionality. I hope to have that up very soon (possibly even just as an Excel download or Google Docs accessible with a password.
-
RE: Merging Niche Site
Agree with Patrick - those resources should help and in general, I'd bias to merging the sites. You may even lose out short term, but long term, you'll almost certainly be better off focusing your efforts and energy, and earning the branding benefits of a single site and name.
-
RE: Spammy backlinks are working?!
Hey Charles - I think some of the first points made in the comments are essential reading on top of the article. Specifically, the first comment notes that until/unless the poster shows the keywords targeted, it's very tough to trust the data or know if the inputs noted are causal. It's also impossible to replicate or investigate. The reply from the author points out that he hasn't tried replicating in 2015, and isn't confident it would work again.
I'd say both of those points make the techniques shown at least somewhat less trustworthy.
If you want to try spamming Google, my advice would be to experiment with sites you're willing to lose entirely and never recover. Also be ready for very spiky and short-term-focused revenue. You could make a lot of money for a few months, then make nothing for a long time (as many spammers have found).
Also recommended reading: http://inbound.org/post/view/confessions-of-a-google-spammer - a good piece from a former spammer on the techniques and process used, and on Google's evolution around webspam and manipulative links.
-
RE: Google's 'related:' operator
Hi Thom - unfortunately, I don't have much to give you, but I can say that this isn't necessarily a problem. Tons of sites that do really well in search results, have popular brands, and are legitimate don't have "Related:" results. Moz itself is one, but we've seen others. There may be elements about Google having issues understanding your content or not seeing many powerful links from sources to your site that also link to other places, but we don't know for sure.
Long story short - if you're not seeing other issues with your site in Google, I wouldn't worry about it.
-
RE: Is there a good website builder that can gain links?
Hi Scott - some website builders that might be worth investigation (and will almost definitely be more SEO-friendly than the experience you've described) include:
- http://www.wix.com/
- http://squarespace.com/
- https://www.drupal.org/
- https://wordpress.org/
- https://pagely.com/
I'm partial to Wordpress (and there's lots of good hosting options) because of its flexibility, but there's plenty of benefits to other platforms as well.
-
RE: Keyword difficulty - benchmarks
Hi Walter - I can help a little on that.
Basically, I'd urge you to think of thee percentages as good relative numbers that are most directly connected to Domain Authority and Page Authority scores. For example, say there's a keyword with a difficulty of 30%. You could roughly expect that to compete, you'd need to be in the Domain Authority and Page Authority 30(ish) range to get onto page 1 (and you'd need to be higher to be at the top). That's most certainly not all there is to it, but it's a reasonable rough way to think about it.
Hours needed is a metric I don't think is a good way to think about Keyword Difficulty. Some marketers and some sites may require 5 hours of work to create and market a piece of content that could compete for the same difficulty that it would take months or even years for another site to work on.
http://moz.com/academy/competition-keyword-analysis is a great resource you should check out and dives into detail on how to use the tool (including a bit about the percentages).
The way I always use keyword difficulty is to help identify the relative challenge in ranking for a term. My goal is to target terms with high search volume and low difficulty first and to recognize that really hard keywords are going to take much more outstanding content (and outstanding marketing to earn links). Some low difficulty terms (e.g. anything below 25 or 30) might just take good keyword targeting and hitting publish (because I have a pretty strong blog already). High difficulty terms (anything over 60 or 65) are usually going to take serious concerted effort and a lot of links (with some nice anchor text) to rank well.
-
RE: September's Mozscape Update Broke; We're Building a New Index
Two potential solutions for you - 1) watch "Just Discovered Links" in Open Site Explorer - that tab will still be showing all the links we find, just without the metrics. And 2) Check out Fresh Web Explorer - it will only show you links from blogs, news sites, and other things that have feeds, but it's one of the sources I pay attention to most, and you can set up good alerts, too.
-
Verified News Sources: A New Feature in Fresh Web Explorer Launched Today!
Hey gang - I wanted to mention a small update we've made to Fresh Web Explorer (I was the architect on this one). It enables you to filter your searches and/or your Alerts to show only those sources/domains we've observed in Google News. E.g. here's a search for GigaOm with news filtering on (180 results in the last 7 days) vs. the same search with news filtering off (1,811 results).
It's mostly useful for folks who are getting overwhelmed by the quantity of mentions they see in FWE and Alerts (I was experiencing that myself with mentions for my name and Moz) and just want to filter to the most important/noteworthy stuff. It's not perfect - Google News does have some weird domains that show in there occasionally - but it's definitely a much better filter if you have a bigger brand or term/phrase you're watching.
Any questions or feedback, please let us know!
-
RE: 10/14 Mozscape Index Update Details
Hi Donna - you are most certainly not alone in your frustration. I would call my own feelings bordering on desperation. I'm frustrated, angry, nervous, guilty, and overwhelmed with a sense of powerlessness. It seems that every time we think we've identified a problem at the root of our Mozscape issues, things just get worse and new problems we never imagined arise.
On the padding issue, I have good news and depressing news. The good news is that we pad every estimate by nearly 2X. In a normal, problem-free index cycle, we can get it done in 12-14 days.... And yet, we never estimate less than 30-31 days for an index release. In the early part of this year, you might recall that we had a number of indices released back to back in that 2-3 week window. Then things took a turn for the worse and we've been struggling ever since.
I want to be honest - my belief is that we are going to get better, but the evidence of the last 6 months is against me. I want to believe my team and I know they are trying hard and doing everything they can to get this fixed. However, I think it's wise to have skepticism given the trajectory of the recent past.
Hope that's helpful and thank you for the comment.
-
RE: Keyword Explorer is Now Live; Ask Me Anything About It!
Hi Gyorgy - you can use the old KW Difficulty tool, but the search volumes there aren't accurate for non-US either, and the difficulty scores are less accurate, too. I'd suggest using KW Explorer if you want all the other scores (those are accurate) and AdWords KW Planner in the meantime.
We're prioritizing UK, Canada, and Australia volume data very soon - hopefully will be in the product in the next 60 days.
-
RE: Spring is here and so is our May Index Update!
Thanks Ian. I wanted to add on that we've got a little bit of strange issue with some domains in this index that may affect some sorting and link lists.
Basic story: We blacklist domains that have more than 10,000 subdomains and have other features that we've observed make them, almost always, webspam. Since they can adversely impact metrics and scores, we just don't include them in the index. The last 6 months, this process has gone really well, but this index, we hit a bug, and accidentally excluded ~750 domains that aren't all spam, including big ones like Etsy.com, Wikia.com, Blogspot.com, Quora.com, and others.
Initially, folks noticed because the PA/DA scores for these sites were all 1. That part has been fixed, and the metrics should now be more accurate. But, we don't have the link lists for these sites, and we don't have PA scores for the interior pages on these domains. As such, if you're sorting your link lists or if you're researching links that point to a page on Etsy.com or Blogspot.com or Wikia.com or a few others, you'll see no data. This will be fixed in our next index, but we couldn't fix it in this one. The bug was caught, but caught too late, and thus the next 28(ish) days will have some of these sorting and link list issues on these ~750 domains (of which, I estimate, only ~150 or so are actually legit sites - most of the others are, indeed, spam).
If you've got any additional questions I can answer about this, please let me know. Apologies for the error from me and from the team - we'll make sure to get this right in the future, and we've set up new kinds of tests to prevent any similar issues from going unnoticed.
-
RE: How non-US Moz customers will use Keyword Explorer after the Keyword Difficulty tool is retired?
Hi Gyorgy - I think you asked this same question in the reply to another thread, but I'll paste my answer to that below
Some important things to keep in mind about volume in KWE:
-
It uses United States volume only right now (if you search by default in AdWords, you may be getting global search volume unless you choose "US")
-
It's modifying that data with actual clickstream numbers to give greater accuracy
-
It normalizes trend volume, so if Google has a big spike or dip, they'll show that one number, whereas KWE tries to estimate the monthly average data.
-
-
RE: Did Google's Farmer Update Positively/Negatively Affect Your Search Traffic?
Kris - awesome that you "rand" some tests. I like to do that myself
One thing we've been noticing is that sites with very aggressive ads (AdSense, overlays, display, etc.) seem unusually hard hit, while content farms that are less agro on that front weren't. Maybe a user/usage data thing?
-
RE: How cool is this Q&A Forum!
It's funny - I'm presenting today to the entire staff at SEOmoz for our quarterly "all hands" meeting, and a screenshot of this thread is exactly what I needed
The barrier to entry thing is interesting - at first we worried it would mean that participation might not be as heavy as others, but the barrier to entry clearly carried value. Will make sure to give props to the team members who worked on this.
And just FYI - we plan lots more improvements in the future, so if you've got suggestions, don't hesitate to make 'em!
EGOL - I've been trying to lure you to Moz for years. Who knew this is all it took
-
RE: Thoughts on Net Neutrality Repeal & Digital Marketing
Hey Patrick - good question, because I think this will have a dramatic, mostly negative, definitely industry-shaking-up effect on web access and how web marketing will have to be done. I covered my thoughts here: https://moz.com/blog/net-neutrality-seo-whiteboard-friday
-
RE: Direct traffic is up 2100% (due to a bot/crawler I believe)
It's weird that the bot is accepting cookies, but with a bounce rate that high, I agree it's probably something automated (though it could be people who were looking for something else or were directed there by an email or an app accidentally). You can look through your logs to see IP addresses and then do as Atakala says and block the traffic if you're worried about bandwidth. You can also just filter it out in GA by excluding US traffic (if your'e worried about analytics being messed up).
-
RE: Correlation between PageRank and MozRank
We've found this to be very close as well! I believe the correlation coefficient was around 0.85 last time we ran it.
In terms of the value of mozRank - our intent was always to mimic Google's PageRank algo, not to build a metric that best represents how a page/site might rank. You're totally spot on to say PA/DA are a good choice for that (and we have some tests running to make them even better in the near future).
BTW - I should mention here since it's relevant that Linkscape's estimated index update of July 11 is now moved back to July 20th due to some processing errors on the Amazon cluster (and some mistakes on our end, too). Sorry for the delay, but on the plus side, we do expect it to be quite a high quality index update (deeper crawls and better domain diversity, too).