Drastic Monthly Fluctuations in Page Link Metrics
-
We have experienced very drastic changes in our root domain numbers and as a result, have seen an odd impact on our DA. The data does not seem at all reliable at all. We went one month with over 50 recorded root domains and the next month that dropped over 75%.
It doesn't make sense to be paying for a monthly pro account when the data is so clearly unreliable. What is going on? Looking for a good answer before closing our account!
-
Hi Nima
Google will always show more links than our crawler as they are a much larger company and have multiple seeds to crawl more pages. You should use multiple tools to obtain a full link profile which is something I've seen recommended many times in our community. The 68 links we found mean those links meet the criteria we are measuring to discover them. This also means the 158 other links may not be considered as high ranking sites or your links are too deep for us to crawl or they have been crawled and are just waiting to appear in a future index update. No single index will be the same as each company crawls differently.
As I mentioned earlier:
- We grab the most recent index
- We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
- We start crawling from the top down until we've crawled 90,000,000,000 pages (which is about 35% the amount in Google's index).
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index. Google does not crawl this way. This and our metrics are proprietary to Moz so it isn't that our data is unreliable or is being crawled incorrectly.
The domain authority is effected by a lot of things. It is hard to pin point it without being a SEO consultant or the specific web designer on your website. Domain and Page Authority scores are both calculated using Moz's Ranking Models work. In essence, we take a lot of rankings data from the search engines (by running queries) and then try to build a predictive scoring system using our own on-page analyses and Mozscape link data to construct an algorithm that will effectively reproduce the search engines' results. Our current accuracy hovers in the 70% range, but over time, we expect to improve.
Once we have a ranking model (which we internally call "uber"), we can create scores that best approximate the combinations of all our page-specific link metrics or domain-specific link metrics (removing the keyword-specific features like anchor text, on-page keyword usage, etc). These scores represent the model's query-independent or non-keyword-based ranking inputs.
In simple terms, Domain Authority is our best prediction about how content would perform in search engine rankings on one site vs. another. Page Authority answers the same question for an individual page. Both are amalgamations of all the link metrics (number of links, linking root domains, mozRank, mozTrust, etc.) we have into a single, predictive score.
It's important to note that both Domain Authority and Page Authority are on a 100-point, logarithmic scale. Thus, it's much more difficult to grow your score from 70 to 80 than it would be to grow from 20 to 30.
Here's some places to really delve into what is going on:
http://moz.com/blog/whiteboard-friday-domain-trust-authority
http://moz.com/blog/googles-algorithm-pretty-charts-math-stuff
http://moz.com/blog/whiteboard-friday-domain-authority-page-authority-metricsHere are some good resources to help you take a look at the factors.
http://moz.com/blog/whiteboard-friday-domain-authority-page-authority-metrics
http://apiwiki.seomoz.org/w/page/20902104/Domain Authority
http://moz.com/blog/whiteboard-friday-domain-trust-authorityI would recommend starting a new thread on the forum to seek advice from other marketers as I am only able to explain how our tools work from the technical side.
Hope this helps!
-
BTW on our webmaster tools we have 226 linking root domains. on moz we have 68.
our consistent drop in DA seems to be a factor of Moz not crawling properly. I remember back in the day the discrepancy between WMT and moz was less.
Can you please shed some light? we use moz primarily as a performance measurement software (if it can be called that), but if the data is unreliable that will make it difficult for us to do that.
I am still looking forward to your trouble shooting advice.
Kind regards,
Nima
-
thank you very much, but the answer is so technical that it does not allow me to understand much. The interesting thing from our perspective is that we had barely any links from above DA sites 70 before and we had 31 DA and now we have a lot of links from DA sites above 70 and we actually dropped.
When we posted this question, suddenly within a day many of our links appeared on your tracking system. This may indicate something happening on your end as a result of us posting this question. Many links are still missing, inlcuding one link from inc.com, which is odd.
I have also seen many sporadic changes at times in the way your metrics evaluates us. It makes it difficult to use your data for anything.
two questions that arise from the minimal amount i understood:
1- inc.com is a very high DA site, why is this link not indexed yet? http://www.inc.com/jessica-stillman/5-weird-ways-to-de-stress.html
2- why did a large number of our links get detected right after we posted this question?
thank you very much in advance,
Nima
-
Hello!
Sorry to hear your metrics have fallen. While I can't provide any consulting advice I can clarify how we index sites for Mozscape.
- We grab the most recent index.
- We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
- We start crawling from the top down until we've crawled 90,000,000,000 pages (which is about 35% the amount in Google's index).
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index.
We update our Mozscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always collecting data. When we need to put the index together, we grab all the data they have collected and start processing which can take up to 3 weeks to determine which of those links are the most important. You can see our most recently updated schedule here: http://moz.com/products/api/updates
Mozscape focuses on a breadth-first approach. Therefore we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. However, deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates before we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Mozscape must be linked-to by other documents on the web or our index will not include them.
The index is based on freshness so domains with higher page/domain authority tend to be re-indexed each month but lower ranked pages can fall off or not be re-crawled if there are not consistently new links being built on high quality sites. We never re-crawl the same sites each index unless strong links take us to your site. It starts over completely from scratch.
Hope this helps and I will set this as a discussion so other community members can offer any advice.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ranking keyword and links
I have Created the website 3 months ago related to APk Games. And updating content daily and linking but the linked results in Moz are not showing Properly. my site have some ranking keywords on google at 1st but moz showing 0 ranking keywords. my question is how many time will take moz to show actual results. Thanks
Link Explorer | | khannnnnndnnnnnnnnnnss0 -
DA score Decreased but Linking Domains Improved
Hi, I have observed that even if number of linking domains increased but DA value can still decreased. Can anyone Please let me know how to improve DA score. Focusing on gaining quality backlinks is already in process and number of linking domains also increased over the last 2 months but DA score got down. Website - https://valueofficefurniture.com.au/ Awaiting responses!
Link Explorer | | insidewebanalytics
Thanks. pt3y1n pt3ycl0 -
Moz Pro: Linking RDs to Page much lower than Google Search Console
I'm trying to use the Analyze Keyword tool in Moz Pro, and in the SERP Analysis table, my page has a PA of one, and zero root domains linking to it. If I look at the page in Google Search console, it says I have 229 root domains linking to the page from well known domains like github.com, meetup.com, stackoverflow.com, etc. This particular keyword has been tracked in Moz for the last 6 months, but I just noticed that it was extremely low. I am relatively new to Moz, so forgive me if I sound confused, but can someone explain to me how the numbers can be so low?
Link Explorer | | jakebellacera0 -
Backlinks not showing on Open Site Explorer's Just-Discovered Links
Hi guys! My name is Edgar. I've recently been adding a bunch of backlinks to the site i work for, and most of them have a very high DA (DA > 80. Wohoo!). Nevertheless i'm often feel a bit frustrated when i check my site with the OSE, because just a few part of these links are showed in the just-discovered section, although i created them few days ago. So, i would like to know why is this happening, how long it take for my links to get crawled, and when can i stop getting frustrated on this? Than you very much! Edgar.
Link Explorer | | PCBU0 -
Top Pages Report References URLs from a Different Domain
I am running a Top Pages Report using Open Site Explorer and it's returning that the top pages for examplesiteA.org are examplesiteB.com. Important to note that examplesiteA.org was a dormant domain for a few years. We recently purchased it and are in the process of reviving it. Has anyone experienced this before? Would this suggest that canonical link elements were put in place prior to it going dormant and that the top pages for this site are now assumed to be on examplesiteB.com?
Link Explorer | | seoaustin0 -
DA/PA Fluctuations: How to Interpret, Apply, & Understand These ML-Based Scores
Howdy folks, Every time we do an index update here at Moz, we get a tremendous number of questions about Domain Authority (DA) and Page Authority (PA) scores fluctuating. Typically, each index (which release approximately monthly), many billions of sites will see their scores go up, while others will go down. If your score has gone up or down, there are many potential influencing factors: You've earned relatively more or less links over the course of the last 30-90 days.
Link Explorer | | randfish
Remember that, because Mozscape indices take 3-4 weeks to process, the data collected in an index is between ~21-90 days old. Even on the day of release, the newest link data you'll see was crawled ~21 days ago, and can go as far back as 90 days (the oldest crawlsets we include in processing). If you've done very recent link growth (or shrinkage) that won't be seen by our index until we've crawled and processed the next index. You've earned more links, but the highest authority sites have grown their link profile even more
Since Domain and Page Authority are on a 100-page scale, the very top of that represents the most link-rich sites and pages, and nearly every index, it's harder and harder to get these high scores and sites, on average, that aren't growing their link profiles substantively will see PA/DA drops. This is because of the scaling process - if Facebook.com (currently with a DA of 100) grows its link profile massively, that becomes the new DA 100, and it will be harder for other sites that aren't growing quality links as fast to get from 99 to 100 or even from 89 to 90. This is true across the scale of DA/PA, and makes it critical to measure a site's DA and a page's PA against the competition, not just trended against itself. You could earn loads of great links, and still see a DA drop due to these scaling types of features. Always compare against similar sites and pages to get the best sense of relative performance, since DA/PA are relative, not absolute scores. The links you've earned are from places that we haven't seen correlate well with higher Google rankings
PA/DA are created using a machine-learning algorithm whose training set is search results in Google. Over time, as Google gets pickier about which types of links it counts, and as Mozscape picks up on those changes, PA/DA scores will change to reflect it. Thus, lots of low quality links or links from domains that don't seem to influence Google's rankings are likely to not have a positive effect on PA/DA. On the flip side, you could do no link growth whatsoever and see rising PA/DA scores if the links from the sites/pages you already have appear to be growing in importance in influencing Google's rankings. We've done a better or worse job crawling sites/pages that have links to you (or don't)
Moz is constantly working to improve the shape of our index - choosing which pages to crawl and which to ignore. Our goal is to build the most "Google-shaped" index we can, representative of what Google keeps in their main index and counts as valuable/important links that influence rankings. We make tweaks aimed at this goal each index cycle, but not always perfectly (you can see that in 2015, we crawled a ton more domains, but found that many of those were, in fact, low quality and not valuable, thus we stopped). Moz's crawlers can crawl the web extremely fast and efficiently, but our processing time prevents us from building as large an index as we'd like and as large as our competitors (you will see more links represented in both Ahrefs and Majestic, two competitors to Mozscape that I recommend). Moz calculates valuable metrics that these others do not (like PA/DA, MozRank, MozTrust, Spam Score, etc), but these metrics require hundreds of hours of processing and that time scales linearly with the size of the index, which means we have to stay smaller in order to calculate them. Long term, we are building a new indexing system that can process in real time and scale much larger, but this is a massive undertaking and is still a long time away. In the meantime, as our crawl shape changes to imitate Google, we may miss links that point to a site or page, and/or overindex a section of the web that points to sites/pages, causing fluctuations in link metrics. If you'd like to insure that a URL will be crawled, you can visit that page with the Mozbar or search for it in OSE, and during the next index cycle (or, possibly 2 index cycles depending on where we are in the process), we'll crawl that page and include it. We've found this does not bias our index since these requests represent tiny fractions of a percent of the overall index (<0.1% in total). My strongest suggestion if you ever have the concern/question "Why did my PA/DA drop?!" is to always compare against a set of competing sites/pages. If most of your competitors fell as well, it's more likely related to relative scaling or crawl biasing issues, not to anything you've done. Remember that DA/PA are relative metrics, not absolute! That means you can be improving links and rankings and STILL see a falling DA score, but, due to how DA is scaled, the score in aggregate may be better predictive of Google's rankings. You can also pay attention to our coverage of Google metrics, which we report with each index, and to our correlations with rankings metrics. If these fall, it means Mozscape has gotten less Google-shaped and less representative of what influences rankings. If they rise, it means Mozscape has gotten better. Obviously, our goal is to consistently improve, but we can't be sure that every variation we attempt will have universally positive impacts until we measure them. Thanks for reading through, and if you have any questions, please leave them for us below. I'll do my best to follow up quickly.13 -
Open Site Explorer - Just Discovered Inbound Links
Hello Moz Forum, Thanks in advance for any help. My website was recently featured in a bunch of high DA journals and magazines. I don't see the inbound links on the open site explorer, but I do see them in recently discovered. My website's Domain Authority hasn't changed a wink. How long does it take, typically to see the recently discovered links under the inbound links portion? Thanks! j1pqVRt
Link Explorer | | FutureSolutionsMedia0 -
Why MoZ Bar gives different PA and DA to my homepage? Surly they should have the same metrics?
Hi guys, is it normal that when looking at my homepage with the mozbar i get the same DA and PA? It seems that it should provide the same results. Thankss
Link Explorer | | artdivision0