Ose question
-
Hey
why do my websites www.gardeners.scot & www.landscaping.scot not have any links showing in ose?
thanks
-
Hey there!
It's less an issue of a bug in our system and more a factor of how the index was formed this time around. We expect data to normalize and stabilize after our next index update.
-
hi everyone, thanks for the advice, much appreciated
how long do you think it will take for your tech guys to resolve this issue?
-
Another vote for Screaming Frog. Fantastic SEO tool!
-
https://www.screamingfrog.co.uk/seo-spider/ offers a free service (okay for a small site, below 500 links).
That'll cover you as your gardeners.scot site is well below that.
-
Google Search Console is a free one that lists links to your website. I know a lot of people use Ahrefs or Majestic which aren't free to use. Personally I don't have any experience with them.
-
hey
can you recommend another tool for link checking?
ta
-
Another update from the team! This is the most up-to-date list of TLDs that Open Site Explorer currently supports: https://raw.githubusercontent.com/publicsuffix/list/abde055b790f9577d02116da8c819838248a9258/public_suffix_list.dat
It looks like .scot doesn't appear in that list, so my original answer applies here - that TLD just isn't currently supported by our index. Sorry about that!
-
Well, thanks for the update, Tawny; we have clarity at least. There absolutely needs to be a definitive in-house list in place of this nigh-on useless third-party one. This is step two of your 'not showing up on OSE' troubleshooting list; if it had been valid we could have identified it as the problem in minutes, rather than hours.
-
I'm afraid we won't be offering refunds for this issue - Open Site Explorer is one of our free research tools, so there's no cost to get started using this info.
-
well that's disappointing
i guess you'll be offering a refund?
-
I do think that would be helpful especially given that your colleague Samantha said .cc isn't supported here, however, in the list .cc is listed above the line, right under .ca, so according to what you said it should be supported.
-
Back again with a bit more clarity - it turns out that we don't support every TLD in this list, just the ones that appear above the line // List of new gTLDs imported from https://newgtlds.icann.org/newgtlds.csv on 2016-11-29T01:06:51Z
It looks like .scot is actually below that line, so it's not supported by Open Site Explorer. Sorry about that! It appears that this list has been updated since we began using it as a reference. I will check with the team to see if we could put some work into coming up with a Moz-specific list of TLDs that are supported by the tools rather than relying upon a 3rd party resource.
I do apologize for not being able to provide answers that inspire confidence!
-
It doesn't fill you with confidence does it?
Either the list isn't a true reflection of supported TLDs (and they aren't supported - and never have been). or, potentially even more unsettling, they should be/were supported and something is broken within the API.
Neither scenario is good news, however.
I'll contact support and share their reply, here.
Update: See below.
-
You also suggested the same was true for someone recently asking about a .cc TLD - and someone else recently had the same issue with what I believe was a .kitchen TLD - if it is correct that you support all TLDs listed in that link, then all .cc and .kitchen domains should be supported. In fact that list seems to include all sorts of custom TLDs including .landrover, .lidl, .skype, .tkmaxx etc.
Is it possible that this list you reference is simply not correct?
-
hey
does this mean there is a bug in your system?
-
This issue with the .Scot domain appears to cover the whole TLD. I've tested a number of other .Scot domains and They all show No DA/PA and No Established Links. (However, Page Social Metrics do show).
The last one I tested was isleofjura.scot (https://moz.com/researchtools/ose/links?site=isleofjura.scot&filter=&source=external&target=domain&group=0&page=1&sort=page_authority&anchor_id=&anchor_type=&anchor_text=&from_site=)
If this is TLD wide one can't help but wonder if this issue is more widespread?
Could this be part of something bigger, namely the drop in DA/PA after the recent Mozscape API update?
I think this has the potential to be the tip of the iceberg
-
What the deuce?!?!
If .Scot IS supported. you'd expect the Scottish Parliament (http://www.parliament.scot) to show one or two inbound links?! As it stands, it doesn't show any:
Something is amiss, here.
I don't know if you've seen Braveheart, Tawny, but the Scots don't take kindly to being oppressed. If you don't want a lot of hairy, blue-faced fellows showing what is worn beneath their kilts, I'd give someone in the Moz tech department a nudge
-
Shoot - you're totally right - that TLD is supported by OSE.
In that case, it looks like we just don't have much linking information for your site in our index.
Open Site Explorer and the Link Analysis page of campaigns are both tied to our Mozscape index, which tends to update roughly once a month.
Just a few points on how we compile our index:
- We grab the most recent index.
- We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
- We start crawling from the top down until we've crawled ~130 billion URLs
The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available!
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons - including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This does bias our index slightly in favor of newer links - older links are only cached in the index for about 180 days, so unless the tools are prompted to return to those sites and re-crawl them, they may fall back out of our index after a few months.
You can see our most recently updated schedule here as well as some more technical metrics on our Mozscape API Updates page. You can also see when the last and next updates happened on the Open Site Explorer (OSE) homepage at any time.
Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.
The best thing to do if you want us to find backlinks to your site is to do just what you're already doing - search for those sites in Open Site Explorer. That should prompt our tools to visit your sites and crawl the web looking for links pointing back to them. Depending on where we are with processing the current index, you should start to see more backlink information after one or two index updates.
I know that's a ton of info, so if you've still got questions, feel free to shoot us a note at help@moz.com and we'll do our best to answer them for ya!
-
I'm very confused, Tawny. I checked that list (and referenced it myself, above) as this was one of my concerns. if you search the list for Scot, you'll find:
**// scot : 2014-01-23 Dot Scot Registry Limited scot** Which, unless I'm very much mistaken, would suggest you do (or should) support it - assuming this list gives a true reflection of TLDs you support. I'm quite happy to be mistaken but would be more concerned if the resource you're referencing isn't current or accurate. Perhaps an internally generated list of supported TLDs would be a better resource? **UPDATE:** Having checked this against the Scottish parliament's own: www.parliament.scot/ it appears that you're absolutely correct, Tawny, the .Scot isn't supported. However, there does need to be some clarification about supported TLDs. EDIT (for clarity): I checked www.parliament.scot in OSE and it shows no established links.
-
Hi there! Tawny from Moz's Help Team here.
I'm afraid we don't currently support custom Top Level Domains (TLD) such as .scot - this would explain why there's no data for this site in our Index and why Open Site Explorer isn't returning any data for your site.Unfortunately, I don't know of a good workaround for this - it's a technical limitation of the tool. I'll make a note to bring it up with our product team - I know we plan to expand the list of TLDs we support, but I'm not sure what kind of timeframe we can expect before new TLDs will be supported. You can find the current list of TLDs that we do support here: https://publicsuffix.org/list/effective_tld_names.dat
If you've got any more questions, feel free to shoot us a note at help@moz.com and we'll do our best to answer them for ya!
-
Hi,
Well, there is some good news: A search using: site:www.landscaping.scot and site:www.gardeners.scot shows both are indexing and I can see that there are inbound links to the site.
It could simply be that OSE hasn't crawled the site, yet: "Keep in mind that, while large, our index doesn't cover the entire web. If you have a smaller or new site, it's possible we haven't crawled it yet. When we discover a new link, it’s added to our queue, but it can take 1-2 index updates before we actually crawl the page." However, it does appear to be getting on for a year old, so, unless your inbound links are from sites that Moz hasn't crawled, you'd expect to be showing by now - unless there's another issue...
.Scot is a recognised TLD within Moz:
https://publicsuffix.org/list/effective_tld_names.dat so this shouldn't be a problem.UPDATE: See my reply in response to Tawny, below:
Have you recently changed the robots.txt file, perhaps?
Looking at: http://www.landscaping.scot/robots.txt I note that Sitemaps is showing at the very top of robots.txt when I would expect to see it at the bottom (See 'Sitemap Parameters', here: https://moz.com/learn/seo/robotstxt). I'm not sure if this is enough to trip Roger up, but I'd certainly start my investigations there, as an incorrectly configured robots.txt can cause chaos.
The fact that Google is seeing the site is a good sign, obviously.
Let us know you have you made any other structural changes recently (i.e within the last month or so).
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
A few questions regarding Moz tools + E-commerce strategy
Hi everyone 🙂 I'm currently in the midst of optimizing a Scandinavian E-commerce site. I have a few questions, that hopefully someone will be able to help me get answered. Firstly, GoogleBots should be able to recognize "ø" as "oe", "æ" as "ae" and "å" as "aa" in the URL title. I've noticed that Moz' On-page grader does not support this unfortunately - has something changed or do Scandinavians just receive a little less love than the English? Secondly, how does one avoid keyword stuffing on E-commerce sites? The products that are displayed in category pages all make use of the same keyword that is targeted for that category. As such, some pages have 40+ mentions of the keyword, although in reality there are less than 15 (the rest being in the product names). Any tips or tricks on how to get this optimized or does Google simply recognize the site as an E-commerce site and somewhat ignores keyword stuffing - as long as the website has sufficient content? Thirdly, has something happened to Moz' Open Site Explorer? It seems like something has changed and when I checked for backlinks for the site today, only 3 was found. I know for a fact that many many more exist (which other tools also confirm when they scrape the site). Looking forward to hearing from all of you! Best, Mark
Link Explorer | | osn0 -
OSE Says no Contact Information
For quite some time the Open Site Exporer has been labeling one of my sites has having "no contact info", the site has the phone number all over the website as well as the full business address in the footer. It's been like this for some time. Does the OSE make mistakes or am I missing something? The contact information is fully indexable, I ran a spider simulator and it was able to pick up the phone & address.
Link Explorer | | OliverNeely0 -
Having problems with OSE
I am getting error messages from OSE - it just keeps saying 'loading data' and then after a while 'there was a problem getting your data'. Any ideas what wrong, I really need this for a meeting this afternoon in 1.5 hours. Thanks Andy
Link Explorer | | Andy-Halliday0 -
Moz cannot crawl domain. Also OSE does not work properly on this specific domain?
Hi all, Moz cannot crawl the domein http://www.hoesjescases.nl.
Link Explorer | | Guapa_zwolle
When I open the crawl report I only see one line: <colgroup><col width="229"><col width="287"><col width="420"><col width="370"><col width="141"></colgroup>
| URL | Time Crawled | Title Tag | Meta Description | HTTP Status Code |
| http://www.hoesjescases.nl | 2015-10-05T12:20:48Z | 404 : Received 404 (Not Found) error response for page. | Error attempting to request page; see title for details. | 404 | Also when running OSE on this domain, Moz only can find 4 root domains while Majestic can find 91 domains. Google seems not to have any problems. What can be the problem for MOZ? Greetings!0 -
What is the difference between - This page v this subdomain v this root domain in OSE
What is the difference between - This page v this subdomain v this root domain in OSE, while checking website backlinks See inbound links to the page, subdomain, or root domain you've entered and analyze the linking pages.
Link Explorer | | Green.landon0 -
OSE Advanced Reports - behaving inconsistently
Yesterday, there was an error message on the Advanced Reports tab in OSE: "Oops! It looks like our system is temporarily down. Please check back soon." That cleared up after a couple of hours. After that, I ran an Advanced Report that I run on occasion, and it only returned 3500 links, whereas it normally returns around 23K. Since then, I've run the same report 9 times in total over the course of about 1 day, and each time the number of links keeps increasing. 3500, 10150, 21550, 25400, 28650, 35450, 37400, 38200, 50375. I'm wondering what is happening? I want to do some evaluating of my backlinks, however I'd prefer to work on a "complete" list of links and the number of links keeps going up, when generating reports using the same criteria. When can I expect Advanced Reports to be "stable" where it's producing roughly the same number of links? Also, I opened up a support ticket on this with Moz.com over 24 hours ago, and haven't received any response. I've updated the ticket 2 times now, but no response. Disappointing they are not responding to support tickets. Thanks.
Link Explorer | | azonetwork1 -
Inbound links export report link redirect to OSE
Hi there, I have received email notification that my new inbound links CSV report is ready to export and download. I follow the OSE link but I am redirected straight back to the main page of OSE not the download. Is there a problem? Are the access problems due to the Moz.com and Research Tools Outage. reported on Moz Health? David
Link Explorer | | David-E-Carey0 -
OSE Says I have 3,115 total links but I can only export 660 via advanced reports
Hi Everyone! My site (http://carwow.co.uk) has 3,115 total links according to OSE, I need to export them all but advanced reports export will only give me 660. Please advise! Thanks, James
Link Explorer | | JamesPursey0