Original content, widely quoted - yet ignored by Google
-
Our website is https://greatfire.org. We are a non-profit working to bring transparency to online censorship in China. By helping us resolve this problem you are helping us in the cause of internet freedom.
If you search for "great firewall" or "great firewall of china", would you be interested in finding a database of what websites and searches are blocked by this Great Firewall of China? We have been running a non-profit project with this objective for almost a year and in so doing have created the biggest and most updated database of online censorship in China. Yet, to this date, you cannot find it in Google by searching for any relevant keywords.
A similar website, www.greatfirewallofchina.org, is listed as #3 when searching for "great firewall". Our website provides a more accurate testing tool, as well as historic data. Regardless of whether our service is better, we believe we should at least be included in the top 10.
We have been testing out an Adwords campaign to see whether our website is of interest to users using these keywords. For example, users searching for "great firewall of china" end up browsing on average 2.62 pages and spending 03:18 minutes on the website. This suggests to us that our website is of interest to users searching for these keywords.
Do you have any idea what the problem could be that is grave enough to not even include us in the top 100 for these keywords?
We have recently posted this same question on the Google Webmaster Central but did not get a satisfactory answer: http://www.google.com/support/forum/p/Webmasters/thread?tid=5c14a7e16c07cbb7&hl=en&fid=5c14a7e16c07cbb70004b5f1d985e70e
-
Thanks very much for your reply Jerod!
Google Webmaster Tools is set up and working. Some info:
-
No detected malware
-
1 crawl error (I think this must have been temporary. Only reported once, and this url is not in the robots.txt now):
- http://greatfire.org/url/190838
- URL restricted by robots.txt
- Dec 10, 2011
-
Pages crawled per day, average: 1102
-
Time spent downloading a page (in milliseconds), average: 2116
The robots.txt is mostly the standard one provided by Drupal. We've added "Disallow: /node/" because all interesting urls should have a more interesting alias than that. We'll look more into whether this can be the cause.
Anything else that you notice?
-
-
Hi, GreatFire-
We had a very similar problem with one of the sites we manage at http://www.miwaterstewardship.org/. The website is pretty good, the domain has dozens of super high-quality backlinks (including EDU and GOV links), but The Googles were being a real pain and not displaying the website in a SERP no matter what we did.
Ultimately, we think we found the solution in robots.txt. The entire site had been disallowed for quite a long time (at the client's request) while it was being built and updated. After we modified the robots.txt file, made sure Webmaster tools was up and running, pinged the site several times, etc. it was still being blocked in the SERPs. After two months or more of researching, trying fixes, and working on the issue, the site finally started being displayed. The only thing we can figure is that Google was "angry" (for all intents and purposes) at us for leaving the site blocked for so long.
No one at Google would come out and tell us that this was the case or even that it was a possibility. It's just our best guess at what happened.
I can see that greatwall.org also has a rather substantial robots.txt file in place. It looks like everything is in order in that file but it might still be causing some troubles.
Is Webmaster tools set up? Is the site being scanned and indexed properly?
You can read up on our conversation with SEOmoz users here if you're interested: http://www.seomoz.org/q/google-refuses-to-index-our-domain-any-suggestions
Good luck with this. I know how frustrating it can be!
Jerod
-
Hi GreatFire,
With regard to the homepage content - you really don't have much there for the search engines to get their teeth into. I would work on adding a few paragraphs of text explaining what your service does and what benefits it provides to your users.
I disagree that your blog should be viewed as only an extra to your website. It can be a great way to increase your keyword referral traffic, engage with your audience and get picked up by other sites.
Just because Wikipedia have written about your topic already doesn't mean you should't cover the subject in more detail - otherwise no one would have anything to write about!
As you have the knowledge on the subject, involved with it everyday, and have a website dedicated to it - you are the perfect candidate to start producing better content and become the 'hub' for all things related to the how China uses the internet.
Cheers
Andrew
-
Hi Andrew,
Thank you very much for your response. The two main differences you point out are very useful for us. We will keep working on links and social mentions.
One thing I am puzzled about though is the labeling of the site as "not having a lot of content". I feel this is misunderstanding the purpose of the website. The blog is only an extra. What we provide is a means to test whether any url is blocked or not in China, as well as download speed. For each url in our database, we provide a historic, calendar-view to help identify when a website was blocked or unblocked in the past.
So our website first and foremost offers a tool and a lot of non-text data. To me, expanding on the text content, while I understand the reasoning, sounds like recommending Google to place a long description of what a search engine is on their front page.
If you want to read the history of the Great Firewall of China, you can do it on Wikipedia. I don't see why we should explain it, when they do it better. On the other hand, if you want to know if website X is blocked or not in China, Wikipedia is not practical since it's only manually updated. Our data offers the latest status at all times.
Do you see what I mean? It would be great to hear what you think about this.
-
Hi GreatFire,
Your competitor has a much stronger site in the following two main areas:
- More backlinks (resulting in a higher PR)
- More social mentions
Focus on building more backlinks by researching your competitors domain with Open Site Explorer and MajesticSEO. Keep up your activity in your social circles, and also get going with Google+ if you haven't already.
You should also fix your title tag to include the target keyword at the start - not at the end. So it would read something like 'Great firewall of china - bringing transparency from greatfire.org'
Looking through your site you don't appear to have that much content (this was also mentioned in your Google Support thread) so I would focus on building out the content on the homepage and also further developing your blog. For example your 'Wukan Blocked only on Weibo' blog post is not really long enough to generate you much referral traffic. Larger authority articles of 1000+ words plus with richer content (link references, pictures, Google+ author/social connections) etc will help you far more.
Conduct the relevant keyword research for your blog posts in the same way you did with your root domain. This will keep your website niche focused and generating lots of similar 'china firewall' terms.
Hope that helps.
Cheers,
Andrew
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Rel="prev" / "next"
Hi guys, The tech department implemented rel="prev" and rel="next" on this website a long time ago.
Intermediate & Advanced SEO | | AdenaSEO
We also added a canonical tag to the 'own' page. We're talking about the following situation: https://bit.ly/2H3HpRD However we still see a situation where a lot of paginated pages are visible in the SERP.
Is this just a case of rel="prev" and "next" being directives to Google?
And in this specific case, Google deciding to not only show the 1st page in the SERP, but still show most of the paginated pages in the SERP? Please let me know, what you think. Regards,
Tom1 -
Pages excluded from Google's index due to "different canonicalization than user"
Hi MOZ community, A few weeks ago we noticed a complete collapse in traffic on some of our pages (7 out of around 150 blog posts in question). We were able to confirm that those pages disappeared for good from Google's index at the end of January '18, they were still findable via all other major search engines. Using Google's Search Console (previously Webmastertools) we found the unindexed URLs in the list of pages being excluded because "Google chose different canonical than user". Content-wise, the page that Google falsely determines as canonical instead has little to no similarity to the pages it thereby excludes from the index. False canonicalization About our setup: We are a SPA, delivering our pages pre-rendered, each with an (empty) rel=canonical tag in the HTTP header that's then dynamically filled with a self-referential link to the pages own URL via Javascript. This seemed and seems to work fine for 99% of our pages but happens to fail for one of our top performing ones (which is why the hassle 😉 ). What we tried so far: going through every step of this handy guide: https://moz.com/blog/panic-stations-how-to-handle-an-important-page-disappearing-from-google-case-study --> inconclusive (healthy pages, no penalties etc.) manually requesting re-indexation via Search Console --> immediately brought back some pages, others shortly re-appeared in the index then got kicked again for the aforementioned reasons checking other search engines --> pages are only gone from Google, can still be found via Bing, DuckDuckGo and other search engines Questions to you: How does the Googlebot operate with Javascript and does anybody know if their setup has changed in that respect around the end of January? Could you think of any other reason to cause the behavior described above? Eternally thankful for any help! ldWB9
Intermediate & Advanced SEO | | SvenRi1 -
Something happened within the last 2 weeks on our WordPress-hosted site that created "duplicates" by counting www.company.com/example and company.com/example (without the 'www.') as separate pages. Any idea what could have happened, and how to fix it?
Our website is running through WordPress. We've been running Moz for over a month now. Only recently, within the past 2 weeks, have we been alerted to over 100 duplicate pages. It appears something happened that created a duplicate of every single page on our site; "www.company.com/example" and "company.com/example." Again, according to our MOZ, this is a recent issue. I'm almost certain that prior to a couple of weeks ago, there existed both forms of the URL that directed to the same page without be counting as a duplicate. Thanks for you help!
Intermediate & Advanced SEO | | wzimmer0 -
Google ignoring Canonical and choosing its own
Hey Mozzers, We have several products that all have upto 6 different versions, they are the same product but in a different specification. As users search via these specifications (within our website) it is beneficial to keep all 6 products as different listings on the website. In google however it is not. So we kept all 6 listing but chose 1 to be the google landing page, the only different between them all is the technical specification + occasionally size. But 95% of the pages are the same. Let call the products A, B, C, D, E, F, we made all the canonicals point to C because this is out best selling version of the product. However, google has chosen E to rank instead. What is my best move here? Should i accept the page google has chosen and change the canonicals the point to that version or should I be stubborn and try to get google to change which version it ranks. As always many thanks.
Intermediate & Advanced SEO | | ATP0 -
Rotating content = Google Penalty?
Hi all. We have an ecommerce site which features various product sections. In each section you might have 60 products each displayed neatly in pages of 10. We recently added functionality, so that if a product is out of stock, it will automatically drop that product to the back of the list and bring another in stock one forward. We're just worried that Google will view the same information, repeatedly rotating on the first page of 10 products (the page that ranks) and think we're in some way trying to trick Google into thinking the content is fresh? Does anyone have a throw on this? Is it likely to penalise us? Thank you!!! Ben
Intermediate & Advanced SEO | | bnknowles10 -
Rel="canonical" and rel="alternate" both necessary?
We are fighting some duplicate content issues across multiple domains. We have a few magento stores that have different country codes. For example: domain.com and domain.ca, domain.com is the "main" domain. We have set up different rel="alternative codes like: The question is, do we need to add custom rel="canonical" tags to domain.ca that points to domain.com? For example for domain.ca/product.html to point to: Also how far does rel="canonical" follow? For example if we have:
Intermediate & Advanced SEO | | AlliedComputer
domain.ca/sub/product.html canonical to domain.com/sub/product.html
then,
domain.com/sub/product.html canonical to domain.com/product.html0 -
Google Places
If you rank on google places, I have noticed that you do not rank on the front page as well. I have a site that ranks on front page for it's keywords; however, because they are (1) on google places, they don't show up when someone is localized to that area. They show up on google places but not on front page. If you turn of localization, they are first in serps. How can I get around this? Two separate sites? One for Google+ (Places) and one for SERPS?
Intermediate & Advanced SEO | | JML11790 -
Fetch as GoogleBot "Unreachable Page"
Hi, We are suddenly having an error "Unreachable Page" when any page of our site is accessed as Googlebot from webmaster tools. There are no DNS errors shown in "Crawl Errors". We have two web servers named web1 and web2 which are controlled by a software load balancer HAProxy. The same network configuration has been working for over a year now and never had any GoogleBot errors before 21st of this month. We tried to check if there could be any error in sitemap, .htaccess or robots.txt by excluding the loadbalancer and pointing DNS to web1 and web2 directly and googlebot was able to access the pages properly and there was no error. But when loadbalancer was made active again by pointing the DNS to it, the "unreachable page" started appearing again. This very same configuration has been working properly for over a year till 21st of this month. Website is properly accessible from browser and there are no DNS errors either as shown by "Crawl Errors". Can you guide me about how to diagnose the issue. I've tried all sorts of combinations, even removed the firewall but no success. Is there any way to get more details about error instead of just "Unreachable Page" error ? Regards, shaz
Intermediate & Advanced SEO | | shaz_lhr0