Can Google read content/see links on subscription sites?
-
If an article is published on The Times (for example), can Google by-pass the subscription sign-in to read the content and index the links in the article?
Example: http://www.thetimes.co.uk/tto/life/property/overseas/article4245346.ece
In the above article there is a link to the resort's website but you can't see this unless you subscribe. I checked the source code of the page with the subscription prompt present and the link isn't there.
Is there a way that these sites deal with search engines differently to other user agents to allow the content to be crawled and indexed?
-
Hey Matt,
The best way to tell what the news organization or site is using is to turn off javascript or view the google cache to determine how Google "sees" the page.
This article is using the second option in the article I mentioned - snippets. Here is what the article has to say about that:
"If you prefer this option, please display a snippet of your article that is at least 80 words long and includes either an excerpt or a summary of the specific article." -
Thanks Dan, it doesn't look like the example article is using first click free. So I guess the answer is no, Google can't read the hidden content in this example?
-
Great question! Yes, Google has an effective way to deal with this since 2007. The three ways they deal with this include first click free, subscription designation, and then disallowing content. Here is their official support article on it:
https://support.google.com/news/publisher/answer/40543?hl=en
Here is a quote from the help article:
"To summarize, we will crawl and index your site to the extent that you allow Googlebot to access it. In order to provide the best possible user experience and help more users discover your content, we encourage you to try First Click Free. If you prefer to limit access to your site to subscribers only, we will respect your decision and show a “subscription” label next to your links on Google News."Here is what Matt Cutts said about it in an interview with Search Engine Land:
"First Click Free originated with Google News, but you can use the same way of handling content in web search (show the same page to users and Googlebot, then if the user clicks to read a different article, then you can show them the registration or pay page). Because the same page is presented to users and to Googlebot, it’s not cloaking. So First Click Free is a great way if you have premium content to surface it in Google’s web index without cloaking. Hope that makes sense."It is possible to allow the Googlebot to access the content and simultaneously NOT provide it for free to non-subscribers. The above help article above should answer all of your questions. Hope this helps!
-
I would say no. The content of the article other than what is seen is not in the source code. They could be showing something different to Google, but if they did it would be against Google's terms of service. https://support.google.com/webmasters/answer/66355?hl=en
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Contextual links (is this screen shot considered contextual /editorial links ?)
Hello, Is the screen shot below considered contextual ?https://imgur.com/a/mrbQq and does it have any value or no value What is the value on a scale from 0 to 10 (if you know) of a contextual link versus non contextual links. Thank you, mrbQq
Intermediate & Advanced SEO | | seoanalytics0 -
[Very Urgent] More 100 "/search/adult-site-keywords" Crawl errors under Search Console
I just opened my G Search Console and was shocked to see more than 150 Not Found errors under Crawl errors. Mine is a Wordpress site (it's consistently updated too): Here's how they show up: Example 1: URL: www.example.com/search/adult-site-keyword/page2.html/feed/rss2 Linked From: http://an-adult-image-hosting.com/search/adult-site-keyword/page2.html Example 2 (this surprised me the most when I looked at the linked from data): URL: www.example.com/search/adult-site-keyword-2.html/page/3/ Linked From: www.example.com/search/adult-site-keyword-2.html/page/2/ (this is showing as if it's from our own site) http://a-spammy-adult-site.com/search/adult-site-keyword-2.html Example 3: URL: www.example.com/search/adult-site-keyword-3.html Linked From: http://an-adult-image-hosting.com/search/adult-site-keyword-3.html How do I address this issue?
Intermediate & Advanced SEO | | rmehta10 -
Site not indexed in Google UK
This site was moved to a new host by the client a month back and is still not indexed in Google UK if you search for the site directly. www.loftconversionswestsussex.com Webmaster tools shows that 55 pages have been crawled and no errors have been detected. The client also tried the "Fetch as Google Bot" tactic in GWT as well as running a PPC campaign and the site is still not appearing in Google. Any thoughts please? Cheers, SEO5..
Intermediate & Advanced SEO | | SEO5Team0 -
Google Site Extended Listing Not Indexed
I am trying to get the new Site map to be picked up by Google for the extended listing as its pulling from the old links and returning 404 errors. How can I get the site listing indexed quickly and have the extended listing get updated to point to the right places. This is the site - http://epaperflip.com/Default.aspx This is the search with the extended listing and some 404's - Broad Match search for "epaperflip"
Intermediate & Advanced SEO | | Intergen0 -
Site Wide Link Situation
Hi- We have clients who are using an e-commerce cart that sits on a separate domain that appears to be providing site wide links to our clients websites. Therefore, would you recommend disallowing the bots to crawl/index these via a robots.txt file, a no follow meta tag on the specific pages the shopping cart links are implemented on or implement no follow links on every shopping cart link? Thanks!
Intermediate & Advanced SEO | | RezStream80 -
Any good link buying companies ( http://www.text-link-ads.com )
Hi guys I have been passed this website: http://www.text-link-ads.com Has anyone ever used text-links ads before?? Can anyone please show me the way and suggest any really good lin buying companies? I am really fiding it hard to find good places to place inbound links into our website.. Thanks Gareth
Intermediate & Advanced SEO | | GAZ090 -
So What On My Site Is Breaking The Google Guidelines?
I have a site that I'm trying to rank for the Keyword "Jigsaw Puzzles" I was originally ranked around #60 or something around there and then all of a sudden my site stopped ranking for that keyword. (My other keyword rankings stayed) Contacted Google via the site reconsideration and got the general response... So I went through and deleted as many links as I could find that I thought Google may not have liked... heck, I even removed links that I don't think I should have JUST so I could have this fixed. I responded with a list of all links I removed and also any links that I've tried to remove, but couldn't for whatever reasons. They are STILL saying my website is breaking the Google guidelines... mainly around links. Can anyone take a peek at my site and see if there's anything on the site that may be breaking the guidelines? (because I can't) Website in question: http://www.yourjigsawpuzzles.co.uk UPDATE: Just to let everyone know that after multiple reconsideration requests, this penalty has been removed. They stated it was a manual penalty. I tried removing numerous different types of links but they kept saying no, it's still breaking rules. It wasn't until I removed some website directory links that they removed this manual penalty. Thought it would be interesting for some of you guys.
Intermediate & Advanced SEO | | RichardTaylor0 -
Are links to on-page content crawled / have any effect on page rank?
Lets say I have a really long article that begins with links to <a name="something">anchors on the same page.</a> <a name="something"></a> <a name="something">E.g.,</a> Chapter 1, Chapter 2, etc, allowing the user to scroll down to different content. There are also other links on this page that link to other pages. A few questions: Googlebot arrives on the page. Does it crawl links that point to anchors on the same page? When link juice is divided among all the links on the page, do these links count and page rank is then lost? Thanks!
Intermediate & Advanced SEO | | anthematic0