How to make google crawl our repository to make our site rank but make sure users dont go to our repository ?
-
We have a website that has links to documents related to various sectors. But the challenge is we do not have the documents on the website itself and they are linked to our document repository that has been blocked to google. We have put nofollow and noindex to the repository. Since Google can not read those documents, it has resulted in an impact in our SEO ranking. What would be the best way to make Google crawl the PDF documents in the repository at the same time make it invisible the "repo" not appear in the search engines. Would dofollow and noindex sequence work ?
-
Playing with indexation tags can be dangerous (same goes for robots.txt). Google should still be able to read the repo even if it is no-indexed, as long as you haven't also blocked the repo in robots.txt. Robots.txt is telling Google what to crawl, no-index is telling Google what it can or can-not put in its search results
Of course, if your docs were ranking because of PageRank passed from the repo, the no-index tag will kill the PageRank of the repo (and thus all the docs which it links to, as they are not being 'fed' any more). If a page is no-indexed, it's seen as unimportant for Google and the PageRank is often nullified. Although Google can crawl no-indexed URLs, they crawl them WAY slower as they're seen as really unimportant with no PageRank (at the bottom of the internet)
Why not just put all your PDF docs in a PDF sitemap ans submit to Google in Search Console:
https://stackoverflow.com/questions/1072880/should-i-list-pdfs-in-my-sitemap-file
This will let Google see them all. But if their parent is no-indexed with no PageRank, they may still not rank as well as before...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site migration 301's list on Apache server - when do I take the old migrated URLs off our 301 list?
Howdy! The site was migrated last year to a new platform, therefore the URLs were migrated over using 301 redirect rules. Since then, the server has a huuuuge list of rules for URLs and I wanted to know when can we start taking these off of the list? And, how can I test to see if they are still indexing, do I simply just add the url to the google search bar? Thanks Guys! Kay
SEO Learn Center | | eLab_London0 -
What should I place in the code to connect my html 5 website to Facebook, Google+ etc...?
I know most people use a CMS these days but I created a html 5 website for my small business using Dreamweaver. I'd like to know what, if anything, I should place in the code to link my website with my social media accounts like Facebook, Google+, Twitter, and Pinterest? I've found information about plugins that are useful if you're using a CMS but I'm not. I placed social media buttons on all of the pages of my website already, and when you click on those buttons they go to my social media accounts. But is there anything that should be placed in the code? Thanks for your help
SEO Learn Center | | Ophelia6190 -
What keywords will this page possibly rank for ... tool?
Hi all, I'm looking for a tool that will look at a web page that you submit to it and tell you what keywords it thinks the page is targetted towards and thus hopefully rank for. Does anyone have any suggestions please? Hopefully this type of tool will help us understand why page A is being ranked for keyword X when page B is actually the page that should be being ranked for that keyword. The only tool I have really found so far is http://www.ranks.nl/tools/spider.html I dont want to tell the tool what keywords I am looking at, as with Moz On page grader. Background reason (and possibly some one maybe able to shed some light on this also). We list a group of products... Nextbase Click 9 Lite, Nextbase Click 9 Lite Duo and Nextbase Click 9 Lite Duo Deluxe. All 3 are similar but very different products and each has its own page on our site. For the keyphrase "Nextbase click 9 lite" currently in Google we are listed for the pages containing the "Nextbase Click 9 Lite Duo" and "Nextbase Click 9 Lite Duo Deluxe"... but not the actual specific product page that targets the "Nextbase Click 9 Lite" product. Thus my reason for wanting to try such a tool that may help me understand why this is. Thanks for any hints, tips or pointers.
SEO Learn Center | | jasef0 -
Does canonical links rank in Google?
Our company has many pages that use the canonical tag. Will these pages rank in the search engine or will it pass the strength on to the original page?
SEO Learn Center | | WebRiverGroup0 -
Multilingual site (rel="alternate" hreflang="x")
Hi there, We have a multilingual site with same content translated into different languages. Some are also in different countries with same language. The issue with our site is that international Google SERP shows our US site before their own country/language site. In order to resolve this issue, we implemented the following rel="alternate" hreflang="x" across all locales: After testing these out, our US site still shows up on search results in other countries (checked after 2 months). Are we missing something here? Our guess is that it's not working because we are missing canonical tags or meta language code (). Thanks in advance.
SEO Learn Center | | hjoeng-010 -
What about "CAPS" in site titel.
Can you get down rank in your search result for useing CAPS in your site titels? Many of my compitors useing it. http://www.google.dk/#hl=da&cp=9&gs_id=1h&xhr=t&q=flyttemand&pf=p&sclient=psy-ab&site=&source=hp&pbx=1&oq=flytteman&aq=0&aqi=g4&aql=&gs_sm=&gs_upl=&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=a7d5043c494a2f9f&biw=1392&bih=749 Eksample og search result "flyttemand" Regards
SEO Learn Center | | SortSEO0 -
Location of Rand's Webinar "Using Open Site Explorer to Uncover New Marketing Opportunities"
Aug 26th Archive of Rand's Webinar "Using Open Site Explorer to Uncover New Marketing Opportunities" I don't see this in the archive, anyone know how to get it? Thanks
SEO Learn Center | | josey0 -
Machine Learning - Randomness in Search Engine Ranking Algorithms
I believe, sometimes you may 'deserve' the first position, but get #3. And of course, sometimes you may deserve #3 and instead be #1. All due to a 'randomness factor' in search engine algorithms. I've been holding this hypothesis for quite sometime. Really, it started when I learned about SEOmoz using machine learning to better investigate SEO best practices. I suddenly found myself wanting to learn more about machine learning, and the implications of using it for SEO. I'm by no means able to utilize machine learning myself, but I it appears unsupervised learning would have a real chance of cracking search engine algorithms. Hey, even Stuxnet was cracked! Surely Google/ Bing would know (and account for) this, right? We can agree they'd obviously prefer a highly skilled mathematician not be able to crack the code. Therefore, I'm led to believe that search engines use some sort of randomness in their rankings. Maybe not much. Perhaps not all the time. But if a random percentage of search results, had a random variable of sorts included in their calculations... wouldn't that be enough to prevent the vast majority of cracking attempts? Thoughts, opinions, criticism? Thanks.
SEO Learn Center | | DonnieCooper0