How to make google crawl our repository to make our site rank but make sure users dont go to our repository ?
-
We have a website that has links to documents related to various sectors. But the challenge is we do not have the documents on the website itself and they are linked to our document repository that has been blocked to google. We have put nofollow and noindex to the repository. Since Google can not read those documents, it has resulted in an impact in our SEO ranking. What would be the best way to make Google crawl the PDF documents in the repository at the same time make it invisible the "repo" not appear in the search engines. Would dofollow and noindex sequence work ?
-
Playing with indexation tags can be dangerous (same goes for robots.txt). Google should still be able to read the repo even if it is no-indexed, as long as you haven't also blocked the repo in robots.txt. Robots.txt is telling Google what to crawl, no-index is telling Google what it can or can-not put in its search results
Of course, if your docs were ranking because of PageRank passed from the repo, the no-index tag will kill the PageRank of the repo (and thus all the docs which it links to, as they are not being 'fed' any more). If a page is no-indexed, it's seen as unimportant for Google and the PageRank is often nullified. Although Google can crawl no-indexed URLs, they crawl them WAY slower as they're seen as really unimportant with no PageRank (at the bottom of the internet)
Why not just put all your PDF docs in a PDF sitemap ans submit to Google in Search Console:
https://stackoverflow.com/questions/1072880/should-i-list-pdfs-in-my-sitemap-file
This will let Google see them all. But if their parent is no-indexed with no PageRank, they may still not rank as well as before...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How would you go about influencer marketing in the b2b space?
Hi Mozzers, I have conducted some b2c micro influencer marketing campaigns in my previous gig by reaching out to blogger and having them promote some content we created. I am now working in a Fintech company (b2b) and would like to get a few ideas on how can I leverage the influencer marketing channel within the Fintech industry. Can you share some ideas on different b2b influencer marketing strategies? What are good processes to follow besides cold email, meetups, events? Any influencer marketing softwares you have used and recommending it in that space? Thanks!
SEO Learn Center | | Taysir0 -
Setting up a remote business with their Google Business Account
Hi all, I have a client who operates a remote business and I need to get her Google business/ brand account set up. She doesn't want it listed under her home address (for obvious reasons) but that is where her business is based out of. Apart from getting a PO box and listing her business under that for Google, does anybody know of any other options or best practices? Thanks!
SEO Learn Center | | Zx30 -
Webinar: Get to Know the New Site Crawl Recording
Good morning all, Does anyone have the recording for Friday, June 9th's webinar? I was at an off-site event that day and couldn't listen in. The recording isn't listed in the moz.com/webinars section. Any help is appreciated, thank you!
SEO Learn Center | | Corporate_Synergies1 -
How to ask Google to remove old pages that don't exist
I have moved a site to a new location. There are a number of pages that date back to 2012 and 2011 and we do not want them anymore. Is there a way to ask Google to remove these pages entirely.
SEO Learn Center | | mayanksaxena0 -
SEO Stratergy for new ecom site
Hello Experts, I need expert advice for my ecom site. My site is www nsale dot in , domain is 2 years old, we just revamped the site. Its currently under testing. Dev activity will be complete in another 2-3 day and product addition will start in a week. we are targeting all major categories of product. I am not sure about how to start the SEO and social media marketing. Can someone help on how to plan and execute inorder to get targeted traffic and also steady increase in traffic ? Targetted traffic is India. PR is currently 0. Thanks in advance.
SEO Learn Center | | nsale0 -
Does canonical links rank in Google?
Our company has many pages that use the canonical tag. Will these pages rank in the search engine or will it pass the strength on to the original page?
SEO Learn Center | | WebRiverGroup0 -
Remove image URL from a google crawl
hello one of the warnings Im getting is that one of my urls has too many on page links here's the page: http://cheap-airport-taxis.com/airport-cab/?show=gallery but the url is being auto generated due to the wordpress plugin NextGen Gallery anyway I can exclude these from the site crawl. This page has no purpose, as the core path shows these images anyway. James
SEO Learn Center | | smashseo0 -
How do I get google to crawl white papers that displays a form for human visitors?
How do I get Google to crawl white papers that displays a form for human visitors? I have been looking into this and understand that I need to set the form up as a GET form which has been done. Google said they want you to "avoid" forms that require personal information but to what extent do they want you to do that? The form is used as a lead generator so we need to collect information such as name, company name, email, ect.The information we require currently is: Name, Company name, Email, Phone Number and Number of employees. Once a user puts in their information they have access to the rest of the content and they don't need to re-enter the information in so I assume once Google gets past this feature they can gain access to the rest of the content. I understand that I need to have a form that doesn't ask for personal information which is the dilemma. So what should we do to work around this? Is there a solution that will allow me to obtain some personal information while still allowing Google to crawl the pages? Thoughts and any feedback is much appreciated, TJ
SEO Learn Center | | SEO_com0