Scraping a Password Protected Page
-
Hey Everyone,
I'm looking for some help regarding my website.
I have multiple pages on my site that are password protected (we only give the password to potential clients)....
Somehow, when doing a google search, i found a directory that has successfully scraped and pulled all of the information from the password protected pages and is displaying this info on their site...
This is very bad for us as obviously this information is critical and not intended to be shared with the rest of the world
Does anyone have any advice on how to prevent my password protected pages from being scraped/crawled?
Thanks!
-
Hi there, as Takeshi said, it is difficult to answer this without more information (especially how you are implementing password protection on these pages.) Are you able to offer any information at all, or did you figure this out? We'd love an update so we can help you resolve this.
Christy
-
How are you implementing the password protection? What website is it for? It's hard to say anything without knowing the specifics.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Rel Canonical for the Same Page
Hi, I was looking in my one of my moz accounts and under analyz page under notices is a message that says: Rel Canonical Using rel=canonical suggests to search engines which URL should be seen as canonical. I checked an notice that I do have a rel='canonical' href='http://www.example.com' /> from the home page of http://www.example.com. I guess my question is. Does having a Rel Canonical going to the same page hurt my SEO? I'm not sure why it is there but wanted to make sure I address this correctly. I was under the impression you use Rel Canonical for duplicate or similar pages and you want to let Google know what page to show. But since I've made this mistake to where I am saying to show the home page if you find a similar home page, should I just delete the Rel Canonical. Thanks,
Technical SEO | | ErrickG
Errick0 -
Issue: Duplicate Page Content > Wordpress Comments Page
Hello Moz Community, I've create a campaign in Moz and received hundreds of errors, regarding "Duplicate Page Content". After some review, I've found that 99% of the errors in the "Duplicate Page Content" report are occurring due to Wordpress creating a new comment page (with the original post detail), if a comment is made on a blog post. The post comment can be displayed on the original blog post, but also viewable on a second URL, created by Wordpress. http://www.Example.com/example-post http://www.Example.com/example-post/comment-page-1 Anyone else experience this issue in Wordpress or this same type of report in Moz? Thanks for your help!
Technical SEO | | DomainUltra0 -
Page URL Change
We're planning on rolling out a redesign of an existing page, and at the same time, we're looking to possibly changing the URL of the page. Currently, the URL is www.blah.com/phraseword1-phraseword2-phraseword3-phraseword4 and we're ranking top 3 in Google SERP for that 4-word phrase. The keyword phrase is something we have in our Page Title, Site Copy and the URL. Now, we are planning on simplifying the URL to below.. www.blah.com/phraseword1-phraseword2 The plan is to 301 redirect the original URL to this new URL and actually work the exact phrase into the copy a few more times. My understanding is that URL doesn't get as much weight as it does in the past, but it's still important. So my question is... How important is the URL in this case where we will continue to have it in our page title and also we'll be working more copy on to the page with the appropriate keyword? Will 301 redirect from the old URL address the issue of passing SEO value for that keyword phrase? Thanks,
Technical SEO | | JoeLin
Joe0 -
Page for Link Building
Hello Guys, My question is about a link building process. We all know that some directories/sites do require a reciprocal link. Does it make any sense to creat a page in website exclusively to reciprocal links? And what we do with this webpage in terms of indexing, do folow, crawling...etc. Any sugestions are more then welcome 🙂 Tks in advance! PP
Technical SEO | | PedroM0 -
Home Page .index.htm and .com Duplicate Page Content/Title
I have been whittling away at the duplicate content on my clients' sites, thanks to SEOmoz's pro report, and have been getting push back from the account manager at register.com (the site was built here and the owner doesn't want to move it). He says these are the exact same page and he can't access one to redirect to the other. Any suggestions? The SEOmoz report says there is duplicate content on both these urls: Durango Mountain Biking | Durango Mountain Resort - Cascade Village http://www.cascadevillagehotel.com/index.htm Durango Mountain Biking | Durango Mountain Resort - Cascade Village http://www.cascadevillagehotel.com/ Your help is greatly appreciated! Sheryl
Technical SEO | | TOMMarketingLtd.0 -
Getting on to page 1 again
I have recently set up a website, and as soon as it got indexed by google, I ranked on the first page at number 10... However, Ever since I have started trying to backlink.. the more I seem to do, the more I seem to drop down the rankings! The same for both Bing and Yahoo... I am really not sure what the problem is. My site is www.arilinegamesatc.com, and the ranking words are airline games and airline game. Any help woudl be greatly appreciated!
Technical SEO | | rolls1230 -
How can I prevent duplicate content between www.page.com/ and www.page.com
SEOMoz's recent crawl showed me that I had an error for duplicate content and duplicate page titles. This is a problem because it found the same page twice because of a '/' on the end of one url. e.g. www.page.com/ vs. www.page.com My question is do I need to be concerned about this. And is there anything I should put in my htaccess file to prevent this happening. Thanks!
Technical SEO | | onlineexpression
Karl0 -
If you only want your home page to rank, can you use rel="canonical" on all your other pages?
If you have a lot of pages with 1 or 2 inbound links, what would be the effect of using rel="canonical" to point all those pages to the home page? Would it boost the rankings of the home page? As I understand it, your long-tail keyword traffic would start landing on the home page instead of finding what they were looking for. That would be bad, but might be worth it.
Technical SEO | | watchcases0