Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Cloaking? Best Practices Crawling Content Behind Login Box
- 
					
					
					
					
 Hi- I'm helping out a client, who publishes sale information (fashion sales etc.) In order for the client to view the sale details (date, percentage off etc.) they need to register for the site. If I allow google bot to crawl the content, (identify the user agent) but serve up a registration light box to anyone who isn't google would this be considered cloaking? Does anyone know what the best practice for this is? Any help would be greatly appreciated. Thank you, Nopadon 
- 
					
					
					
					
 Can I say I admire your inventiveness?  You go to some lengths to not register and really, apart from the majority of people not knowing how to do a reverse image search, probably reflects people's attitude to those sorts of lightbox registration forms. You go to some lengths to not register and really, apart from the majority of people not knowing how to do a reverse image search, probably reflects people's attitude to those sorts of lightbox registration forms.
- 
					
					
					
					
 I'm going to respond from a human point of view and not a technical point of view. I've been searching for houses recently on Craigslist. There are a couple of real estate agents who post ads on CL with a link to their site. When you click the link, you get a lightbox requiring that you fill out the lead form to be able to see the details of the house. I do one of two things: - 
I open up IE in private browsing mode and paste in the URL. The private browsing mode has something that prevents this script from running and I can see the house details just fine. 
- 
If the house address is not provided in the CL ad, I'll copy the image URL of one of the CL photos and put that into a Google reverse image search. I'll find a different website that has posted the same house and use their site that doesn't require me to register. (I realize this may not happen in your scenario above). 
 I agree what the other people say about not wanting provide one thing to Google and another to users, and wanted to add that people will try to find ways around the registration. I don't have a solution for you, sadly. 
- 
- 
					
					
					
					
 Heya there, Thanks for asking your question here  My first point would be that human visitors don't like to be given forms when they first visit a site, so would suggest you don't do this. My alternative strategy would be to provide a home page of good content talking about the data etc that is available on your site and then provide a button for people to register if they want to. Don't detect the user agent and provide alternative content as, however good your intentions are, that could be considered cloaking. Google is against you providing Google different content to humans, so don't do it. Do things differently  
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
- 
		
		Moz ToolsChat with the community about the Moz tools. 
- 
		
		SEO TacticsDiscuss the SEO process with fellow marketers 
- 
		
		CommunityDiscuss industry events, jobs, and news! 
- 
		
		Digital MarketingChat about tactics outside of SEO 
- 
		
		Research & TrendsDive into research and trends in the search industry. 
- 
		
		SupportConnect on product support and feature requests. 
Related Questions
- 
		
		
		
		
		
		Duplicate Content and Subdirectories
 Hi there and thank you in advance for your help! I'm seeking guidance on how to structure a resources directory (white papers, webinars, etc.) while avoiding duplicate content penalties. If you go to /resources on our site, there is filter function. If you filter for webinars, the URL becomes /resources/?type=webinar We didn't want that dynamic URL to be the primary URL for webinars, so we created a new page with the URL /resources/webinar that lists all of our webinars and includes a featured webinar up top. However, the same webinar titles now appear on the /resources page and the /resources/webinar page. Will that cause duplicate content issues? P.S. Not sure if it matters, but we also changed the URLs for the individual resource pages to include the resource type. For example, one of our webinar URLs is /resources/webinar/forecasting-your-revenue Thank you! Technical SEO | | SAIM_Marketing0
- 
		
		
		
		
		
		How to allow bots to crawl all but WP-content
 Hello, I would like my website to remain crawlable to bots, but to block my wp content and media. Does the following robots.txt work? I worry that the * user agent may conflict with the others. User-agent: * Technical SEO | | Tom3_15
 Disallow: /wp-admin/
 Disallow: /wp-includes/
 Disallow: /wp-content/ User-agent: GoogleBot
 Allow: / User-agent: GoogleBot-Mobile
 Allow: / User-agent: GoogleBot-Image
 Allow: / User-agent: Bingbot
 Allow: / User-agent: Slurp
 Allow: /0
- 
		
		
		
		
		
		What to do with old content after 301 redirect
 I'm going through all our blog and FAQ pages to see which ones are performing well and which ones are competing with one another. Basically doing an SEO content clean up. Is there any SEO benefit to keeping the page published vs trashing it after you apply a 301 redirect to a better performing page? Technical SEO | | LindsayE0
- 
		
		
		
		
		
		Best practices for types of pages not to index
 Trying to better understand best practices for when and when not use a content="noindex". Are there certain types of pages that we shouldn't want Google to index? Contact form pages, privacy policy pages, internal search pages, archive pages (using wordpress). Any thoughts would be appreciated. Technical SEO | | RichHamilton_qcs0
- 
		
		
		
		
		
		Best practice for URL - Language/country
 Hi, We are planning on having our website localized into more languages. We already have an English and German version. The German version is currently a sub-domain: www.example.com --> English version de.example.com --> German version Is this recommended? Or is it always better to have URLs with language prefixes such a: www.example.com/de www.example.com/es Which is a better practice in terms of SEO? Technical SEO | | Kilgray1
- 
		
		
		
		
		
		Duplicate Content
 We have a ton of duplicate content/title errors on our reports, many of them showing errors of: http://www.mysite.com/(page title) and http://mysite.com/(page title) Our site has been set up so that mysite.com 301 redirects to www.mysite.com (we did this a couple years ago). Is it possible that I set up my campaign the wrong way in SEOMoz? I'm thinking it must be a user error when I set up the campaign since we already have the 301 Redirect. Any advice is appreciated! Technical SEO | | Ditigal_Taylor0
- 
		
		
		
		
		
		Merging several sites into one - best practice
 I had 2 sites on the web (www.physicseditor.de, www.texutrepacker.com) and decided to move them all under one single domain (www.codeandweb.com) Both sites were ranking very good for several keywords. I not redirected the most important pages from the old domains with a 301 redirect to the new subpages (www.texturepacker.com => www.codeandweb.com/texturepacker) Google still delivers the old domains but the redirect take people directly to the new content. I've already submitted the new site map to google webmaster tools. Pages are already in the index but do not really show up in the search results. How long does it take until google accepts the new domain and delivers the new content in the search results? Was it ok what I did? Or is there some room for improvement? SeoMoz will of course not find any information about the new page since it is not yet directly linked in google. But I can't get ranking information for the "old" pages since SeoMoz tells me that it can't crawl the old domains.... Technical SEO | | gossi740
- 
		
		
		
		
		
		Crawling image folders / crawl allowance
 We recently removed /img and /imgp from our robots.txt file thus allowing googlebot to crawl our image folders. Not sure why we had these blocked in the first place, but we opened them up in response to an email from Google Product Search about not being able to crawl images - which can/has hurt our traffic from Google Shopping. My question is: will allowing Google to crawl our image files eat up our 'crawl allowance'? We wouldn't want Google to not crawl/index certain pages, and ding our organic traffic, because more of our allotted crawl bandwidth is getting chewed up crawling image files. Outside of the non-detailed crawl stat graphs from Webmaster Tools, what's the best way to check how frequently/ deeply our site is getting crawled? Thanks all! Technical SEO | | evoNick0
 
			
		 
			
		 
			
		 
					
				 
					
				 
					
				 
					
				 
					
				 
					
				 
					
				