Like Daniel said you can use robots.txt to block spiders, but this won't guarantee exclusion of URLs showing up in search results. You could use x-robots-tag in the server headers. Generate a 403 every time user-agent hits the sub domain.
Latest posts made by roimediaworks
-
RE: How do you block development servers with robots.txt?posted in On-Page Optimization
Best posts made by roimediaworks
-
RE: How do you block development servers with robots.txt?posted in On-Page Optimization
Like Daniel said you can use robots.txt to block spiders, but this won't guarantee exclusion of URLs showing up in search results. You could use x-robots-tag in the server headers. Generate a 403 every time user-agent hits the sub domain.