Effect duration of robots.txt file.
-
in my web site there is demo site in that also, index in Google but no need it now.so i have created robots file and upload to server yesterday.in the demo folder there are some html files,and i wanna remove all these in demo file from Google.but still in web master tools it showing
User-agent: *
Disallow: /demo/How long this will take to remove from Google ?
And are there any alternative way doing that ?
-
Google Webmaster Tools also has a remove URL function where you can remove an entire directory, which may be of help to you.
-
And, if they are already indexed, you have to wait for them to be recrawled, then fall out of index, so it's not an immediate thing. Sometimes it takes days, sometimes weeks.
-
Hello,
The robots directive will only prevent google from crawling the pages. In order t remove the pages from index you need to add "meta noindex" to the pages you want to have removed.
<meta name="robots" content="noindex">
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can Image Quality In Online Store Effect Main Site SEO?
My client is building a new shopping cart that will be mobile friendly BUT will have terrible image quality - tiny images, with text on them. How will these tiny images impact the SEO on the main site? Should they put the store on the mainsite.com or on a subdomain (store.mainsite.com)? Does google see subdomains as part of the main site? I would think that having thousands of shopping product pages could be beneficial to the main site SEO, as long as the images don't negate the content. Thoughts?
Intermediate & Advanced SEO | | jerrico10 -
Frequent Blog Content-Effective in Improving Ranking?
To what extent will posting quality blog posts 2 to 3 times per week have the following effects: 1. Improve ranking for specific keywords?
Intermediate & Advanced SEO | | Kingalan1
2. Create backlinks to our website
3. Increase MOZ domain authority
4. Increase organic search traffic Assume that the blog posts are geared towards answering user inquiries and are also posted on our social media accounts. Would such an approach be better than engaging in a link building campaign, in the sense that the links will be created organically by users that want to link to our site? Thanks,
Alan0 -
Domain remains the same IP address is changing on same server only last 3 digits changing. Will this effect rankings
Dear All, We have taken and a product called webacelator from our hosting UKfast and our ip address is changing. UKFasts asked to point DNS to different IP in order to route the traffic through webacelator, which will enhance browsing speed. I am concerned, will this change effect our rankings. Your responses highly appreciated.
Intermediate & Advanced SEO | | tigersohelll0 -
Help with Effective Meta Description
Can someone give me a sample of an effective meta description tag? All of the best practice stuff I read doesn't talk about how to raise CTR. It seems to me that this is a neglected area of SEO, and we want to do this right. Obviously, we will need to test. For example, my main home page keyword is "IT Support" Things I might want to put in the tag: Free Network Assessments 100% Risk Free Trials "Relentless IT Support" (Major Theme) 30 Years of Experience (since 1984) Eliminate your IT Support Headaches Forever (Too long) Call to action? BTW, Thanks to everyone for your help. This is a great community. Solid advice from experts. Here's an example of what I would create Relentless IT Support Since 1984. Trust and Accountability. 100% Risk Free Trials. Contact us today for Free IT Assessment.
Intermediate & Advanced SEO | | CsmBill0 -
Hyphen domain effect SEO?
Hi Guys, I am looking to buy some domain that have the keyword I want in - but my question is; Does using hypehns in a domain effect your SEO? Thanks Gareth
Intermediate & Advanced SEO | | GAZ090 -
Migrating online store to subdomain using shopify and effects on seo and energy down the road for seo
I'm looking for some clarity... Looking at using Shopify for an existing online store that we have to migrate. Setting up the store with shopify means we will be using a subdomain such as shop.mywebsite.com instead of mywebsite.com/shop. The following are points to consider when responding The client currently has an online store, however it's a proprietary shopping store and CMS that has since gone defunct and they need to migrate to an alternative in order to survive online against new CMS systems that allow the site and its content to be better optimized. There is a lot of existing SEO done on the current site that we don't want to loose PR on. There is roughly 2000 products Client has a fixed budget, dealing with checkout issues, custom work and various other "bugs" seems to be easier controlled with Shopify...thus budget can be used more on content/strategy and migration We want to run the main site in Wordpress and are wanting to use Shopify since it supports a gateway, has great features and seems like it would allow us to get more bang for the buck and can focus more on the main site and content strategy and drive traffic to the subdomain store if needed Or main concern is the effort of migrating 2000+ products to shopify and the traffic and PR it gives the current site will have a negative effect on the main domain itself. Should we really be considering this path? The domain is diveidc.com One main benefit to the subdomain is the ability to clearly segment products from the service portion of the site in the analytics and focus 2 clear strategies and track it in a very defined manner. We're really on the fence with this...any thoughts are welcome.
Intermediate & Advanced SEO | | MAGNUMCreative0 -
Robots.txt disallow subdomain
Hi all, I have a development subdomain, which gets copied to the live domain. Because I don't want this dev domain to get crawled, I'd like to implement a robots.txt for this domain only. The problem is that I don't want this robots.txt to disallow the live domain. Is there a way to create a robots.txt for this development subdomain only? Thanks in advance!
Intermediate & Advanced SEO | | Partouter0 -
Sitemaps. When compressed do you use the .gz file format or the (untidy looking, IMHO) .xml.gz format?
When submitting compressed sitemaps to Google I normally use the a file named sitemap.gz A customer is banging on that his web guy says that sitemap.xml.gz is a better format. Google spiders sitemap.gz just fine and in Webmaster Tools everything looks OK... Interested to know other SEOmoz Pro's preferences here and also to check I haven't made an error that is going to bite me in the ass soon! Over to you.
Intermediate & Advanced SEO | | NoisyLittleMonkey0