How can i block the below URLs
-
Google indexed plugins pages for my website. Please check below. How can stop them to be indexed on google.?
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/glass/
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/borderlesslight3d/
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/defaultskin/
My robots.txt file is -
User-agent: * Disallow: /wp-admin/
-
robots.txt:
Disallow:
/wp-admin/
/wp-content/plugins/
Then, deindex them as mentioned in the first response.
-
It's also worth remembering that blocking a URL in a robots.txt file does not automatically mean that the URL will be deindexed. The robots.txt file will prevent robots, such as the Googlebot, from seeing or accessing the file and that's all. Now obviously, if a robot repeatedly tries to access a file and gets denied it will eventually stop trying to, which is what leads to the file being deindexed (the same principle applies to 404 and 410 errors).
Therefore, if you want a quicker and more definite deindexing solution, you should use the explicit noindex robot command, as recommended above. This will tell any visiting robot to not index it straight away, which will reduce the number of revisits and drop the page from the index faster.
-
Take a look over here - http://www.robotstxt.org/robotstxt.html and we can't forget the Moz version - http://moz.com/learn/seo/robotstxt
Alternatively you can just add meta noindex on the pages with with the added bonus of letting link juice flow better as well as being a bit more stern to robots ( recommend the noindex tag!)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL structure
Hello all, I am about to sort out my websites link structure, and was wondering which approach to our services page would be best. should we have: services/digital-marketing & services/website-design etc or: digital-marketing/website-design & digital-marketing/seo Basically I see digital marketing as the top level category that is the umbrella term for all of our digital services. But would it make more sense to have service to be the main category and digital marketing within that (along with all the other services from web design to seo)? all thoughts welcome!
On-Page Optimization | | wseabrook0 -
Optimal URL structure for location-specific pages
I'm in the middle of revamping a website for a restaurant that has multiple locations and am trying to decide what the best URL/internal link structure would be. Right now, each restaurant has a single location page, but we are going to add additional pages for catering. Sitewide-linked pages exist for /catering and /locationname. The way I see it, we have two basic options: Option #1: Catering page - /locationname/catering/ Option #2: Catering page - /catering/locationname/ In both cases, there would be links from the /locationname an /catering pages to the location-specific catering pages. Is either option preferable to the other?
On-Page Optimization | | mblair0 -
Wordpress category url problem.
I have set up wordpress categories but the permalinks are showing as www.mydomain.com/?cat=12 as opposed to the category name. The child categories though work fine and show as www.mydomain.com/category/chidcatgegory I've obviously got my permalink settings wrong somewhere. How do I fix this?
On-Page Optimization | | SamCUK0 -
How many keywords max can I optimize each page for?
I don't want to over optimize by doing 1 keyword per 1 page, but then if I do more, seomoz on-page tool report doesn't give an A grade for each keyword I optimize. I usually optimize for max 3 keywords that are very closely related, meaning they use the same words. Ex. dentist los angeles, los angeles dentist, dentist in los angeles Am I on the right track or what's your recommendation? Should I create different landing pages for each keyword?
On-Page Optimization | | sub90900 -
Can I add a blog to different pages of my site?
I would like to ad a blog to a few different pages of my site, is this possible? I would prefer it to not be a stand alone blog, because I am pretty sure it would begin to outrank my landing pages in search results. I built my site using dreamweaver, but I have seen a reference somewhere to a utility for adding a word press blog to an existing site? This site seems to be doing what I had in mind? http://www.drynclean.com/
On-Page Optimization | | ayetti0 -
Importance of URL Structure
We are trying to restructure our onpage SEO and want to make sure we have our URLs correct. The problem is we did the URLs incorrectly in the first place and the ones we currently have are several years olds. We have some URLs such as: http://www.firebrandtraining.co.uk/courses/management/prince2.asp and
On-Page Optimization | | RobertChapman
http://www.firebrandtraining.co.uk/courses/cisco/ccna_2007.asp which are not ideal but user experience aside does it make sense for us to change the URLs and use 301 redirects to the new ones or is the damage done to our natural rankings simply not worth making the change? I have read different articles saying different things, some say that URL structure has little weight (if any weight at all) on rankings while other people seem to say it is quite important. In addition we have heard that changing the URLs with a 301 redirect will cause a large drop in ranking which will take months to recover from and contrarily that 301s are now considered "ok" by Google and we shouldn't see too much change at all in our rankings. Any advice would be much appreciated.0 -
Trailing slash on URLs
Hi everyone My question is regarding trailing URLs in Wordpress A designer setup a site and made the URL structure something like: www.website.com/description/ but I want to change it to www.website.com/description as this is more SEF A trailing / was added to the permalink structure in Wordpress, if I change this will google see these as new URLs and will all the current URLs become 404? Or should I just leave it? Thanks in advance
On-Page Optimization | | webseoservices0 -
Is having the word catalog in an ecommerce site url detrimental to seo.
IS: www.example.com/catalog/category%/product% better than www.example.com/category%/product% category and product are dynamic values that change with the diff. categ. and products displayed while catalog is constant.
On-Page Optimization | | no6thgear0