How can i block the below URLs
-
Google indexed plugins pages for my website. Please check below. How can stop them to be indexed on google.?
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/glass/
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/borderlesslight3d/
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/defaultskin/
My robots.txt file is -
User-agent: * Disallow: /wp-admin/
-
robots.txt:
Disallow:
/wp-admin/
/wp-content/plugins/
Then, deindex them as mentioned in the first response.
-
It's also worth remembering that blocking a URL in a robots.txt file does not automatically mean that the URL will be deindexed. The robots.txt file will prevent robots, such as the Googlebot, from seeing or accessing the file and that's all. Now obviously, if a robot repeatedly tries to access a file and gets denied it will eventually stop trying to, which is what leads to the file being deindexed (the same principle applies to 404 and 410 errors).
Therefore, if you want a quicker and more definite deindexing solution, you should use the explicit noindex robot command, as recommended above. This will tell any visiting robot to not index it straight away, which will reduce the number of revisits and drop the page from the index faster.
-
Take a look over here - http://www.robotstxt.org/robotstxt.html and we can't forget the Moz version - http://moz.com/learn/seo/robotstxt
Alternatively you can just add meta noindex on the pages with with the added bonus of letting link juice flow better as well as being a bit more stern to robots ( recommend the noindex tag!)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Change of URL + SEO Impact
We recently changed the URLs from our site http://www.website.com (old)
On-Page Optimization | | ederdesign
https://website.com/us/en/ (new) Our ranking is plummeting every since and I wonder if the new URL had something to do with it. Do you know if that change, could have impacted the ranking?0 -
Can we use "top 10" in title?
Hi All, This is a query regarding title tag.I have a listing page with list of hotels in each city. To improve CTR, I used "Top 10 hotels in Boston" as title even though I am showing 25 hotels per page. The page takes about "hotels" but I didn't use "top 10" term anywhere in the page. Is this called keyword stuffing? Should I use "top 10" somewhere in the page as my title says so? Thanks for your help!
On-Page Optimization | | Avinash_12340 -
Should I be using the town or city in url with my keyword or keyphrase?
should I be using the town or city in url with my keyword or keyphrase? So lets say I'm trying to rank for butchers in home town should i put the town in the url as well so www.website.com/butchers-in-mytown is that bad? Or would it be best to just put www.website.com/butchers?
On-Page Optimization | | genkee0 -
How can I reduce Too Many On-Page Links? I am looking for best method through which I can reduce by on page link.
Hello, As I have the Pro Account in SEOMOZ . I have created the campaign for my website and I have seen the warring for on page analysis for Too Many On-Page Links. As per my knowledge in past it's matter that you can put maximum 100 links per page but now is it still matter or harm if pages has Too Many On-Page Links? And if yest then please let me know the best method to reduce my On-Page Links with out doing any major changes in website
On-Page Optimization | | jemindesai0 -
New Url Structure
Hey Guys We are working on a new site and in order to implement some of the new functions we need to restructure our url's , Will redirect everything correctly but I was looking for advice on the structure we need the word product / category subfolder for speed but would there be any benefit making them shorter ? what would you guys advise ? Category Current http://www.freestylextreme.com/uk/Home/Brands/DC-Shoe-Co-/default.aspx **New ** freestylextreme.com/uk/category/dc-shoe-co Product Current http://www.freestylextreme.com/uk/Home/Brands/DC-Shoe-Co-/Mens-DC-Shoe-Co-T-shirts/DC-Black-Unwind-T-Shirt---.aspx New freestylextreme.com/uk/product/dc-black-unwind-t-shirt
On-Page Optimization | | elbeno0 -
Is having the word catalog in an ecommerce site url detrimental to seo.
IS: www.example.com/catalog/category%/product% better than www.example.com/category%/product% category and product are dynamic values that change with the diff. categ. and products displayed while catalog is constant.
On-Page Optimization | | no6thgear0 -
Can someone please help me identify where all these URLS to my homepage are coming from?
Hi. I installed the SEOmoz toolbar for Firefox, and analyzed my home page, then clicked on 'get a full site analysis at Site Explorer'. This is what came up: http://www.opensiteexplorer.org/www.frs-solutions.com%252Fcontent%252Fhome/a!links?src=mb I hope that link works. If not, the URL is www.frs-solutions.com Anyway, there are about 57 different URLS within my site all pointing to my homepage! I have no idea where they are coming from. Can someone with an experienced eye take a quick look and tell me what I might be up against? Thank you!
On-Page Optimization | | aprilm-1890400