How can i block the below URLs
-
Google indexed plugins pages for my website. Please check below. How can stop them to be indexed on google.?
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/glass/
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/borderlesslight3d/
http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/defaultskin/
My robots.txt file is -
User-agent: * Disallow: /wp-admin/ -
robots.txt:
Disallow:
/wp-admin/
/wp-content/plugins/
Then, deindex them as mentioned in the first response.
-
It's also worth remembering that blocking a URL in a robots.txt file does not automatically mean that the URL will be deindexed. The robots.txt file will prevent robots, such as the Googlebot, from seeing or accessing the file and that's all. Now obviously, if a robot repeatedly tries to access a file and gets denied it will eventually stop trying to, which is what leads to the file being deindexed (the same principle applies to 404 and 410 errors).
Therefore, if you want a quicker and more definite deindexing solution, you should use the explicit noindex robot command, as recommended above. This will tell any visiting robot to not index it straight away, which will reduce the number of revisits and drop the page from the index faster.
-
Take a look over here - http://www.robotstxt.org/robotstxt.html and we can't forget the Moz version - http://moz.com/learn/seo/robotstxt
Alternatively you can just add meta noindex on the pages with with the added bonus of letting link juice flow better as well as being a bit more stern to robots ( recommend the noindex tag!)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
can we include iframe content if we properly reference the source in the url?
can we safely include iframe content if we properly source the source url. eg we have a travel company we are looking to produce another website, but use the same hotel descriptions in the hotel directory. can we do this?
On-Page Optimization | | Direct_Ram0 -
Changing existing URL's to improve SEO
Here's a general question: At what PR/page rank (or Moz 'page authority') would you no longer change an existing URL that's cryptic to one that includes the related page keyword or at least relevant terms? Does using a 301 redirect to the new URL preserve the page rank? Thanks!
On-Page Optimization | | flyntime_tx0 -
How do you handle URLs with slashes?
I asked this question before, but with a different scenario. I upgraded my plan to a more advanced cart and all of my URLs changed about 1.5 years ago. I knew nothing about redirects and such, so none of that was done. Basically, let's say my site was: http://www.abc.com, but when people actually visit my site, they are directed to https://www.abc.com/. I have asked my host about redirecting and she that it is not possible. In the past, the link shared has been just www.abc.com . Will this hurt my ranking? My second question is ...let's say I have a link http://www.abc.com/blog , but now, the link is http://www.abc.com/blog/ . Will I be affected, since all my old links omit the slash?
On-Page Optimization | | tiffany11030 -
Can you set objectives for an SEO campaign?
We have been working with a client for 3 months and they've been happy with the progress we've made on their SEO and the resulting increase in rankings, traffic etc. They are renewing their contract for another 6 months but this time want to set some objectives and want us to suggest what those objectives are. Is it possible to set objectives when a great deal of the results can be out of our hands, eg, how Google works, level of interest in the client's products in the target market, whether they take on board our recommendations for changing their website content to get better seo results etc? What would other people be happy being set as their objectives for a campaign?
On-Page Optimization | | mfrgolfgti0 -
Too Many On-Page Links, Can You HELP???
This is the best architecture I found to help my visitors find there furnace filter size. Does it hurt my SEO? Index page has, 210 links and most other pages has, 190 links. Thank you, BigBlaze
On-Page Optimization | | BigBlaze2050 -
How long can rich snippets be?
How long (how many words) can a description tag of a rich snippets code?
On-Page Optimization | | seoflorida0 -
URL Rewrite
(By Google Traductor) Hello, I wanted to ask about some changes that we are evaluating for the issue of passing the url with variables to be more descriptive, for example: http://www.agroads.com.ar/detalle.asp?clasi=139592 tohttp://www.agroads.com.ar/humedimetro-para-cereales-draminski-gmm-139592.html In this case corresponds to the breakdown of a product if you have long published andcan be well positioned to change the title of this position would be lost unless youmanage it with a 301, as one would manage when you have more than 30000 products and title may change several times? There are tools to manage this? Finally, we must apply this to all listed with their respective filters, recommends doingtheir part with 301 redirects and analyze what funciene well to continue with the rest or implement a complete change? I hope I can bring a little light to implement this. Greetings and thanks! Roberto
On-Page Optimization | | romaro0 -
Does google treat all urls equal?
Sorry for the lame title, i couldn't think of a better one. I want to know if google treats this: http://www.domain.com/products/some-product-name the same as it would treat: http://www.domain.com/?products=some-product-name if not, could you tell me the differences?
On-Page Optimization | | adriandg0
