Welcome to the Q&A Forum

Here is what I have changed it to...found various articles including the one listed above and decided to go with this, not sure if it is good or bad.

www.ensoplastics.com/robots.txt

I need to know if anyone has tips on creating a good robots.txt. I have read a lot of info, but I am just not clear on what I should allow and not allow on wordpress.  For example there are pages and posts, then attachments, wp-admin, wp-content and so on. Does anyone have a good robots.txt guideline?

I have the same problem looks like MSN bot is disallowed from accessing wordpress content. So pages show up as ?page=111 so from what I understand so far anything that shows as below is blocked from MSNbot. I don't have a definite answer for you as to what to do, but I can tell you will need to "allow" msn bot the googlebot is.

Disallow: /key-west-blog/*?*

So basically this site was duplicated and apparently the robots.txt file was duplicated. There is no sitemap for the blog created for the enso plastics site, so I am not sure how to proceed at this point. Should I just create a new robots.text file for enoplastics and replace this one? Or do I edit this one, and go create a sitemap for my blog?

Well that is a problem isn't it? Like I said I am new to a lot of this and I didn't develop either site, this robot.txt file is pointing to the wrong site map. So I am going to change that.

However I am guessing I may need to change some of the rules to get it to where it is not blocking wordpress content.

Well that is a problem isn't it? Like I said I am new to a lot of this and I didn't develop either site, this robot.txt file is pointing to the wrong site map. So I am going to change that.

However I am guessing I may need to change some of the rules to get it to where it is not blocking wordpress content.

Here is my robots.txt from google webmaster tools. These are the pages that are being blocked and I am not sure which of these to get rid of in order to unblock blog posts from being searched.

http://ensoplastics.com/theblog/?cat=743

http://ensoplastics.com/theblog/?p=240

These category pages and blog posts are blocked so do I delete the /? ...I am new to SEO and web development so I am not sure why the developer of this robots.txt file would block pages and posts in wordpress. It seems to me like that is the reason why someone has a blog so it can be searched and get more exposure for SEO purposes.

IS there a reason I should block any pages contained in wodrpress?

Sitemap: http://www.ensobottles.com/blog/sitemap.xml

User-agent: Googlebot

Disallow: /*/trackback

Disallow: /*/feed

Disallow: /*/comments

Disallow: /?

Disallow: /*?

Disallow: /page/
User-agent: *

Disallow: /cgi-bin/

Disallow: /wp-admin/

Disallow: /wp-includes/

Disallow: /wp-content/plugins/

Disallow: /wp-content/themes/

Disallow: /trackback

Disallow: /commentsDisallow: /feed

Thanks, more thumbs up for you! That worked...got it done...if you read this response can you explain if that will resolve my duplicate page issues when site is crawled. I have been ending up with duplicate page titles for 52 pages I was told to go and add a canonical tag to each page but will this resolve that duplicate page detail?

I will try this and see what happens, so I add the www and the root without www. is that what you are saying? Thats what I am reading at least.

So I am trying to set the URL preference in google webmaster tools for my site. However when I try to save it it tells me to verify that I own the site. I have already done this so where can I go to verify I own the site exactly? Maybe I am wrong and I have not done this already but even on the homepage of webmaster tools I don't see an option to "verify".