Do robot.txts permanently affect websites even after they have been removed?

[Driver72]

A client has a Wordpress blog to sit alongside their company website. They kept it hidden whilst they were developing what it looked like, keeping it un-searchable by Search Engines. It was still live, but Wordpress put a robots.txt in place. When they were ready they removed the robots.txt by clicking the "allow Search Engines to crawl this site" button.

It took a month and a half for their blog to show in Search Engines once the robot.txt was removed.

Google is now recognising the site (as a "site:" test has shown) however, it doesn't rank well for anything. This is despite the fact they are targeting keywords with very little organic competition.

My question is - could the fact that they developed the site behind a robot.txt (rather than offline) mean the site is permanently affected by the robot.txt in the eyes of the Search Engines, even after that robot.txt has been removed?

Thanks in advance for any light you can shed on the situation.

[mkleinrock]

No problem! Good Luck!

[Driver72]

That is a very fair point. It is a completely new site and I hadn't even thought about things like the domain age. It does show up under a "site:http://www.____.com" search, I was just wondering if this is one of those things Google keeps a memory of, if that makes sense.

Thanks for your response Mike.

[Driver72]

That is a very good suggestion. I'll try it (a useful URL also so thanks for sharing).

Thanks for the response Matthew.

[94501]

I think the much more likely culprit is that it is a new site. What do you get when you enter "site:http://www._____.com" in google? If the pages are indexed, one can't blame for the robots file for lack of rank.

Good luck!

Mike

[mkleinrock]

Have you submitted the updated robots.txt to google? This is separate from updating the sitemap. Here is a google page to help you do this.

https://support.google.com/webmasters/answer/6078399?hl=en

Best!

Matthew