How to find out if I have been penalized?

[socialtowards]

I have launched a new website beginning January this year and have seen slowly more and more traffic coming from google to the website until the 20th of March where suddenly there are no more visitors from the google search engine. The only traffic left is from google images, social networks or other search engines. Without visitors from google search this reduces our overall traffic by ~66%.

I can't easily find anymore our website in the search results of google by using terms which we usually ranked quite well. Nevertheless, the website is still indexed as I can find it using the "site:" search query. In google webmaster tools there are no messages and we have only been doing a bit of link building on website and blog directories  (nothing excessive and nothing paid neither).

Is there any way to find out if google penalized my website? I guess it has... and what would be the best thing to do right now?

The website is hellasholiday (dot) com

Thanks in advance for your idea and suggestions

[AlanMosley]

I am not a fan of CMS, i realize there are pros and cons, but when you try to do too much and be all things to all people you tend to have a lot of compromises.

There is one other reason i dont like to use robots,txt, i remeber Matt Cutts saying that it is a spam signal because they can not see what you are hiding, not that it is going to get you flaged by itself, but with other signals it can. If i remember correctly he was talking about hiding malware in scripts blocked by robots.

If you are interested, the best CMS for SEO i had found was Orchard CMS but even that has some silly errors, it puts more then one H1 tag in pages, but is still the best solution I have looked at. It is more customizable via code.

[socialtowards]

After having read your post and all the linked articles you have recommended I understand the issue and have adapted the robots.txt accordingly. Basically only leaving one single Disallow for the WordPress plugins. I hope this will help but I suppose I will see this in the next few days...

Now regarding WordPress I would suggest them to adapt their documentation as it is really misleading. Also I think they should implement all these noindex meta tags where necessary natively into wordpress and not by having to use a plugin for that, but this is another story.

[AlanMosley]

Wordpress do many things that are not recommened, and blocking by robots is not recomened, what they are suggesting is a extream messure to solve the softewares problems. there are better ways to solve duplicate content without giveing away your link juice

Read this section "WordPress Robots.txt blocking Search results and Feeds"

on this page http://yoast.com/example-robots-txt-wordpress/

These plug-ins like yoast and word press itself, do not produse very good results. I have crawled many wordpress sites and they all have the same old problems many caused by the yoast plugin.

What google is refereing to in the link, is not getting pages of little value into their index, this is for their advantage not yours.

Its quite simple, if you block a page, the links pointing to that page waste their link juice, if you dont, or at least allow follow with a meta tag, you will get the link juice back.

See this article where Dr Pets calls it an extream messure, search for robots.txt you will see many comments refering to my point http://www.seomoz.org/blog/duplicate-content-in-a-post-panda-world

See Dr pets comments here http://www.seomoz.org/blog/serious-robotstxt-misuse-high-impact-solutions

[socialtowards]

I thought it would be no use for google to index and cache small icons, logos and cached resized images which have no meaningful name or so. So now I have at least removed the Disallow for these but for WordPress blog I want to keep the Disallow rules as recommended by WordPress itself for SEO purposes as documented here http://codex.wordpress.org/Search_Engine_Optimization_for_WordPress#Robots.txt_Optimization assuming they know what they are speaking about.

Anyhow I don't have the feeling this is really the problem why my website doesn't show up anymore in the google search engine results...

[AlanMosley]

The question should be why block them?

its like cutting off your hand, because you have a splinter.

If duplicate content is a problem, then you can (in order of prefrance) fix it, use a canonical, a noindex,follow meta tag, but not robots

[socialtowards]

Many thanks Alan for your answer!

Regarding the robots.txt, basically I just would like to block/disallow some cached images and small icons/pictures from the website as well as some stuff for the associated WordPress blog which is also host on the same website. For the blog I am disallowing the admin pages, feeds, comments, trackbacks, content theme files etc. Here wold be the complete list just in case:

Disallow /wp-admin

Disallow: /wp-includes

Disallow: /wp-content/plugins

Disallow: /wp-content/cache

Disallow: /wp-content/themes

Disallow: /trackback

Disallow: /feed

Disallow: /comments

Disallow: /category//

Disallow: /*/trackback

Disallow: /*/feed

Disallow: /*/comments

Disallow: /?

Disallow: /*?

So maybe I should change my question to "what URLs should I disallow for a WordPress blog?"

Also where can I see all the pages which are blocked by my robots.txt file?

[AlanMosley]

You can ask for reconsideration from google though webmaster tools. But since you have no warnings and you are still in the index, i have doubts that you have been flagegd manualaly, but you may have been algorthmicly.

I notived that you have blokced hundreds of pages with robots.txt, thios had led to thousonds of links pointing to pages that are not indexed, this means these links are puiring away link juice into nowhere.

You should not use robots text to block pages that are linked to, its a waste of valuable link juice.

if you must no-index the pages, use a meta noindex,follow tag, this way you will get most of the link juice back though the pages outlinks