I have to agree with Matt on this one for a few reasons. His niche is search engine optimization I respect that you've been running your own site for 15 years and I think what's being missed here is Matt is giving you a a lot of his time and resources to look into. I would be spending my time looking at the robots.txt it's a nightmare Yoast suggests using this for WordPress
User-Agent: *
Disallow: /wp-content/plugins/
your robot text currently looks like this
User-agent: *
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: rogerbot
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: exabot
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: MJ12bot
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: dotbot
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: gigabot
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: AhrefsBot
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: Yandex
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: Baiduspider
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: JikeSpider
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: YoudaoBot
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
User-agent: BoardReader
Disallow: /
Allow: /wp-content/uploads/
Disallow: /abovethelaw/
Disallow: /adsother/
Disallow: /advice/
Disallow: /animation/
Disallow: /cache/
Disallow: /cse/
Disallow: /cgi-bin/
Disallow: /code/
Disallow: /codeny/
Disallow: /codesnips/
Disallow: /dev/
Disallow: /dictionary/
Disallow: /error/
Disallow: /form/
Disallow: /forum/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/converse.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/memberlist.php
Disallow: /forums/members.php
Disallow: /forums/member.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/forums/
Disallow: /forums/images/
Disallow: /forums/jobs/
Disallow: /forums/legal-help/
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/z/
Disallow: /generator/
Disallow: /geoip/
Disallow: /includes/
Disallow: /job/
Disallow: /lawyers/
Disallow: /LAWYERS_CURRENT/
Disallow: /legalforms/
Disallow: /menu/
Disallow: /news/
Disallow: /nlmailer/
Disallow: /siteforms/
Disallow: /tmp/
Disallow: /feed
Disallow: /*/feed
Disallow: /xmlrpc
Disallow: /wp-
Disallow: /?p=
Disallow: /*trackback
Sitemap: http://www.thelaw.com/jobs/sitemap_index.xml
he currently is working with HTML 5 and Genesis 2 so he keeps saying do not use what's on my website it could cause problems however the snippet above is something he does recommend this this is current sites robots.txt
Yoasts gets 1 million visitors a month and he only needs this
User-Agent: *
Disallow: /wp-content/plugins/
Disallow: /out/
Disallow: /bugs/
Disallow: /suggest/
Allow: /wp-content/plugins/vipers-video-quicktags/resources/jw-flv-player/player.swf
but recommends this for everyone
User-Agent: *
Disallow: /wp-content/plugins/
if you want to look into more changes with your robots.txt I suggest using
http://www.internetmarketingninjas.com/seo-tools/robots-txt-generator/
it is a fine tool that allows you to see the changes real-time in comparison if you were to alter your robots.txt
http://yoast.com/articles/robots-meta-tags/
http://yoast.com/example-robots-txt-wordpress/
Robots.txt is a way to tell a search engine which pages it’s allowed to spider, to “see”, and which pages it cannot “see”. Because of that, robots.txt differs from `meta name="robots"` tags, which tell search engines on those individual pages, whether they can include them in their index or not. The difference is subtle, but important. Because of that, the [suggested robots.txt in the codex](http://codex.wordpress.org/Search_Engine_Optimization_for_WordPress#Robots.txt_Optimization) is wrong. Let me explain:
Google sometimes lists URLs that it’s not allowed to spider, because it’s blocked by robots.txt, because a lot of links point to a URL. A good example of this is a search for [[RTL Nieuws](https://www.google.com/search?q=rtl+nieuws&pws=0)] (disclosure: RTL is a client of mine). rtlnieuws.nl 301 redirects to the [news section of rtl.nl](http://www.rtl.nl/actueel/rtlnieuws/home/). But… rtlnieuws.nl/robots.txt exists… And has the following content:
| `1` | `User-agent: *` |
| `2` | `Disallow: /` |
Because of that, the links towards rtlnieuws.nl don’t count toward the news section on rtl.nl, and Google displays rtlnieuws.nl in the search results. This is unwanted behavior that we’re trying to fix but for now it’s a good example of what I wanted to explain. By _blocking_ /wp-admin/ and /trackback/ in your robots.txt, you’re not preventing them from showing up.
Unfortunately, recently the /wp-admin/ block was added to WordPress core, because of[this Trac ticket](http://core.trac.wordpress.org/ticket/18465). In the discussion on that ticket, I’ve proposed another solution in [this patch](http://core.trac.wordpress.org/attachment/ticket/18465/noindex.patch). This solution involves sending an X-Robots-Tag header, which is the HTTP header equivalent of a `meta name="robots"` tag. This _would_ in fact remove all wp-admin directories from Google search results.
I believe both of you guys have a lot of passion for what you do and it shows. What needs to happen or what should happen is either somebody has to start looking into things because honestly too many other sites are doing just fine it is not google singling you out however I do feel that Google has rough policies but regardless of that.
Please do not respond with anything that Will be at all confrontational as I am taking the time to try to help I'm doing this as a volunteer you know not really anything more.
I am hoping that you guys can get this fixed obviously there is something wrong with your website I would pay a well recognized search engine optimization company to fix it if I were you however that doesn't seem to be what you want to do.
I'm certain if you took it to the
http://evolvingseo.com
http://distilled.net
http://www.portent.com/
http://Internetmarketingninjas.com
or
http://www.seerinteractive.com/
every one of them would be able to solve your problem. It's most likely that you're blocking the search engines in some form or combination of many things but it's not an impossibility to fix. If you believe spending a few thousand dollars on something that you've worked 15 years for is to much I don't know what to tell you.
I would take into account that search engine optimization and whatever the subject is don't always have to coincide you might be surprised to find out that the best marketers are not part of the same field their promoting meaning they have no history of working in it, but they know it gets things done on their end.
I would honestly take it to a reputable SEO as there are issues with the site.
Sincerely,
Thomas