Most of my sites uses Wordpress, what I did to never have any duplicate content is the following:
First, I have the following plugins installed
- .html on PAGES
- All in ONe SEO Pack
- cbnet Ping Optimizer
- Google XML Sitemaps
Then, added the following text to the robots.txt
User-agent: *
Disallow: /wp-adminDisallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackbackDisallow: /comment
Disallow: /categoria//
Disallow: */trackback
Disallow: */comments
Disallow: /sem-categoria
Disallow: /pollsarchive
Disallow: /category
Disallow: /?
Disallow: /*?
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Allow: /wp-content/uploads
Google ImageUser-agent: Googlebot-ImageDisallow:Allow: /* # Google AdSenseUser-agent: Mediapartners-GoogleDisallow:Allow: / # digg mirrorUser-agent: duggmirrorDisallow: / Sitemap: http://www.YOURSITE.com.br/sitemap.xml
On the admin in the PERMALINK config tab, in the COMMON SETTINGS i choose
- Custom Structure -> /%postname%.html
In SETTINGS > cbnet Ping Optimizer I inserted
http://api.moreover.com/RPC2
http://bblog.com/ping.php
http://blogsearch.google.com/ping/RPC2
http://ping.weblogalot.com/rpc.php
http://ping.feedburner.com
http://ping.syndic8.com/xmlrpc.php
http://ping.bloggers.jp/rpc/
http://rpc.pingomatic.com/
http://rpc.weblogs.com/RPC2
http://rpc.technorati.com/rpc/ping
http://rpc.reader.livedoor.com/ping
http://topicexchange.com/RPC2
http://www.blogpeople.net/servlet/weblogUpdates
http://xping.pubsub.com/ping
http://api.my.yahoo.com/RPC2
http://audiorpc.weblogs.com/RPC2
http://blogsearch.google.pt/ping/RPC2
http://blogpeople.net/ping
I use this plugin to make sure that wordpress will not ping those services everytime I change or update something on the page,actually I seted the plugin to ping only after 30 min after the last ping.
So If you just posted something on Wordpress it will ping all those services for a fast index, however, if for some reason you have to edit the post when you save this plugin will make sure the services will not get pingged again in such a short space of time.
In the All in One Seo config I have the following config
UNCHECKED
- Use noindex for Categories
- Use noindex for Tag Archives
- Autogenerate Description
I also set Exlcude Pages, Additional Post Headers, Additional Page Headers and Additional Home Headers all blank
With all this configuration if someone access any post of my blogs they get a page with http://domain/post permalink structure, doesnt matter if the access came from a TAG, Categorie or normal Search.
Aditionally the main Tag page and Categorie page are indexed, so i ended up using my tags for some sort of relevance to the posts that belongs to that Tag. In google search the same post could appear by many related TAGs filter but the content of the post appears in an unique page.
Hope that helps
PS.: Suggestions to improve this config are welcome