Wp-login.php
-
To block or not to block with robots.txt?
-
Michael, the best thing to do for you would be to check Google Webmasters Tools and see what Google sees there as duplicate content.
This will help you decide whether you want or not to block /tag/.
-
I get that. Just not sure about blocking them. A lot of sites don't block them and yoast.com is one of them. So, if big seo guys aren't blocking tag then are we 100% sure that Google sees "/tag/" as duplicate content?
-
If you're happy with your results, why block anything in Robots? The short term answer is tags are working for you. The long term answer is you're creating duplicate content. So it just depends - are you optimizing for today and tomorrow's sales or for long term success. There is no "right" answer to that but that's the question.
-
With all the talk here about not indexing /tag/ why do you not include it in your robots?
-
I understand that but what's happening is that i have tag results in the top 10 and not the post result. The post result is indexed but not anywhere close to the tag. If i block tags, in my head, I think i run the risk of losing traffic. I have to be 100% certain of this for I'm getting and been getting a lot of traffic from those "tag" results.
-
You don't want to index tags in Wordpress. You'll have duplicate content on any post that has more than one tag on it (which is almost all of them.) You want to get the canonical permalinks indexed and that's it. Search for a post you have indexed under a tag and then do a keyword search to see if it appears a few times (archives, multiple tag pages, etc.) That's bad - you want to fix that.
-
I would start with a robots.txt like the one below and add in anything else you don't want to be crawled/indexed:
Sitemap: http://www.example.com/sitemap.xml # Google Image User-agent: Googlebot-Image Disallow: Allow: /* # Google AdSense User-agent: Mediapartners-Google* Disallow: # digg mirror User-agent: duggmirror Disallow: / # global User-agent: * Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /trackback/ Disallow: /feed/ Disallow: /comments/ Disallow: /category/*/* Disallow: */trackback/ Disallow: */feed/ Disallow: */comments/ Disallow: /*? Allow: /wp-content/uploads/
-
Not sure I follow. I have indexed tag results on Google that have the /tag/ in the url. If i noindex them, won't i lose those urls that get indexed?
-
I block wp-admin and wp-includes just because they don't need to be indexed.
As far as tags, you can easily no-follow all of those with any half decent Wordpress SEO plugin like Yoast or All in One SEO. I would definitely noindex them.
-
You may want to add a short description and custom titles for those tags. There are some ways to do that in WordPress.
This way your content will have something unique and will not be 100% duplicate.
-
Thanks for the reply. Makes sense. What about tags? Should they be blocked in robots.txt too? Reason I ask is that i have indexed tag content in Google and if I blog /tags/ from being indexed then I'm worried that I'll lose that indexed content. The reason for the blocking of /tags/ is because seomoz is finding duplicate content from /tags/ and I don't want that to hurt me.
-
Do you have any links from your main pages pointing to wp-login.php?
If not, the search engines will not know about that URL, so you can choose to not block it.
But if you have links to wp-login.php and don't want the page indexed, then you should block it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Many of my pages are showing up as 4xx errors because they require a login/subscription to view. How can I fix this?
Should these pages be redirecting to a login page? Or does there need to be something other than an Access Denied message? I feel like these are bringing the overall site score down.
On-Page Optimization | | CaraMc0 -
Does anyone use No Sweat WP Internal Links Lite?
Would you recommend it? Are there any other plugins that you'd recommend instead?
On-Page Optimization | | --Chantal--0 -
One Company, Two Brands with Two Blogs, but One WP Panel for Blog?
I work with a company that has 2 brands. Both brands have separate sites (currently on a WP multisite install). We want each brand to have its own blog, but for ease of content creation have ONE wp install to create the blog content and depending on what category is clicked (Brand 1 and/or Brand 2), it will publish to that sites blog. 2 questions: 1. Is one WP install for blog syndication for 2 separate sites advisable (as client is requesting)? Or should we just bite the bullet and have each site have it's separate posting through it's own WP install? 2. Sometimes one blog post will be published to BOTH blogs (i.e. category Brand 1 and Brand 2 clicked OR if we use two separate wp installs for each site, publish to both blogs). Is using a rel=canonical for the original post (we need to decide which brand takes precedence) sufficient to overcome duplicate content problem? Thanks in advance! Stephan
On-Page Optimization | | stephanwb0 -
WordPress image urls - need a WP maven
We were having a conversation re urls that are indexed for images that are stored in various media plugins in WP. My question for anyone who is an uberWP person is: What is your opinion re best media storage plugins and how these URLs affect pages on a site for ranking, etc. I realize this is broad, but it is driven out of my concern that I cannot touch everything. When I see a url like this: http://www.drumbeatmarketing.net/wp-content/themes/drumbeat2/img/DB-LOGO-White.png I know there is no way with all the sites and clients we handle that I can get it perfect but this just bugs me for some reason. Should I just chill since it (seemingly) affects so little....?
On-Page Optimization | | RobertFisher1 -
The correct way to go from PHP site to HTML site?
I have a website fully coded in PHP and I am doing a re-design over to an HTML site. I searched through the Q&A and there were some conflicting answers. Some said you will need to 301 all the pages. Others said to use the .htaccess to parse all the files as html. What is the correct way I should go about this? Thanks in advance!
On-Page Optimization | | reliabox0 -
Indexed iframed content behind login
Hi, I have a question regarding iframed content. I would like to get my non cms content which is served via an iframe solution (from the same domain) behind a anonymous or personal login indexed by search engines. How can we make this work? I've looked at the following solutions: http://googlewebmastercentral.blogspot.nl/2008/10/first-click-free-for-web-search.htmlhttp://productforums.google.com/forum/#!topic/webmasters/l9n8oGLQRkUBut I would like the content to be crawlable deeper than the just one page (if this is possible using the iframe solution).We could also setup different new pages in our CMS with the same content...Any suggestions?Thanks!Arnout
On-Page Optimization | | hellemans0 -
Can anyone recomend a good, solid, review generator for WP?
Looking for some feedback on a good review generator for customer input and hopefully generate some testimonials. Thanks in advance.
On-Page Optimization | | tvw1300 -
Rename index.php or keyword in URL?
It is important for me to get good search results for keyword + city name For instance: tulips amsterdam What would be better: renaming index.php or adding the cityname to the URL? www.example.com/amsterdam/tulips OR www.example.com/pages/tulips-amsterdam
On-Page Optimization | | svdg0