Wp-login.php
-
To block or not to block with robots.txt?
-
Michael, the best thing to do for you would be to check Google Webmasters Tools and see what Google sees there as duplicate content.
This will help you decide whether you want or not to block /tag/.
-
I get that. Just not sure about blocking them. A lot of sites don't block them and yoast.com is one of them. So, if big seo guys aren't blocking tag then are we 100% sure that Google sees "/tag/" as duplicate content?
-
If you're happy with your results, why block anything in Robots? The short term answer is tags are working for you. The long term answer is you're creating duplicate content. So it just depends - are you optimizing for today and tomorrow's sales or for long term success. There is no "right" answer to that but that's the question.
-
With all the talk here about not indexing /tag/ why do you not include it in your robots?
-
I understand that but what's happening is that i have tag results in the top 10 and not the post result. The post result is indexed but not anywhere close to the tag. If i block tags, in my head, I think i run the risk of losing traffic. I have to be 100% certain of this for I'm getting and been getting a lot of traffic from those "tag" results.
-
You don't want to index tags in Wordpress. You'll have duplicate content on any post that has more than one tag on it (which is almost all of them.) You want to get the canonical permalinks indexed and that's it. Search for a post you have indexed under a tag and then do a keyword search to see if it appears a few times (archives, multiple tag pages, etc.) That's bad - you want to fix that.
-
I would start with a robots.txt like the one below and add in anything else you don't want to be crawled/indexed:
Sitemap: http://www.example.com/sitemap.xml # Google Image User-agent: Googlebot-Image Disallow: Allow: /* # Google AdSense User-agent: Mediapartners-Google* Disallow: # digg mirror User-agent: duggmirror Disallow: / # global User-agent: * Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /trackback/ Disallow: /feed/ Disallow: /comments/ Disallow: /category/*/* Disallow: */trackback/ Disallow: */feed/ Disallow: */comments/ Disallow: /*? Allow: /wp-content/uploads/
-
Not sure I follow. I have indexed tag results on Google that have the /tag/ in the url. If i noindex them, won't i lose those urls that get indexed?
-
I block wp-admin and wp-includes just because they don't need to be indexed.
As far as tags, you can easily no-follow all of those with any half decent Wordpress SEO plugin like Yoast or All in One SEO. I would definitely noindex them.
-
You may want to add a short description and custom titles for those tags. There are some ways to do that in WordPress.
This way your content will have something unique and will not be 100% duplicate.
-
Thanks for the reply. Makes sense. What about tags? Should they be blocked in robots.txt too? Reason I ask is that i have indexed tag content in Google and if I blog /tags/ from being indexed then I'm worried that I'll lose that indexed content. The reason for the blocking of /tags/ is because seomoz is finding duplicate content from /tags/ and I don't want that to hurt me.
-
Do you have any links from your main pages pointing to wp-login.php?
If not, the search engines will not know about that URL, so you can choose to not block it.
But if you have links to wp-login.php and don't want the page indexed, then you should block it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO and auto login with redirect
Hi, we have a site where user we know (cookie-based) is automatic redirected to private internal page. Unknown guests are staying on the page google search sent them. The redirect is done with a 302 to the private page. When I examine organic search and landing pages in analytics I see that more than 50% of the landing pages are private pages (inaccessible for the google-bot). This is because analytics does not register the redirecting page, just the target page. Is this harmful to SEO that so many users are redirected to other pages, or is this data not available to google-bot. The google bots is not redirect since they do not have the correct credentials. Do chrome or analytics send such data back to Google search? Is better to manipulate analytics page track call to use the address of the redirecting page? Or maybe skip to redirect, and just render the private content on the orginal page?
On-Page Optimization | | Morten.Berg0 -
Should I block google indexing "search.php"
My question is I have a search page on our website , you can search by date, number of people staying and so on, I am just wondering should block this in the robots.txt ? Because we have pretty URL'S already for searching by county and searching by towns. I cannot see any benefit of having e.g "search/search.php?sp_dateFrom=16%2F12%2F2015&sp_dateTo=23%2F12%2F2015&sec_drop%5B%5D=727&spesh_town_id=764&q=&occupants=5&bedrooms=3&submit=SEARCH#search" indexed. Would I be correct in doing this ?
On-Page Optimization | | McCaldin0 -
Yoast WP Plugin - Social - G+ Author
Hi The author for homepage box/dropdown menu is ony showing 'dont show' or the G+ company page name (which i presume is publsiher not author) as options. The G+ personal page profile does link to the website in links section so wandering what might be going wrong here that i'm not being giving the option of selecting the author ? any help here appreciated ? all best dan
On-Page Optimization | | Dan-Lawrence0 -
The crawl diagnosis indicated that my domain www.mydomain.com is duplicate with www.mydomain.com/index.php. How can I correct this issue?
How can I fix this issue when crawl diagnosis indicated that my www.mydomain.com is duplicate with www.mydomain.com/index.php? That suppose to be the same page and not duplicate, right?
On-Page Optimization | | jsevilla0 -
Rename index.php or keyword in URL?
It is important for me to get good search results for keyword + city name For instance: tulips amsterdam What would be better: renaming index.php or adding the cityname to the URL? www.example.com/amsterdam/tulips OR www.example.com/pages/tulips-amsterdam
On-Page Optimization | | svdg0 -
Titles for WP feeds and categories
SeoMoz comes back with duplicate title errors. However these duplicate errors are /feed pages in Wordpress. http://www.guardianpoolfence.com/choose-pool-fence/feed/ http://www.guardianpoolfence.com/how-to-choose-pool-fence/feed/ Why does this happening with only selected pages? I have many more posts but only these two are coming up with duplicate title errors. Also how do I set up meta descriptions for categories and archive pages for example www.guardianpoolfence.com/2011/07 I am not sure how these pages were created and how to give these individual titles. I mean, titles at all... Anyone?
On-Page Optimization | | Viktoria0 -
Too many On-Page Links on a WP based Website
Hi, I've already browsed through various of the Q&As on the "too many On-Page links" issue, but I would really need some advice concerning a WP Site with a dropdown navigation. As outlined in the on-page report, every site has about 180 outgoing links, which pretty much is the number of site featured in the navigation. Even though the 100 link limit is somewhat outdated I'm still worried about the distribution of linkjuice from the starting page and how Google perceives the importance of the various pages. Would it make sense to adapt the structure of the navigation, so that the starting page only links to the 5 category pages and the category pages only link to the detail pages they contain? The site has good rankings for several pages and I assume that Google can tell that the large number of links is caused by the navigation. But with every page having appr. 180 links it may be difficult for Google to tell, which of those pages are the most important regarding internal link structure... Looking foward to your opinion and insights! Cheers, Chris
On-Page Optimization | | adwordize0