Disallow robots on a url effect?
-
Hello, I am using wordpress and on some of my top tag urls e.g : http://www.designzzz.com/tag/brushes/ there is an avg page rank of 4.
I was reading somewhere that my huge jump in not selected count in index status in GWT is becuase of tags/categories urls.
So i added disallow in my robots.txt for the url /tag
What sort of effect will it have on my tag urls rankings or PR ?
-
If your site is ranking "well", well enough for you I mean don't change anything if users are finding there way to what they want. Now for the noindex-follow tag that will not rank the certain pages in the serps but the links on the page will pass pagerank . For the nofollow-noindex or nofollow-index will just hurt the site rankings as they are average pagerank 4 cutting of the power of the posts in the certain category-tag pages not passing any link juice. The best thing to do is leave the pages as is. If the category-tag pages rank better than the posts then i would just noindex them leaving follow on the pages to pass juice towards the posts.
-
but the problem in my case is that for example tags like font, brushes. doesn't have there seperate categories. like font related posting goes under freebies categ and brushes related posts goes under photoshop category . Photoshop categories lots of types of postings like tutorials, downloads, brushes, actions etc.
What i am concerned right now is that if i noindex tag pages. how will they affect my pr and rnakings of these pages?
Or if i nofollow them in meta tag, how will that effect ?
-
The fix i did was to 301 redirect the tag pages to the similar category pages or content pages. This might not be the right thing for your site but I saw new rankings within a day or two. I think this might have lifted some kind of penalty and also siloed the pagerank of the tag pages towards the categories and articles giving them a boost in serps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is there a tool that will find a string from the source code from a list of URLs?
I need a tool that will run through a long list of URLs (from multiple domains), look for a specific string in the source code and then produce a CSV showing in which URLs the the string was found. This is for legitimate research, not for any nefarious purposes Any ideas of such a tool already exists?
Search Behavior | | richdan0 -
Help with robots.txt on Magento
Hi everybody, I need your help in order to fix some problems with HTML errors and Crawling errors generated by Magento on my client's website www.casabiancheria.it I have some problems with duplicate meta informations due to the fact that there are a lot of links such as /stampe-romagnole/tovaglie-con-tovaglioli**/colore/**beige,marrone,giallo,lilla/show/all.html /stampe-romagnole/tovaglie-con-tovaglioli**/colore/**beige,marrone,lilla/show/all.html that are generated by the filter /colore/ and so they have duplicate content and meta information on them. I activated the canonicals on Magento but this hasn't fixed the problem yet. On the sitemap there are only 1 link for each product, so it seems that the canonicals are working, but bot Google Webmaster Tools and SEO Moz are giving me errors on duplicate content and meta informations. I would like to solve these problems by excluding from robots.txt all the urls that contain the filter parameters, such as /colore/, /price/, /dimensions/, etc. (take a look to the attachment). I tried different solutions in order to exclude these links from robots, but I wasn't able to succeed. Below you can find my current robots.txt... can someone help me in order to write the correct form of this file and finally exclude all these urls generated by filters on Magento? Finally, is it worth it to exclude also the images from Magento? (take a look to the final lines of the robots below). Thank you very much for your help! Alberto User-agent: *
Search Behavior | | OptimizedGroup
Disallow: /CVS
Disallow: /.svn$
Disallow: /.idea$
Disallow: /.sql$
Disallow: /.tgz$
Disallow: /w1nL1f3L0g1c/
Disallow: /app/
Disallow: /downloader/
Disallow: /errors/
Disallow: /includes/
Disallow: /lib/
Disallow: /pkginfo/
Disallow: /shell/
Disallow: /var/
Disallow: /404/
Disallow: /cgi-bin/
Disallow: /magento/
Disallow: /report/
Disallow: /scripts/
Disallow: /shell/
Disallow: /skin/
Disallow: /stats/
Disallow: /api.php
Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /get.php
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /README.txt
Disallow: /RELEASE_NOTES.txt
Disallow: /?dir
Disallow: /?dir=desc
Disallow: /?dir=asc
Disallow: /?limit=all
Disallow: /?mode*
Disallow: /index.php/
Disallow: /?SID=
Disallow: /checkout/
Disallow: /onestepcheckout/
Disallow: /customer/
Disallow: /customer/account/
Disallow: /customer/account/login/
Disallow: /catalogsearch/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /cgi-bin/
Disallow: /cleanup.php
Disallow: /apc.php
Disallow: /memcache.php
Disallow: /phpinfo.php
Disallow: /control/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/
Disallow: /catalog/product/gallery/
Disallow: /?*
Disallow: //colore/
Disallow: //price/
Disallow: //misura/
Disallow: //marca/
Disallow: //sort-by/
Disallow: //combinazione/
Disallow: /*/seleziona-colore/
Disallow: /colore/
Disallow: /price/
Disallow: /misura/
Disallow: /marca/
Disallow: /sort-by/
Disallow: /combinazione/
Disallow: /seleziona-colore/
Disallow: /*colore/
Disallow: /*price/
Disallow: /*misura/
Disallow: /*marca/
Disallow: /*sort-by/
Disallow: /*combinazione/
Disallow: /*seleziona-colore/ UmuEX4z0 -
Canonical url for eshop, to do or not to do?
Hello everyone, I'm having a headache right now in designing a url base for a eshop. I would be thankful for any suggestions. Just to be clear about my problem, I'll be using this product example:
Search Behavior | | donatasBart
Product name: Product
Categories: Sale, Women
Brand: Superbrand Basicly every name of product, category or brand will resolve as url, if I enter url: /product it will open Product page /superbrand it will open brand page /superbrand/product will open product page from superbrand page /superbrand/sale/product would open product page selected from superbrand, then category sales /superbrand/woman/product would open product page selected from superbrand, then category woman So I will have many variations how to reach the product page, and of course i should set canonical url for it. I could set it just to /product , but then I don't know if I won't loose in terms of seo for other more detailed urls. Will google still index my other urls for product if i will state the canonical url in each product page? I know, maybe it would be the best solution to create a link like this: /superbrand_sale_product , which I can, but however it will be a duplicate content issue, since there will me many url pointing to the same product, and the only difference between them will be category, brand or whatever else... Thanks for all your ideas!0 -
GWMT - "Tag Site For Child Directed Treatment" Effect On Search / Rankings?
Hi All, We have a client who has been directed to tag their site for "Child Directed Treatment" in Webmaster tools to comply with AdExchange policies. The site is, generally speaking, directed at those between the ages of 13 and 16 along with their parents, but does NOT collect any data (No sign in, login, signups etc). You can find out more about the specific tag here (unfortunately not much more about it) https://support.google.com/webmasters/answer/3221080?hl=en Our concern is that we have never heard of this specific tag, and our client is asking us to let them know if that this will have no effect on search traffic or ranking. I can't find much in the way of anyone who HAS implemented this tag and the effects it has had on their site. They are ad supported, receive millions of unique hits a month, and the majority of their traffic is from branded keywords. Would love to hear from anyone with ANY experience or thoughts on this process and what to be aware of. Your assistance is muchly appreciated.
Search Behavior | | SearchMarketers0 -
Would it be worthwhile to update URLs to include "for sale"
My team is new to the world of SEO. We were both in other departments when our last "expert" left and we became the entire department. Needless to say, we have many questions. I have a site for an existing customer - she wants to know if it is a good idea to change her URLs. She went to a webinar that suggested she include "for sale" in URL, since that is the site's primary purpose These are the current inventory URLs: http://www.dealerskinssite.com/used-inventory http://www.dealerskinssite.com/new-inventory The change would give something like: http://www.dealerskinssite.com/used-cars-for-sale
Search Behavior | | DealerskinsSearch
http://www.dealerskinssite.com/new-cars-for-sale I did some searching and could not find a solid recommendation one way or another. If it is true that people typically search for "used cars for sale" rather than a "used inventory", then there might be benefit to including that as part of the URL... but that is as far as my knowledge got me. Please let me know your thoughts Steve0 -
Two sites with different URLs
I help organize an ecommerce site for a company that is named after the state that the company was founded in (ex: Florida Pipes). Our programmer is thinking about creating a duplicate site that would have a name that was more location agnostic so people shopping on the site would not think that they may be ordering from far away hence incurring extra shipping charges. He said that the site would only have a different URLs, name and homepage but would link to all of the same stuff that is on the site that is up now. He said that there would be no way to tell (possibly for the layperson) that the two sites were related. Is this a good practice? Would we be penalized in search results for having two URLs linking to the same content? (We are a 30 year old company that ranks very high for our main keyword) Thank you for your input!
Search Behavior | | Winoman0 -
Trailing slash at end of URLs?
Hi, I'm just about to put up a new site and I need to decide between having no trailing slash at the end of the all the URLs, or putting one in there. I think Matt Cutts has a slight preference for them, as stated here: http://www.seomoz.org/blog/whiteboard-interview-googles-matt-cutts-on-redirects-trust-more "Matt says he would slightly advocate for using a trailing slash simply because it clearly indicates that a URL is a folder and not a document." However, I'm really wondering. I mean, if people link to my site, they'll tend not to insert a trailing slash, I'm thinking... Your thoughts would be welcome on this one! Cheers, Luke
Search Behavior | | McTaggart0 -
Where can I find research into consumer behavior in URL input?
I have a hypothesis that the % of users that type URLs is decreasing. Instead I think users will be arriving at sites by clicking on links in emails, search and social. Where can I find data / research to back me up?
Search Behavior | | TomCritchlow0