Disallow robots on a url effect?
-
Hello, I am using wordpress and on some of my top tag urls e.g : http://www.designzzz.com/tag/brushes/ there is an avg page rank of 4.
I was reading somewhere that my huge jump in not selected count in index status in GWT is becuase of tags/categories urls.
So i added disallow in my robots.txt for the url /tag
What sort of effect will it have on my tag urls rankings or PR ?
-
If your site is ranking "well", well enough for you I mean don't change anything if users are finding there way to what they want. Now for the noindex-follow tag that will not rank the certain pages in the serps but the links on the page will pass pagerank . For the nofollow-noindex or nofollow-index will just hurt the site rankings as they are average pagerank 4 cutting of the power of the posts in the certain category-tag pages not passing any link juice. The best thing to do is leave the pages as is. If the category-tag pages rank better than the posts then i would just noindex them leaving follow on the pages to pass juice towards the posts.
-
but the problem in my case is that for example tags like font, brushes. doesn't have there seperate categories. like font related posting goes under freebies categ and brushes related posts goes under photoshop category . Photoshop categories lots of types of postings like tutorials, downloads, brushes, actions etc.
What i am concerned right now is that if i noindex tag pages. how will they affect my pr and rnakings of these pages?
Or if i nofollow them in meta tag, how will that effect ?
-
The fix i did was to 301 redirect the tag pages to the similar category pages or content pages. This might not be the right thing for your site but I saw new rankings within a day or two. I think this might have lifted some kind of penalty and also siloed the pagerank of the tag pages towards the categories and articles giving them a boost in serps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Direct 100+ URLs to our sister site?
Hi, We have two websites www.eurekasolutions.co.uk and www.eurekaaddons.co.uk - currently the eurekasolutions site has eureka addon products on it - we are thinking of re-directing these to www.eurekaaddons.co.uk instead (there are 100+ pages which are being ranked and picked up by google) Would it be foolish to re-direct these from eurekasolutions straight to eurekaaddons after them being on eurekasolutions for many many years? if so what is the best way to go about it and would this have a direct negative effect on our SEO? Both sites are serving for similar terms. Thanks,
Search Behavior | | EurekaSolutions0 -
Help with robots.txt on Magento
Hi everybody, I need your help in order to fix some problems with HTML errors and Crawling errors generated by Magento on my client's website www.casabiancheria.it I have some problems with duplicate meta informations due to the fact that there are a lot of links such as /stampe-romagnole/tovaglie-con-tovaglioli**/colore/**beige,marrone,giallo,lilla/show/all.html /stampe-romagnole/tovaglie-con-tovaglioli**/colore/**beige,marrone,lilla/show/all.html that are generated by the filter /colore/ and so they have duplicate content and meta information on them. I activated the canonicals on Magento but this hasn't fixed the problem yet. On the sitemap there are only 1 link for each product, so it seems that the canonicals are working, but bot Google Webmaster Tools and SEO Moz are giving me errors on duplicate content and meta informations. I would like to solve these problems by excluding from robots.txt all the urls that contain the filter parameters, such as /colore/, /price/, /dimensions/, etc. (take a look to the attachment). I tried different solutions in order to exclude these links from robots, but I wasn't able to succeed. Below you can find my current robots.txt... can someone help me in order to write the correct form of this file and finally exclude all these urls generated by filters on Magento? Finally, is it worth it to exclude also the images from Magento? (take a look to the final lines of the robots below). Thank you very much for your help! Alberto User-agent: *
Search Behavior | | OptimizedGroup
Disallow: /CVS
Disallow: /.svn$
Disallow: /.idea$
Disallow: /.sql$
Disallow: /.tgz$
Disallow: /w1nL1f3L0g1c/
Disallow: /app/
Disallow: /downloader/
Disallow: /errors/
Disallow: /includes/
Disallow: /lib/
Disallow: /pkginfo/
Disallow: /shell/
Disallow: /var/
Disallow: /404/
Disallow: /cgi-bin/
Disallow: /magento/
Disallow: /report/
Disallow: /scripts/
Disallow: /shell/
Disallow: /skin/
Disallow: /stats/
Disallow: /api.php
Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /get.php
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /README.txt
Disallow: /RELEASE_NOTES.txt
Disallow: /?dir
Disallow: /?dir=desc
Disallow: /?dir=asc
Disallow: /?limit=all
Disallow: /?mode*
Disallow: /index.php/
Disallow: /?SID=
Disallow: /checkout/
Disallow: /onestepcheckout/
Disallow: /customer/
Disallow: /customer/account/
Disallow: /customer/account/login/
Disallow: /catalogsearch/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /cgi-bin/
Disallow: /cleanup.php
Disallow: /apc.php
Disallow: /memcache.php
Disallow: /phpinfo.php
Disallow: /control/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/
Disallow: /catalog/product/gallery/
Disallow: /?*
Disallow: //colore/
Disallow: //price/
Disallow: //misura/
Disallow: //marca/
Disallow: //sort-by/
Disallow: //combinazione/
Disallow: /*/seleziona-colore/
Disallow: /colore/
Disallow: /price/
Disallow: /misura/
Disallow: /marca/
Disallow: /sort-by/
Disallow: /combinazione/
Disallow: /seleziona-colore/
Disallow: /*colore/
Disallow: /*price/
Disallow: /*misura/
Disallow: /*marca/
Disallow: /*sort-by/
Disallow: /*combinazione/
Disallow: /*seleziona-colore/ UmuEX4z0 -
GWMT - "Tag Site For Child Directed Treatment" Effect On Search / Rankings?
Hi All, We have a client who has been directed to tag their site for "Child Directed Treatment" in Webmaster tools to comply with AdExchange policies. The site is, generally speaking, directed at those between the ages of 13 and 16 along with their parents, but does NOT collect any data (No sign in, login, signups etc). You can find out more about the specific tag here (unfortunately not much more about it) https://support.google.com/webmasters/answer/3221080?hl=en Our concern is that we have never heard of this specific tag, and our client is asking us to let them know if that this will have no effect on search traffic or ranking. I can't find much in the way of anyone who HAS implemented this tag and the effects it has had on their site. They are ad supported, receive millions of unique hits a month, and the majority of their traffic is from branded keywords. Would love to hear from anyone with ANY experience or thoughts on this process and what to be aware of. Your assistance is muchly appreciated.
Search Behavior | | SearchMarketers0 -
Would it be worthwhile to update URLs to include "for sale"
My team is new to the world of SEO. We were both in other departments when our last "expert" left and we became the entire department. Needless to say, we have many questions. I have a site for an existing customer - she wants to know if it is a good idea to change her URLs. She went to a webinar that suggested she include "for sale" in URL, since that is the site's primary purpose These are the current inventory URLs: http://www.dealerskinssite.com/used-inventory http://www.dealerskinssite.com/new-inventory The change would give something like: http://www.dealerskinssite.com/used-cars-for-sale
Search Behavior | | DealerskinsSearch
http://www.dealerskinssite.com/new-cars-for-sale I did some searching and could not find a solid recommendation one way or another. If it is true that people typically search for "used cars for sale" rather than a "used inventory", then there might be benefit to including that as part of the URL... but that is as far as my knowledge got me. Please let me know your thoughts Steve0 -
Two sites with different URLs
I help organize an ecommerce site for a company that is named after the state that the company was founded in (ex: Florida Pipes). Our programmer is thinking about creating a duplicate site that would have a name that was more location agnostic so people shopping on the site would not think that they may be ordering from far away hence incurring extra shipping charges. He said that the site would only have a different URLs, name and homepage but would link to all of the same stuff that is on the site that is up now. He said that there would be no way to tell (possibly for the layperson) that the two sites were related. Is this a good practice? Would we be penalized in search results for having two URLs linking to the same content? (We are a 30 year old company that ranks very high for our main keyword) Thank you for your input!
Search Behavior | | Winoman0 -
How long til meta robots noindex takes effect?
I have a wordpress site with about 3,000 posts and over 1,000 tags. All of the tag archives are currently indexed in Google and I don't want them to be. I just set the meta robots to no-index all the tag archives and was wondering how long it will take til they're out of the search engines? Since there are close to 1,500 of these and they are duplicate content it would be nice to have them gone asap. I noticed Webmaster Tools allows me to resubmit my site to index if my site has changed significantly... should I try that?? Any other advice would be greatly appreciated!
Search Behavior | | gfreeman230 -
Trailing slash at end of URLs?
Hi, I'm just about to put up a new site and I need to decide between having no trailing slash at the end of the all the URLs, or putting one in there. I think Matt Cutts has a slight preference for them, as stated here: http://www.seomoz.org/blog/whiteboard-interview-googles-matt-cutts-on-redirects-trust-more "Matt says he would slightly advocate for using a trailing slash simply because it clearly indicates that a URL is a folder and not a document." However, I'm really wondering. I mean, if people link to my site, they'll tend not to insert a trailing slash, I'm thinking... Your thoughts would be welcome on this one! Cheers, Luke
Search Behavior | | McTaggart0 -
Where can I find research into consumer behavior in URL input?
I have a hypothesis that the % of users that type URLs is decreasing. Instead I think users will be arriving at sites by clicking on links in emails, search and social. Where can I find data / research to back me up?
Search Behavior | | TomCritchlow0