Disallow robots on a url effect?
-
Hello, I am using wordpress and on some of my top tag urls e.g : http://www.designzzz.com/tag/brushes/ there is an avg page rank of 4.
I was reading somewhere that my huge jump in not selected count in index status in GWT is becuase of tags/categories urls.
So i added disallow in my robots.txt for the url /tag
What sort of effect will it have on my tag urls rankings or PR ?
-
If your site is ranking "well", well enough for you I mean don't change anything if users are finding there way to what they want. Now for the noindex-follow tag that will not rank the certain pages in the serps but the links on the page will pass pagerank . For the nofollow-noindex or nofollow-index will just hurt the site rankings as they are average pagerank 4 cutting of the power of the posts in the certain category-tag pages not passing any link juice. The best thing to do is leave the pages as is. If the category-tag pages rank better than the posts then i would just noindex them leaving follow on the pages to pass juice towards the posts.
-
but the problem in my case is that for example tags like font, brushes. doesn't have there seperate categories. like font related posting goes under freebies categ and brushes related posts goes under photoshop category . Photoshop categories lots of types of postings like tutorials, downloads, brushes, actions etc.
What i am concerned right now is that if i noindex tag pages. how will they affect my pr and rnakings of these pages?
Or if i nofollow them in meta tag, how will that effect ?
-
The fix i did was to 301 redirect the tag pages to the similar category pages or content pages. This might not be the right thing for your site but I saw new rankings within a day or two. I think this might have lifted some kind of penalty and also siloed the pagerank of the tag pages towards the categories and articles giving them a boost in serps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Dark Traffic & Long URLs - Clarification
Hi everyone, I've been reading the 2017 report by Groupon and the 2017 article by Neil Patel r.e. dark traffic. Both of these articles work on the assumption that most long URLs are not direct traffic because people wouldn't type a long URL into their browser. However, what happens to me personally all the time is that I start typing a URL into the browser, and the browser brings up a list of pages I've visited recently, and I click on some super long URL that I didn't bookmark but have visited in the past. That is legitimate direct traffic, but it's a long URL. I'm just wondering if there's something flawed in my reasoning or in the reasoning of Patel and Groupon. Maybe most people aren't relying on browsers like I am, or maybe things have changed a lot in the past 3 years. What do you think? And are there any more recent resources/articles that you would recommend r.e. trying to parse out dark traffic? https://neilpatel.com/blog/dark-traffic-stealing-data/ Thanks!
Search Behavior | | LivDetrick0 -
Is there a tool that will find a string from the source code from a list of URLs?
I need a tool that will run through a long list of URLs (from multiple domains), look for a specific string in the source code and then produce a CSV showing in which URLs the the string was found. This is for legitimate research, not for any nefarious purposes Any ideas of such a tool already exists?
Search Behavior | | richdan0 -
Direct 100+ URLs to our sister site?
Hi, We have two websites www.eurekasolutions.co.uk and www.eurekaaddons.co.uk - currently the eurekasolutions site has eureka addon products on it - we are thinking of re-directing these to www.eurekaaddons.co.uk instead (there are 100+ pages which are being ranked and picked up by google) Would it be foolish to re-direct these from eurekasolutions straight to eurekaaddons after them being on eurekasolutions for many many years? if so what is the best way to go about it and would this have a direct negative effect on our SEO? Both sites are serving for similar terms. Thanks,
Search Behavior | | EurekaSolutions0 -
Help with robots.txt on Magento
Hi everybody, I need your help in order to fix some problems with HTML errors and Crawling errors generated by Magento on my client's website www.casabiancheria.it I have some problems with duplicate meta informations due to the fact that there are a lot of links such as /stampe-romagnole/tovaglie-con-tovaglioli**/colore/**beige,marrone,giallo,lilla/show/all.html /stampe-romagnole/tovaglie-con-tovaglioli**/colore/**beige,marrone,lilla/show/all.html that are generated by the filter /colore/ and so they have duplicate content and meta information on them. I activated the canonicals on Magento but this hasn't fixed the problem yet. On the sitemap there are only 1 link for each product, so it seems that the canonicals are working, but bot Google Webmaster Tools and SEO Moz are giving me errors on duplicate content and meta informations. I would like to solve these problems by excluding from robots.txt all the urls that contain the filter parameters, such as /colore/, /price/, /dimensions/, etc. (take a look to the attachment). I tried different solutions in order to exclude these links from robots, but I wasn't able to succeed. Below you can find my current robots.txt... can someone help me in order to write the correct form of this file and finally exclude all these urls generated by filters on Magento? Finally, is it worth it to exclude also the images from Magento? (take a look to the final lines of the robots below). Thank you very much for your help! Alberto User-agent: *
Search Behavior | | OptimizedGroup
Disallow: /CVS
Disallow: /.svn$
Disallow: /.idea$
Disallow: /.sql$
Disallow: /.tgz$
Disallow: /w1nL1f3L0g1c/
Disallow: /app/
Disallow: /downloader/
Disallow: /errors/
Disallow: /includes/
Disallow: /lib/
Disallow: /pkginfo/
Disallow: /shell/
Disallow: /var/
Disallow: /404/
Disallow: /cgi-bin/
Disallow: /magento/
Disallow: /report/
Disallow: /scripts/
Disallow: /shell/
Disallow: /skin/
Disallow: /stats/
Disallow: /api.php
Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /get.php
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /README.txt
Disallow: /RELEASE_NOTES.txt
Disallow: /?dir
Disallow: /?dir=desc
Disallow: /?dir=asc
Disallow: /?limit=all
Disallow: /?mode*
Disallow: /index.php/
Disallow: /?SID=
Disallow: /checkout/
Disallow: /onestepcheckout/
Disallow: /customer/
Disallow: /customer/account/
Disallow: /customer/account/login/
Disallow: /catalogsearch/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /cgi-bin/
Disallow: /cleanup.php
Disallow: /apc.php
Disallow: /memcache.php
Disallow: /phpinfo.php
Disallow: /control/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/
Disallow: /catalog/product/gallery/
Disallow: /?*
Disallow: //colore/
Disallow: //price/
Disallow: //misura/
Disallow: //marca/
Disallow: //sort-by/
Disallow: //combinazione/
Disallow: /*/seleziona-colore/
Disallow: /colore/
Disallow: /price/
Disallow: /misura/
Disallow: /marca/
Disallow: /sort-by/
Disallow: /combinazione/
Disallow: /seleziona-colore/
Disallow: /*colore/
Disallow: /*price/
Disallow: /*misura/
Disallow: /*marca/
Disallow: /*sort-by/
Disallow: /*combinazione/
Disallow: /*seleziona-colore/ UmuEX4z0 -
Tracking forwarding / vanity URLS
Buongiorno from 18 Degrees sunny wetherby UK 🙂 Am i right in thinking you can track a forwarding urls via Google analytics. Put another way if a site points www.kissmymelons.co.uk to www.lickmybanana.co.uk can you measure how many referrals where generated from kissmymelons.co.uk I'm 69% sure the referral data will appear in the referral source section in Google analytics but I need to be 100% sure 😉 Grazie tanto,
Search Behavior | | Nightwing
David0 -
Would it be worthwhile to update URLs to include "for sale"
My team is new to the world of SEO. We were both in other departments when our last "expert" left and we became the entire department. Needless to say, we have many questions. I have a site for an existing customer - she wants to know if it is a good idea to change her URLs. She went to a webinar that suggested she include "for sale" in URL, since that is the site's primary purpose These are the current inventory URLs: http://www.dealerskinssite.com/used-inventory http://www.dealerskinssite.com/new-inventory The change would give something like: http://www.dealerskinssite.com/used-cars-for-sale
Search Behavior | | DealerskinsSearch
http://www.dealerskinssite.com/new-cars-for-sale I did some searching and could not find a solid recommendation one way or another. If it is true that people typically search for "used cars for sale" rather than a "used inventory", then there might be benefit to including that as part of the URL... but that is as far as my knowledge got me. Please let me know your thoughts Steve0 -
Two sites with different URLs
I help organize an ecommerce site for a company that is named after the state that the company was founded in (ex: Florida Pipes). Our programmer is thinking about creating a duplicate site that would have a name that was more location agnostic so people shopping on the site would not think that they may be ordering from far away hence incurring extra shipping charges. He said that the site would only have a different URLs, name and homepage but would link to all of the same stuff that is on the site that is up now. He said that there would be no way to tell (possibly for the layperson) that the two sites were related. Is this a good practice? Would we be penalized in search results for having two URLs linking to the same content? (We are a 30 year old company that ranks very high for our main keyword) Thank you for your input!
Search Behavior | | Winoman0 -
How long til meta robots noindex takes effect?
I have a wordpress site with about 3,000 posts and over 1,000 tags. All of the tag archives are currently indexed in Google and I don't want them to be. I just set the meta robots to no-index all the tag archives and was wondering how long it will take til they're out of the search engines? Since there are close to 1,500 of these and they are duplicate content it would be nice to have them gone asap. I noticed Webmaster Tools allows me to resubmit my site to index if my site has changed significantly... should I try that?? Any other advice would be greatly appreciated!
Search Behavior | | gfreeman230