Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

How to disallow google and roger?

Technical SEO

1427

iFix last edited by

Hey Guys and girls,

i have a question, i want to disallow all robots from accessing a certain root link:

Get rid of bots

User-agent: *

Disallow: /index.php?_a=login&redir=/index.php?_a=tellafriend%26productId=*

Will this make the bots not to access any web link that has the prefix you see before the asterisk? And at least google and roger will get away by reading "user-agent: *"? I know this isn't the standard proceedure but if it works for google and seomoz bot we are good.
1 Reply Last reply
Reply Quote 0
RyanKent last edited by

In short, yes that will work.

To be clear, you can have multiple links which all lead to the same target page. The target of that link could still be indexed if there are other unblocked links which point to it.
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Google Cache issue

Hi, We’ve got a really specific issue – we have an SEO team in-house, and have had numerous agencies look at this – but no one can get to the bottom of this. We’re a UK travel company with a number of great positions on the search engines – our brand is www.jet2holidays.com. If you try ‘Majorca holidays’, ‘tenerife holidays’, ‘gran canaria holidays’ etc you’ll see us in the top few positions on Google when searching from the UK. However, none of our destination pages (and it’s only the destination pages), show a ‘cached’ option next to them. Example: https://www.google.com/search?q=majorca+holidays&oq=majorca+holidays&aqs=chrome..69i57j69i60l3.2151j0j9&sourceid=chrome&ie=UTF-8 This isn’t affecting our rankings, but we’re fairly certain it is affecting our ability to be included in the Featured Snippets. Checked and there aren’t any noarchive tags on the pages, example: https://www.jet2holidays.com/destinations/balearics/majorca Anyone have any ideas?
Technical SEO | | fredgray

0
I disappeared from Google, but not Bing?

A few weeks ago the company I work for created a website called Nabceptraining.com in hopes to find clientele who are looking to become NABCEP Certified. So we installed WordPress and a nice looking theme that we edited, and we saw that without any real content on the page we were already rank 12 on Google! We never even submitted our site. So we decided to Search Engine Optimize our website to gain even better rankings. So on the first day we added an SEO Tool that would allow us to change the page titles and descriptions on our site, a Sitemap generation tool so we could submit the site to Google, Bing, and Yahoo. At the end of the day we submitted the site and was beginning to create our content. That next morning our ranking was gone and we were not to be found on Google, but we were rank 3 on Bing!? What happened? Why did we disappear? We didn't receive any messages from Google on the webmaster tools saying we were blacklisted. Does anyone have an idea?
Technical SEO | | edlinkim

0
Titles in google serps incorrect

If you do the following query in google: site:orlandovisiting.com legend sparrow You'll see in the results the title appears as: legend-captain-jack-sparrow-1149.html Where the title tag in the source is: <title>The Legend of Captain Jack Sparrow Opens at Disney’s Hollywood Studiostitle> Eventually it normally rights itself, but does anyone else get this with their sites? Thanks.
Technical SEO | | walshy99

0
A site is not being indexed by Google Yahoo or Bing

This site - http://adoptionconnection.org/ is not being indexed by any of the search engines. I checked the easy stuff - robots text is: <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">all, index, follow</a>" /> <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noodp</a>" /> <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noydir</a>" /> I have checked what I can determine would cause the issue but have found nothing to prevent it from being indexed. I'm thinking it may be re-directs etc. Any answer would be great. Thanks in advance,
Technical SEO | | Intergen

0
Google plus

" With a single Google search, you can see regular search results, along with all sorts of results that are tailored to you -- pages shared with you by your friends, Google+ posts from people you know" Would i be able to see my own post which i shared with someone in my Google plus circle, when i do a search ?
Technical SEO | | seoug_2005

0
Adding Google +1 To My Site

This may seem like a silly question, but I had my site designed by a third party and I don't know how to find this info. I want to add Google +1 button to my site. I've already added the script that makes it work in my , but don't know how to add the to the part of my site I'd like the icon to appear. If you take a look at my site, http://www.youdrivethesuccess.com/, you will see the twitter and facebook icons that follow you around the site in the top right of the page. I'd like to drop in the +1 right next to them, but I don't know how to get to it. How do I access this on the server? What section of my FTP do these lil icons live? A folder? Any direction is greatly appreciated...thanks.
Technical SEO | | brentmitchell

0
Why is Google only indexing 3 of 8 pages?

Hi everyone, I have a small 8 page website I launched about 6 months ago. For the life of me I can not figure out why google is only indexing 3 of the 8 pages. The pages are not duplicate content in any way. I have good internal linking structure. At this time I dont have many inbound links from others, that will come in time. Am I missing something here? Can someone give me a clue? Thanks Tim Site: www.jparizonaweddingvideos.com
Technical SEO | | fasctimseo

0
Some site pages are removed from Google Index

Hello, Some pages of my clients website are removed from Google Index. We were in top 10 position for some keywords but now I cannot find those pages neither in top 1000. Any idea what to do in order to get these pages back? thank you
Technical SEO | | besartbajrami

0