Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Right way to block google robots from ppc landing pages
-
What is the right way to completely block seo robots from my adword landing pages? Robots.txt does not work really good for that, as far I know.
Adding metatags noindex nofollow on the other side will block adwords robot as well. right?
Thank you very much,
Serge
-
Thank you very much
-
Gotcha. I did some searching around and you will not block the AdWords bot unless you explicitly block AdsBot-Google. A wildcard user agent disallow will not block the AdsBot-Google. Hope that helps!
-
Thanks guys. AdWords robots scans the page to determine it's relevancy to your ad group.
When you block it in robots.txt ONLY, the link to this page will be indexed and show up in your serp results site:example.com
I wondered if you add meta tags noindex nofollow you also blocking all robots to scan the page, adwords as well
-
If you have a specific directory for PPC pages then you can follow these steps that have worked for us wonderfully:
Block the directory /ppc to the robots.txt file for all user agents. This can even be live before the /ppc directory even exists.
User-agent: *
Disallow: /ppc
Add noindex, nofollow in the meta tags for all pages in /ppc
I'm not sure what you are referring to when you mention an AdWords robot?
-
Dear Serge,
Your question is difficulty to answer, because you have several possibilities. If you use the noindex, will stop all the robots.
There is a post at SEOMoz blog writed by Lindsay that i think will answer your question. You will find it here: http://www.seomoz.org/blog/robot-access-indexation-restriction-techniques-avoiding-conflicts
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Optimization Error
Hi, I am trying to track a page optimization feature for one of my project, https://shinaweb.com but i keep getting this below error: "PAGE OPTIMIZATION ERROR
On-Page Optimization | | shinawebnavid
There was a problem loading this page. Please make sure the page is loading properly and that our user-agent, rogerbot, is not blocked from accessing this page." I checked robots.txt file, it all looks fine. Not sure what is the problem? Is it a problem with Moz or the website?0 -
Landing page separate from product page
Hello there, I have a wordpress website with a woocommerce plugin. I have 4 landing pages that describe my products and at the end of the pages, I have a CTA to my product page. is it bad for SEO? my website: https://relationadviser.ir
On-Page Optimization | | Aaron.be1 -
Will Google Count Links Loaded from JavaScript Files After the Page Loads
Hi, I have a simple question. If I want to put an image with a link to another site like a banner ad on my page, but do not want it counted by Google. Can I simply load the link and banner using jQuery onload from a separate .js file? The ideal result would be for Google to index a script tag instead of a link.
On-Page Optimization | | CopBlaster.com1 -
Getting the Titles and Headings Right on Product Pages. Userbility vs SEO
Hey Mozzers, I am optimising a chaotic section of the site including many similar products. Writing unique content etc. The titles and urls were all over the place so my first job was to tidy them up so I could make some sense of the situation, especially as sometimes they didn't even match! I should point out were on Magento, so product name = Both the Heading and Title of the page, the meta title can be set separately. When i refer to title I mean both <title>and <h1></strong><br /><br />Before they existed as such<br />URL: domain.com/200-x-0-5-g-rs-232-balance.html<br />TItle: PC-1234 200 x 0.5g x 0.3 RS-232 Balance<br /><br />This format was (Product Code, Capacities, Resolutions, Accuracy, Product Title)<br /><br />The issue was all 60 products in a page followed this format. Navigating through the page was a nightmare and was just a jumble of numbers and highly confusing even to me who learnt what they all mean, especially when you had 8 products from the same range you got presented with<br /><br />APC-1234 200 x 0.5g x 0.3 RS-232 Balance<br />APC-1235 500 x 1g x 0.3 RS-232 Balance<br />APC-1236 1000 x 2g x 0.3 RS-232 Balance<br />APC-1238 5000 x 10g x 0.3 RS-232 Balance<br />APC-1239 10000 x 15g x 0.3 RS-232 Balance<br />APC-1210 20000 x 25g x 0.3 RS-232 Balance<br />APC-1211 50000 x 50g x 0.3 RS-232 Balance</p> <p>I changed them to something more user friendly.<br /><br />URL: domain.com/200g-precision-balance.html<br />Title: 200g Precision Balance<br /><br />This has seen the following benefits<br />- URL is now clear and means something to the user<br />- Product titles are easy to navigate and the page is more pleasing to the eye<br />- The jumble of numbers in the title are now all labelled and shown below each product listing in bullet point so the user can see the basic spec of a product without having to decipher any titles<br /><br />Upon reflection I has a couple of concerns I was hoping you could discuss, I am wondering if I have made the titles too simple.<br />1) I have no product code in the title<br />We have our own products manufactured and sell existing brands with their own product codes. Some of these can be lengthy. Adding them makes them hard to the eye and the page looked cramped.<br /><br />The codes are listed beneath each product title on category pages and on a list on the actual product page, but no where in the titles. <br /><br />2)None of our products have a brand listed in the title<br />None of the products on the site had brand names in anything but the images when i started and as such it snuck under my radar. But should i pre-fix all titles with a brand name?<br /><br />Should </p> <p>URL: domain.com/200g-precision-balance.html<br />Title: 200g Precision Balance</p> <p>become</p> <p>URL: domain.com/BRAND1-200g-precision-balance.html<br />Title: BRAND1 200g Precision Balance<br /><br />My instinct tells me to change things to include brands as its useful to the customer and should have an SEO benefit, but to leave out product codes as they are accessible to the customer where they are now and dont make things messy and unreadable.<br /><br />As always, thanks for the input!</p></title>
On-Page Optimization | | ATP0 -
Why is Google replacing my meta title with the business name on home page?
For all queries that return the home page, Google is not showing my meta title. Instead it replaced it with the official business name which of course makes it harder to rank for key terms since they don't exist now in the meta title. You can see this is you search on "mt view estate planning attorney". The site in question is dureelaw.com and the title showing is "The Law Office of Daniel L. DuRee." View the source and you'll see my meta title. Why is Google substituting it?
On-Page Optimization | | katandmouse0 -
How do i block an entire category/directory with robots.txt?
Anyone has any idea how to block an entire product category, including all the products in that category using the robots.txt file? I'm using woocommerce in wordpress and i'd like to prevent bots from crawling every single one of products urls for now. The confusing part right now is that i have several different url structures linking to every single one of my products for example www.mystore.com/all-products, www.mystore.com/product-category, etc etc. I'm not really sure how i'd type it into the robots.txt file, or where to place the file. any help would be appreciated thanks
On-Page Optimization | | bricerhodes0 -
Landing Pages: New Domain or Sub Folder?
I use premise for landing pages. I have some extra domain names that are fantastic in my industry. I'm wondering if I should use those domains for these landing pages? The header, nav, footer, would be the same as my main site, the body and content would be totally different. will google penalize me if I have the same header and footer on a landing page?
On-Page Optimization | | homebizsmart0 -
Avoiding "Duplicate Page Title" and "Duplicate Page Content" - Best Practices?
We have a website with a searchable database of recipes. You can search the database using an online form with dropdown options for: Course (starter, main, salad, etc)
On-Page Optimization | | smaavie
Cooking Method (fry, bake, boil, steam, etc)
Preparation Time (Under 30 min, 30min to 1 hour, Over 1 hour) Here are some examples of how URLs may look when searching for a recipe: find-a-recipe.php?course=starter
find-a-recipe.php?course=main&preperation-time=30min+to+1+hour
find-a-recipe.php?cooking-method=fry&preperation-time=over+1+hour There is also pagination of search results, so the URL could also have the variable "start", e.g. find-a-recipe.php?course=salad&start=30 There can be any combination of these variables, meaning there are hundreds of possible search results URL variations. This all works well on the site, however it gives multiple "Duplicate Page Title" and "Duplicate Page Content" errors when crawled by SEOmoz. I've seached online and found several possible solutions for this, such as: Setting canonical tag Adding these URL variables to Google Webmasters to tell Google to ignore them Change the Title tag in the head dynamically based on what URL variables are present However I am not sure which of these would be best. As far as I can tell the canonical tag should be used when you have the same page available at two seperate URLs, but this isn't the case here as the search results are always different. Adding these URL variables to Google webmasters won't fix the problem in other search engines, and will presumably continue to get these errors in our SEOmoz crawl reports. Changing the title tag each time can lead to very long title tags, and it doesn't address the problem of duplicate page content. I had hoped there would be a standard solution for problems like this, as I imagine others will have come across this before, but I cannot find the ideal solution. Any help would be much appreciated. Kind Regards5
