No Index help
-
Hi, I have alot of notices about webpages that are apparently blocked by no index robots, I look through the code on the site and cannot see any coding for this and am confused how to fix the problem. Help anyone?
-
Thanks for your great help Sha!
-
Hi Steven,
There are two different robot related warnings in your SEOmoz PRO App report. The second one refers to pages or directories blocked by a disallow statement in a robots.txt file.
Next step is to check whether you have a robots.txt file in place for your site and if so, then check to see if there are files or directories disallowed in the file.
If the robots.txt file is not an issue, then the your best course of action is to email the SEOmoz Help team direct - help [at] seomoz.org. Give them the details of your campaign and explain the problem and they will take a look at it for you.
Hope that helps,
Sha
-
Thanks for you reply! I have looked at the meta tags and there is no meta noindex tag, this is where i am getting confused on how to make the page an indexed one? I would expect to see either of the meta tags you displayed earlier however they are not there? Thanks
-
Hi Steven,
This message refers to a meta tag which appears in the head statement for the page.
the code looks like this:
OR
This tag tells crawlers not to index the page (or to remove the page from the index if it already appears there) and to either follow or not follow links on the page.
In most cases I would use noindex, follow as this allows any link juice being passed to the page to flow on through the site. The meta noindex tag is the most reliable way to ensure that pages are not indexed since it occurs at the page level. Other methods such as disallowing pages in a robots.txt file can be unreliable as crawlers that come to a page via an external link do not see the disallow statement, so will index the page.
The noindex tags on your pages may have been placed in order to exclude pages that were producing errors or are duplicates of other pages.
Hope that helps,
Sha
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Removing non www and index.php
Hi, I'm green when it comes to altering the htaccess file to remove non www and index.php. I think I've managed to redirect the urls to www however not sure if I've managed to remove the index.php. I'm pasting the contents of the htaccess file here maybe someone can identify if I have unwanted lines of code and if it is up to standard (there are a lot of comments in #) not sure if needed but I've left them as I don't want to screw up anything. Thanks 🙂 @package Joomla @copyright Copyright (C) 2005 - 2016 Open Source Matters. All rights reserved. @license GNU General Public License version 2 or later; see LICENSE.txt READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE! The line 'Options +FollowSymLinks' may cause problems with some server configurations. It is required for the use of mod_rewrite, but it may have already been set by your server administrator in a way that disallows changing it in this .htaccess file. If using it causes your site to produce an error, comment it out (add # to the beginning of the line), reload your site in your browser and test your sef urls. If they work, then it has been set by your server administrator and you do not need to set it here. No directory listings IndexIgnore * Can be commented out if causes errors, see notes above. Options +FollowSymlinks
On-Page Optimization | | KeithBugeja
Options -Indexes Mod_rewrite in use. RewriteEngine On
RewriteCond %{REQUEST_URI} ^/index.php/
RewriteRule ^index.php/(.*) /$1 [R,L] Begin - Rewrite rules to block out some common exploits. If you experience problems on your site then comment out the operations listed below by adding a # to the beginning of the line. This attempts to block the most common type of exploit attempts on Joomla! Block any script trying to base64_encode data within the URL. RewriteCond %{QUERY_STRING} base64_encode[^(]([^)]) [OR] Block any script that includes a0 -
Meta Titles On Site Different From Google Index Page
This is very embarrassing but I hope someone can help. The Meta Titles on my site are not being shown correctly on the Google site index. For example, when I got directly to the page the appropriate page title is shown obviously. However when I go view that page on Google, the title in completely different. Page title: <a class="attribute-value">Web Design, SEO, PPC, Mobile Development In Philadelphia, Bucks| Infinity Digital Agency</a> Google Shows: Our Services - Infinity Digital Agency Here is the result page. I am currently running WordPress and Yoast. Any thoughts would be greatly appreciated. https://www.google.com/search?q=site%3Awww.infinitydigitalagency.com%2Fservices%2F&ie=utf-8&oe=utf-8
On-Page Optimization | | infinitydigitalagency0 -
Indexed Link Removal request in GWT, good idea?
Hello, I used a plugin to no-index a lot of pages on my website, and its been couple months and they never disappeared from serps, so i used the google webmaster tool "remove urls" to ask google to remove them. Is that a good idea? Or does it look bad in googles eyes? any thoughts would help a lot. Thanks
On-Page Optimization | | Rank-and-Grow0 -
Googlebot indexing URL's with ? queries in them. Is this Panda duplicate content?
I feel like I'm being damaged by Panda because of duplicate content as I have seen the Googlebot on my site indexing hundreds of URL's with ?fsdgsgs strings after the .html. They were beign generated by an add-on filtering module on my store, which I have since turned off. Googlebot is still indexing them hours later. At a loss what to do. Since Panda, I have lost a couple of dozen #1 rankings that I've held for months on end and had one drop over 100 positions.
On-Page Optimization | | sparrowdog0 -
Blocking Subdomain from Google Crawl and Index
Hey everybody, how is it going? I have a simple question, that i need answered. I have a main domain, lets call it domain.com. Recently our company will launch a series of promotions for which we will use cname subdomains, i.e try.domain.com, or buy.domain.com. They will serve a commercial objective, nothing more. What is the best way to block such domains from being indexed in Google, also from counting as a subdomain from the domain.com. Robots.txt, No-follow, etc? Hope to hear from you, Best Regards,
On-Page Optimization | | JesusD3 -
Help With Duplicated Content
Hi Moz Community, I am having some issue's with duplicated content, i recently removed the .html from all of our links and moz has reported it as being duplicated. I have been reading up about Canonicalization and would to verify some details, when using the canonical tag would it be placed in the /mywebpage.html or /mywebpage file? I am having a hard time to sort this out so any help from you SEO experts would be great 🙂 I have also updated my htaccess file with the following Thanks in advance
On-Page Optimization | | finelinewebsolutions0 -
Page Indexing
Hello All Nice easy question! I've made some on page changes to page titles, content, H1s etc but wanted to know if there was a way to check if Google has reindexed the page since the changes were made? I appreciate the different factors that will help improve your crawl rate like new content, external links, domain authority etc. I made these changes around 2 weeks ago. Google has cached the pages since I made the changes but not picked up on the new page titles in the search results. Cheers Todd
On-Page Optimization | | todd75850