Meta-robots Nofollow on logins and admins
-
In my SEO MOZ reports I am getting over 400 errors as Meta-robots Nofollow. These are all leading to my admin login page which I do not want robots in. Should I put some code on these pages so the robots know this and don't attempt to and I do not get these errors in my reports?
-
I think that's probably fine - we're warning you just to tell you the pages are blocked, but blocking them sounds perfectly sensible. Is Google indexing them at all?
I assume that the login page is getting multiple URLs, possibly based on the path people take to it? Ideally, it would help to collapse that to one URL (store the parameters in a form field or visitor session/cookie), but my gut reaction is that that's a nice to have. I suspect you're probably ok.
-
Yes you can noindex that page to prevent robots from entering. Go to your robot.txt file and add "disallow with your URL. http://www.robotstxt.org/robotstxt.html
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Role of Robots.txt and Search Console parameters settings
Hi, wondering if anyone can point me to resources or explain the difference between these two. If a site has url parameters disallowed in Robots.txt is it redundant to edit settings in Search Console parameters to anything other than "Let Googlebot Decide"?
Technical SEO | | LivDetrick0 -
Blocking Affiliate Links via robots.txt
Hi, I work with a client who has a large affiliate network pointing to their domain which is a large part of their inbound marketing strategy. All of these links point to a subdomain of affiliates.example.com, which then redirects the links through a 301 redirect to the relevant target page for the link. These links have been showing up in Webmaster Tools as top linking domains and also in the latest downloaded links reports. To follow guidelines and ensure that these links aren't counted by Google for either positive or negative impact on the site, we have added a block on the robots.txt of the affiliates.example.com subdomain, blocking search engines from crawling the full subddomain. The robots.txt file is the following code: User-agent: * Disallow: / We have authenticated the subdomain with Google Webmaster Tools and made certain that Google can reach and read the robots.txt file. We know they are being blocked from reading the affiliates subdomain. However, we added this affiliates subdomain block a few weeks ago to the robots.txt, but links are still showing up in the latest downloads report as first being discovered after we added the block. It's been a few weeks already, and we want to make sure that the block was implemented properly and that these links aren't being used to negatively impact the site. Any suggestions or clarification would be helpful - if the subdomain is being blocked for the search engines, why are the search engines following the links and reporting them in the www.example.com subdomain GWMT account as latest links. And if the block is implemented properly, will the total number of links pointing to our site as reported in the links to your site section be reduced, or does this not have an impact on that figure?From a development standpoint, it's a much easier fix for us to adjust the robots.txt file than to change the affiliate linking connection from a 301 to a 302, which is why we decided to go with this option.Any help you can offer will be greatly appreciated.Thanks,Mark
Technical SEO | | Mark_Ginsberg0 -
Should I nofollow Geo-located links on a site?
I run various sites that use Geo-location to place related links in navigation menus on a page. For example, if you land on the home page, we will see that you are in Florida and then in one of the content boxes on the page, show job listings that this site has in Florida. We also give the option to search for other jobs or use other navigation options. The idea is to try to help the user along the best we can, but ..... What opinions do persons have here on if these links should be nofollowed as GoogleBot will always see links to places in California etc. - wherever Googlebot is crawling from? Would this then be confusing as we are a site that focused on the entire US and not just California etc Thanks!
Technical SEO | | CleverPhD0 -
Deal with links that need login to view
Hi All, Deal with links that need login to view We have member names in the site in many places and when clicked it takes the user to the login page As just logged in members can view the details The redirection type is 302 and Moz Campaign says we have many and need to make them 301 What is the best way as we have a drupal website Thanks
Technical SEO | | mtthompsons0 -
Robots.txt
www.mywebsite.com**/details/**home-to-mome-4596 www.mywebsite.com**/details/**home-moving-4599 www.mywebsite.com**/details/**1-bedroom-apartment-4601 www.mywebsite.com**/details/**4-bedroom-apartment-4612 We have so many pages like this, we do not want to Google crawl this pages So we added the following code to Robots.txt User-agent: Googlebot Disallow: /details/ This code is correct?
Technical SEO | | iskq0 -
Google truncating or altering meta title - affect rankings?
I have a site that the title tag is too long and the title is simply the name of the site (I think they get it from ODP, not sure) Anyway, the rankings for the home page have dropped quite a bit. I'm wondering if the change that Google makes affects rankings (i.e. name of site doesn't have all the keywords).
Technical SEO | | santiago230 -
How many times robots.txt gets visited by crawlers, especially Google?
Hi, Do you know if there's any way to track how often robots.txt file has been crawled? I know we can check when is the latest downloaded from webmaster tool, but I actually want to know if they download every time crawlers visit any page on the site (e.g. hundreds of thousands of times every day), or less. thanks...
Technical SEO | | linklater0