Skip to content

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  • robots.txt ranking

    hi, can anyone let me know if my robot file is correct. my pages and wordpress posts are being indexed but not showing in serps and wondering if my robot file is wrong

    SEO Tactics | | blogwoman1

  • robots.txt indexation

    I have two good high level DA sites that target the US (.com) and UK ( The .com ranks well but is dormant from a commercial aspect - the is the commercial focus and gets great traffic. Issue is the .com ranks for brand in the UK - I want the to rank for brand in the UK. I can't 301 the .com as it will be used again in the near future. I want to block the .com in Robots.txt with a view to un-block it again when I need it. I don't think the DA would be affected as the links stay and the sites live (just not indexed) so when I unblock it should be fine - HOWEVER - my query is things like organic CTR data that Google records and other factors won't contribute to its value. Has anyone ever blocked and un-blocked and whats the affects pls? All answers greatly received - cheers GB

    Technical SEO | | Bush_JSM
  • Unsolved

    crawl error robots.txt

    Hi all, im facing an issue where moz crawler is unable to crawl my site. The following error keeps showing Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. This is my robots.txt file : I'm not sure what else am I missing.. can anyone help

    Product Support | | teikh
  • Unsolved

    robots.txt crawl disallow subdomain sub-domain

    I need Moz to stop crawling a subdomain of my site, and am just checking what the exact text should be in the file to do this. I assume it would be: User-agent: Moz
    Disallow: / But just checking so I can tell the agency who will apply it, to avoid paying for their time with the incorrect text! Many thanks.

    Getting Started | | Simon-Plan

Looks like your connection to Moz was lost, please wait while we try to reconnect.