The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Forum
    2. Categories
    3. SEO Tactics
    4. Technical SEO
    5. Do I need a separate robots.txt file for my shop subdomain?
    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    Do I need a separate robots.txt file for my shop subdomain?

    Technical SEO
    6 4 1.8k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • sjbridle
      sjbridle last edited by

      Hello Mozzers!

      Apologies if this question has been asked before, but I couldn't find an answer so here goes...

      Currently I have one robots.txt file hosted at https://www.mysitename.org.uk/robots.txt

      We host our shop on a separate subdomain https://shop.mysitename.org.uk

      Do I need a separate robots.txt file for my subdomain? (Some Google searches are telling me yes and some no and I've become awfully confused!

      1 Reply Last reply Reply Quote 0
      • sjbridle
        sjbridle @eyepaq last edited by

        Thank you. I want to disallow specific URLs on the subdomain and add the shop sitemap in the robots.txt file. So I'll go ahead and create another!

        1 Reply Last reply Reply Quote 0
        • eyepaq
          eyepaq @sjbridle last edited by

          You go be fine without one. You only need one if you want to manage that subdmain: add specific xml sitemaps links in robots.txt, cut access to specific folders for that subdomain.

          if you don't need any of that - just move forward without one.

          sjbridle 1 Reply Last reply Reply Quote 2
          • sjbridle
            sjbridle @Martijn_Scheijbeler last edited by

            Currently we just have: User-agent: *

            I'm in the process of optimising.

            eyepaq 1 Reply Last reply Reply Quote 0
            • Martijn_Scheijbeler
              Martijn_Scheijbeler last edited by

              It depends what currently is in your robots.txt. Usually it would be useful to have another one for your subdomain.

              sjbridle 1 Reply Last reply Reply Quote 1
              • ThomasHarvey
                ThomasHarvey last edited by

                Yes, I would have a seperate robots.txt files.

                1 Reply Last reply Reply Quote 1
                • 1 / 1
                • First post
                  Last post

                Got a burning SEO question?

                Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                Start my free trial


                Explore more categories

                • Moz Tools

                  Chat with the community about the Moz tools.

                  Getting Started
                  Moz Pro
                  Moz Local
                  Moz Bar
                  API
                  What's New

                • SEO Tactics

                  Discuss the SEO process with fellow marketers

                  Content Development
                  Competitive Research
                  Keyword Research
                  Link Building
                  On-Page Optimization
                  Technical SEO
                  Reporting & Analytics
                  Intermediate & Advanced SEO
                  Image & Video Optimization
                  International SEO
                  Local SEO

                • Community

                  Discuss industry events, jobs, and news!

                  Moz Blog
                  Moz News
                  Industry News
                  Jobs and Opportunities
                  SEO Learn Center
                  Whiteboard Friday

                • Digital Marketing

                  Chat about tactics outside of SEO

                  Affiliate Marketing
                  Branding
                  Conversion Rate Optimization
                  Web Design
                  Paid Search Marketing
                  Social Media

                • Research & Trends

                  Dive into research and trends in the search industry.

                  SERP Trends
                  Search Behavior
                  Algorithm Updates
                  White Hat / Black Hat SEO
                  Other SEO Tools

                • Support

                  Connect on product support and feature requests.

                  Product Support
                  Feature Requests
                  Participate in User Research

                • See all categories

                • Multiple robots.txt files on server
                  mjukhud
                  mjukhud
                  0
                  7
                  4.0k

                • Robots.txt on subdomains
                  Whittie
                  Whittie
                  0
                  3
                  4.5k

                • Does Bing ignore robots txt files?
                  Nightwing
                  Nightwing
                  0
                  3
                  4.2k

                • Subdomain Removal in Robots.txt with Conditional Logic??
                  ErnieB
                  ErnieB
                  0
                  3
                  2.6k

                • Robots.txt File Redirects to Home Page
                  kchandler
                  kchandler
                  0
                  6
                  6.5k

                Get started with Moz Pro!

                Unlock the power of advanced SEO tools and data-driven insights.

                Start my free trial
                Products
                • Moz Pro
                • Moz Local
                • Moz API
                • Moz Data
                • STAT
                • Product Updates
                Moz Solutions
                • SMB Solutions
                • Agency Solutions
                • Enterprise Solutions
                • Digital Marketers
                Free SEO Tools
                • Domain Authority Checker
                • Link Explorer
                • Keyword Explorer
                • Competitive Research
                • Brand Authority Checker
                • Local Citation Checker
                • MozBar Extension
                • MozCast
                Resources
                • Blog
                • SEO Learning Center
                • Help Hub
                • Beginner's Guide to SEO
                • How-to Guides
                • Moz Academy
                • API Docs
                About Moz
                • About
                • Team
                • Careers
                • Contact
                Why Moz
                • Case Studies
                • Testimonials
                Get Involved
                • Become an Affiliate
                • MozCon
                • Webinars
                • Practical Marketer Series
                • MozPod
                Connect with us

                Contact the Help team

                Join our newsletter
                Moz logo
                © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                • Accessibility
                • Terms of Use
                • Privacy

                Looks like your connection to Moz was lost, please wait while we try to reconnect.