The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Forum
    2. Categories
    3. SEO Tactics
    4. Intermediate & Advanced SEO
    5. Disallow URLs ENDING with certain values in robots.txt?

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    Disallow URLs ENDING with certain values in robots.txt?

    Intermediate & Advanced SEO
    4 2 2.6k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jmorehouse
      jmorehouse last edited by

      Is there any way to disallow URLs ending in a certain value? For example, if I have the following product page URL: http://website.com/category/product1, and I want to disallow /category/product1/review, /category/product2/review, etc. without disallowing the product pages themselves, is there any shortcut to do this, or must I disallow each gallery page individually?

      1 Reply Last reply Reply Quote 0
      • Andy.Drinkwater
        Andy.Drinkwater @jmorehouse last edited by

        Excellent stuff. Glad it helped 🙂

        -Andy

        1 Reply Last reply Reply Quote 0
        • jmorehouse
          jmorehouse @Andy.Drinkwater last edited by

          Thanks, Andy! I just tested with Webmaster tool's tester, and it worked.

          Andy.Drinkwater 1 Reply Last reply Reply Quote 1
          • Andy.Drinkwater
            Andy.Drinkwater last edited by

            So you just want to disallow the /review/ element?

            *Disallow: /review/

            I am pretty sure the wildcard will work.

            I would use the Robots.txt tester in Webmaster tools to try it out first before committing to any changes. This will tell you if it works and if so, if it's blocked successfully.

            You then try the URL without the /review/ element on and make sure it passes.

            -Andy

            jmorehouse 1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post

            Got a burning SEO question?

            Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


            Start my free trial


            Explore more categories

            • Moz Tools

              Chat with the community about the Moz tools.

              Getting Started
              Moz Pro
              Moz Local
              Moz Bar
              API
              What's New

            • SEO Tactics

              Discuss the SEO process with fellow marketers

              Content Development
              Competitive Research
              Keyword Research
              Link Building
              On-Page Optimization
              Technical SEO
              Reporting & Analytics
              Intermediate & Advanced SEO
              Image & Video Optimization
              International SEO
              Local SEO

            • Community

              Discuss industry events, jobs, and news!

              Moz Blog
              Moz News
              Industry News
              Jobs and Opportunities
              SEO Learn Center
              Whiteboard Friday

            • Digital Marketing

              Chat about tactics outside of SEO

              Affiliate Marketing
              Branding
              Conversion Rate Optimization
              Web Design
              Paid Search Marketing
              Social Media

            • Research & Trends

              Dive into research and trends in the search industry.

              SERP Trends
              Search Behavior
              Algorithm Updates
              White Hat / Black Hat SEO
              Other SEO Tools

            • Support

              Connect on product support and feature requests.

              Product Support
              Feature Requests
              Participate in User Research

            • See all categories

            • Block session id URLs with robots.txt
              Mat_C
              Mat_C
              1
              4
              972

            • Robots.txt & Disallow: /*? Question!
              vetofunk
              vetofunk
              0
              6
              1.9k

            • Will disallowing URL's in the robots.txt file stop those URL's being indexed by Google
              andyheath
              andyheath
              0
              10
              3.2k

            • Should comments and feeds be disallowed in robots.txt?
              workathomecareers
              workathomecareers
              0
              5
              7.1k

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy

            Looks like your connection to Moz was lost, please wait while we try to reconnect.