Skip to content
    Moz logo Menu open Menu close
    • Products
      • Moz Pro
      • Moz Pro Home
      • Moz Local
      • Moz Local Home
      • STAT
      • Moz API
      • Moz API Home
      • Compare SEO Products
      • Moz Data
    • Free SEO Tools
      • Domain Analysis
      • Keyword Explorer
      • Link Explorer
      • Competitive Research
      • MozBar
      • More Free SEO Tools
    • Learn SEO
      • Beginner's Guide to SEO
      • SEO Learning Center
      • Moz Academy
      • SEO Q&A
      • Webinars, Whitepapers, & Guides
    • Blog
    • Why Moz
      • Agency Solutions
      • Enterprise Solutions
      • Small Business Solutions
      • Case Studies
      • The Moz Story
      • New Releases
    • Log in
    • Log out
    • Products
      • Moz Pro

        Your all-in-one suite of SEO essentials.

      • Moz Local

        Raise your local SEO visibility with complete local SEO management.

      • STAT

        SERP tracking and analytics for enterprise SEO experts.

      • Moz API

        Power your SEO with our index of over 44 trillion links.

      • Compare SEO Products

        See which Moz SEO solution best meets your business needs.

      • Moz Data

        Power your SEO strategy & AI models with custom data solutions.

      NEW Keyword Suggestions by Topic
      Moz Pro

      NEW Keyword Suggestions by Topic

      Learn more
    • Free SEO Tools
      • Domain Analysis

        Get top competitive SEO metrics like DA, top pages and more.

      • Keyword Explorer

        Find traffic-driving keywords with our 1.25 billion+ keyword index.

      • Link Explorer

        Explore over 40 trillion links for powerful backlink data.

      • Competitive Research

        Uncover valuable insights on your organic search competitors.

      • MozBar

        See top SEO metrics for free as you browse the web.

      • More Free SEO Tools

        Explore all the free SEO tools Moz has to offer.

      What is your Brand Authority?
      Moz

      What is your Brand Authority?

      Check yours now
    • Learn SEO
      • Beginner's Guide to SEO

        The #1 most popular introduction to SEO, trusted by millions.

      • SEO Learning Center

        Broaden your knowledge with SEO resources for all skill levels.

      • On-Demand Webinars

        Learn modern SEO best practices from industry experts.

      • How-To Guides

        Step-by-step guides to search success from the authority on SEO.

      • Moz Academy

        Upskill and get certified with on-demand courses & certifications.

      • SEO Q&A

        Insights & discussions from an SEO community of 500,000+.

      Unlock flexible pricing & new endpoints
      Moz API

      Unlock flexible pricing & new endpoints

      Find your plan
    • Blog
    • Why Moz
      • Small Business Solutions

        Uncover insights to make smarter marketing decisions in less time.

      • Agency Solutions

        Earn & keep valuable clients with unparalleled data & insights.

      • Enterprise Solutions

        Gain a competitive edge in the ever-changing world of search.

      • The Moz Story

        Moz was the first & remains the most trusted SEO company.

      • Case Studies

        Explore how Moz drives ROI with a proven track record of success.

      • New Releases

        Get the scoop on the latest and greatest from Moz.

      Surface actionable competitive intel
      New Feature

      Surface actionable competitive intel

      Learn More
    • Log in
      • Moz Pro
      • Moz Local
      • Moz Local Dashboard
      • Moz API
      • Moz API Dashboard
      • Moz Academy
    • Avatar
      • Moz Home
      • Notifications
      • Account & Billing
      • Manage Users
      • Community Profile
      • My Q&A
      • My Videos
      • Log Out

    The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. Home
    2. SEO Tactics
    3. How can I make a list of all URLs indexed by Google?

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    How can I make a list of all URLs indexed by Google?

    SEO Tactics
    redirect indexed urls google url index google broken links
    6
    7
    4524
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with question management privileges can see it.
    • aplusnetsolutions
      aplusnetsolutions Subscriber last edited by

      I have a large site with over 6000 pages indexed but only 600 actual pages and need to clean up with 301 redirects. Haven't had this need since Google stopped displaying the url's in the results.

      BlueprintMarketing SHIJOTACS 3 Replies Last reply Reply Quote 0
      • Kaushalwpandas
        Kaushalwpandas last edited by

        very depressed of being not indexing URL - https://www.wrightleatherworks.com/collections/gun-holsters-for-women please someone help

        1 Reply Last reply Reply Quote 0
        • Natalie305
          Natalie305 last edited by

          Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.

          1 Reply Last reply Reply Quote 1
          • Luna204
            Luna204 last edited by

            Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.

            1 Reply Last reply Reply Quote 0
            • SHIJOTACS
              SHIJOTACS @aplusnetsolutions last edited by

              @aplusnetsolutions create a seperate sitemap for not indexed pages and upload to google web master tools. i had a website had same issues. Also remember it will take some time also

              1 Reply Last reply Reply Quote 0
              • BlueprintMarketing
                BlueprintMarketing @aplusnetsolutions last edited by

                @aplusnetsolutions
                Make sure all these things are selected in screaming frog and crawl analysis selected

                1 Reply Last reply Reply Quote 1
                • BlueprintMarketing
                  BlueprintMarketing @aplusnetsolutions last edited by

                  @aplusnetsolutions

                  Going to Google search console it will tell you how many inches are inside Google search index there are a few other ways to do this if you want

                  #1
                  By far the easiest is to going to search console and select coverage and all known pages
                  alt text

                  number 2

                  For the quick and dirty method, simply perform a simple site search in your Google search bar with “site:yourdomain.com”

                  #3

                  You can crawl your site using screaming frog (it will 500 pages for free but every SEO should have it) use it to index your site and connect it to Google search console API as well as Google analytics and you will have a easy-to-read CSV export of your index and non-index pages
                  more on how this is done

                  ![ you can see the non-indexable URLs on the the right click that](77e61d47-2172-491c-8070-9a6cd5905fff-image.png image url)

                  video https://youtu.be/iYeXSdUt_hg

                  #4
                  or indexer site with any thing that can export a CSV download the CSV uploaded here and it will tell you which pages are in Google's index
                  https://www.rankwatch.com/free-tools/google-index-checker

                  https://searchengineland.com/check-urls-indexed-without-upsetting-google-follow-267472

                  https://seo.co/google-index/

                  1 Reply Last reply Reply Quote 0
                  • 1 / 1
                  • First post
                    Last post

                  Got a burning SEO question?

                  Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                  Start my free trial


                  Browse Questions

                  Explore more categories

                  • Moz Tools

                    Chat with the community about the Moz tools.

                  • SEO Tactics

                    Discuss the SEO process with fellow marketers

                  • Community

                    Discuss industry events, jobs, and news!

                  • Digital Marketing

                    Chat about tactics outside of SEO

                  • Research & Trends

                    Dive into research and trends in the search industry.

                  • Support

                    Connect on product support and feature requests.

                  • See all categories

                  Related Questions

                  • davidvogel

                    Best redirect destination for 18k highly-linked pages

                    redirect redirect chain technical seo 301 redirects

                    Technical SEO question regarding redirects; I appreciate any insights on best way to handle. Situation: We're decommissioning several major content sections on a website, comprising ~18k webpages. This is a well established site (10+ years) and many of the pages within these sections have high-quality inbound links from .orgs and .edus. Challenge: We're trying to determine the best place to redirect these 18k pages. For user experience, we believe best option is the homepage, which has a statement about the changes to the site and links to the most important remaining sections of the site. It's also the most important page on site, so the bolster of 301 redirected links doesn't seem bad. However, someone on our team is concerned that that many new redirected pages and links going to our homepage will trigger a negative SEO flag for the homepage, and recommends instead that they all go to our custom 404 page (which also includes links to important remaining sections). What's the right approach here to preserve remaining SEO value of these soon-to-be-redirected pages without triggering Google penalties?

                    Technical SEO | | davidvogel
                    0
                  • worldviajar.com

                    How can I check my website is not in spam?

                    keyword ranking keywords

                    I have a blogging website where I post about famous food, home remedies, and more. When I started my website's keywords were ranking on Google But Now a single keyword is not in the ranking list. That's why I have concerns about how I can fix it.

                    SEO Tactics | | worldviajar.com
                    0
                  • laurentjb

                    How do I carry out a redirect? Is there a code I need to use?

                    redirect redirect chain

                    How do I carry out a redirect? Is there a code I need to use? Thank you in advance.

                    On-Page Optimization | | laurentjb
                    0
                  • TimoRossa

                    Google doesn't show my Twitter account

                    google search ranking issue

                    Hello, my full name is Timo Rossa and my Twitter (X) account is @TimoRossa. If I search for my name with "Timo Rossa" on Google, it doesn't find any results referencing my Twitter account. It is very important for me that Google does not only show Twitter results for my name but also that those results would be ranked at the top. The only reason I could come up with is that my account name has not separated words. Does this make sense? What would be a simple strategy to achieve my goal? Thank you!

                    SEO Tactics | | TimoRossa
                    0
                  • sasansasyino

                    google webmaster tools Indexing request rejected

                    indexing request rejected webmaster indexing

                    when i try to index my posts in google webmaster tools i see this eror : Indexing request rejected
                    During live testing, indexing issues were detected with the URL
                    Crawl
                    Time
                    Sep 23, 2023, 11:05:05 PM
                    Crawled as
                    Google Inspection Tool desktop
                    Crawl allowed?
                    Yes
                    Page fetch
                    error
                    Failed: Hostload exceeded
                    Indexing allowed?
                    N/A
                    Indexing
                    User-declared canonical
                    N/A
                    Google-selected canonical
                    
                    Only determined after indexing my website : http://123select.ir/

                    SEO Tactics | | sasansasyino
                    0
                  • BakeryTech

                    Unsolved Using NoIndex Tag instead of 410 Gone Code on Discontinued products?

                    ecommerce noindex shopify indexed urls

                    Hello everyone, I am very new to SEO and I wanted to get some input & second opinions on a workaround I am planning to implement on our Shopify store. Any suggestions, thoughts, or insight you have are welcome & appreciated! For those who aren't aware, Shopify as a platform doesn't allow us to send a 410 Gone Code/Error under any circumstance. When you delete or archive a product/page, it becomes unavailable on the storefront. Unfortunately, the only thing Shopify natively allows me to do is set up a 301 redirect. So when we are forced to discontinue a product, customers currently get a 404 error when trying to go to that old URL. My planned workaround is to automatically detect when a product has been discontinued and add the NoIndex meta tag to the product page. The product page will stay up but be unavailable for purchase. I am also adjusting the LD+JSON to list the products availability as Discontinued instead of InStock/OutOfStock.
                    Then I let the page sit for a few months so that crawlers have a chance to recrawl and remove the page from their indexes. I think that is how that works?
                    Once 3 or 6 months have passed, I plan on archiving the product followed by setting up a 301 redirect pointing to our internal search results page. The redirect will send the to search with a query aimed towards similar products. That should prevent people with open tabs, bookmarks and direct links to that page from receiving a 404 error. I do have Google Search Console setup and integrated with our site, but manually telling google to remove a page obviously only impacts their index. Will this work the way I think it will?
                    Will search engines remove the page from their indexes if I add the NoIndex meta tag after they have already been index?
                    Is there a better way I should implement this? P.S. For those wondering why I am not disallowing the page URL to the Robots.txt, Shopify won't allow me to call collection or product data from within the template that assembles the Robots.txt. So I can't automatically add product URLs to the list.

                    Technical SEO | | BakeryTech
                    0
                  • Leviiii

                    slug Link redirect to subdomain?

                    redirect subdomain link building

                    Hi !
                    Im Levi new here and new in the world of SEO, please dont judge if my questions are silly. Back on the days when the site was built we thought it is a good ideea to have subdomains that together with the domain name represent our main keywords.
                    ex. https://stansted.tonorwich.uk, https://heathrow.tonorwich.uk, https://luton.tonorwich.uk, https://gatwick.tonorwich.uk. There is content on this subdomains, would it make any difference from SEO perspective if we create slugs that redirect to these subdomains? for example creating https://tonorwich.uk/taxi-minibus-vip-tesla-norwich-to-stansted that redirects to https://stansted.tonorwich.uk ? Or better create these slugs with slightly different content?
                    Any ideeas would be appreciated.
                    Thanks in advance!

                    Link Building | | Leviiii
                    0
                  • Emily.R.Monrovia

                    When to re-write and redirect a blog url?

                    urlredirect optimization content marketing seo tactics content development redirect

                    What are best practices for rewriting (and then redirecting) blog URLs? I refresh old blog posts on our blog every month and many of them have URLs that are too long or could be improved. However, many of them also already get decent organic traffic and I don't want to lose traffic due to a URL redirect. Are there any best practices or "rules" I can follow when deciding whether to re-write and redirect blog URLs?
                    Thanks!

                    Content Development | | Emily.R.Monrovia
                    0

                  Get started with Moz Pro!

                  Unlock the power of advanced SEO tools and data-driven insights.

                  Start my free trial
                  Products
                  • Moz Pro
                  • Moz Local
                  • Moz API
                  • Moz Data
                  • STAT
                  • Product Updates
                  Moz Solutions
                  • SMB Solutions
                  • Agency Solutions
                  • Enterprise Solutions
                  Free SEO Tools
                  • Domain Authority Checker
                  • Link Explorer
                  • Keyword Explorer
                  • Competitive Research
                  • Brand Authority Checker
                  • MozBar Extension
                  • MozCast
                  Resources
                  • Blog
                  • SEO Learning Center
                  • Help Hub
                  • Beginner's Guide to SEO
                  • How-to Guides
                  • Moz Academy
                  • API Docs
                  About Moz
                  • About
                  • Team
                  • Careers
                  • Contact
                  Why Moz
                  • Case Studies
                  • Testimonials
                  Get Involved
                  • Become an Affiliate
                  • MozCon
                  • Webinars
                  • Practical Marketer Series
                  • MozPod
                  Connect with us

                  Contact the Help team

                  Join our newsletter

                  Access all your tools in one place. Whether you're tracking progress or analyzing data, everything you need is at your fingertips.

                  Moz logo
                  © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                  • Accessibility
                  • Terms of Use
                  • Privacy

                  Looks like your connection to Moz was lost, please wait while we try to reconnect.