How can I make a list of all URLs indexed by Google?
-
I have a large site with over 6000 pages indexed but only 600 actual pages and need to clean up with 301 redirects. Haven't had this need since Google stopped displaying the url's in the results.
-
very depressed of being not indexing URL - https://www.wrightleatherworks.com/collections/gun-holsters-for-women please someone help
-
Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.
-
Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.
-
@aplusnetsolutions create a seperate sitemap for not indexed pages and upload to google web master tools. i had a website had same issues. Also remember it will take some time also
-
@aplusnetsolutions
Make sure all these things are selected in screaming frog and
-
Going to Google search console it will tell you how many inches are inside Google search index there are a few other ways to do this if you want
#1
By far the easiest is to going to search console and select coverage and all known pages
number 2
For the quick and dirty method, simply perform a simple site search in your Google search bar with “site:yourdomain.com”
#3
You can crawl your site using screaming frog (it will 500 pages for free but every SEO should have it) use it to index your site and connect it to Google search console API as well as Google analytics and you will have a easy-to-read CSV export of your index and non-index pages
more on how this is done
image url)video https://youtu.be/iYeXSdUt_hg
#4
or indexer site with any thing that can export a CSV download the CSV uploaded here and it will tell you which pages are in Google's index
https://www.rankwatch.com/free-tools/google-index-checkerhttps://searchengineland.com/check-urls-indexed-without-upsetting-google-follow-267472
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What steps should I take to address damage to my website, including malware insertion and content theft?
The question revolves around the steps required to mitigate damage inflicted upon a website, encompassing issues such as malware insertion and content theft. It prompts a comprehensive exploration of the necessary actions to take in response to these challenges. The inquirer seeks guidance on how to effectively address the damage, indicating a desire for practical solutions and strategies to restore and safeguard their website's integrity. By posing this question, the individual demonstrates an awareness of the severity of the situation and a readiness to undertake corrective measures.
Technical SEO | | ralphbaer0 -
Google search console 380,000 spam backlinks
Hi guys, I recently suffered a major negative seo attack against my site, with more than 380K spam backlinks using more than 5k domains. Because of this, I'm having serious problems tracking my site's statistics in GSC due to the limit of only 1000 query lines. Please, I need help on how I can get access to all these 5,000 domains in the search console so I can create a disavow list. Any tips on how to clean this up?
SEO Tactics | | xurupita0 -
google webmaster tools Indexing request rejected
when i try to index my posts in google webmaster tools i see this eror : Indexing request rejected
SEO Tactics | | sasansasyino
During live testing, indexing issues were detected with the URL
Crawl
Time
Sep 23, 2023, 11:05:05 PM
Crawled as
Google Inspection Tool desktop
Crawl allowed?
Yes
Page fetch
error
Failed: Hostload exceeded
Indexing allowed?
N/A
Indexing
User-declared canonical
N/A
Google-selected canonical
Only determined after indexing my website : http://123select.ir/0 -
Redirecting Homepage to Subdomain Bad or Good Idea??
I have a very old forum that still gets a lot of traffic, but when migrating over to another software that is cloud based we cannot redirect using same domain, SO the only option would to be to change the cname on a subdomain and then REDIRECT all the traffic from the ROOT domain permanently - would this be a bad move as the root domain wouldnt be used anymore as now its just setup to be redirected in order to use the software we need to use? Domain is 17 years old.
Technical SEO | | vbsk0 -
Preserving Authority of closed website
Hi, I have the opportunity to acquire a domain of a company that is closing in a similar industry who currently have a decent Authority - 51DA, 5,752 Linking Domains. I cannot do anything with the new business yet but if I acquire the domain the existing website will turn off. If I crawl the existing site and 302 redirect all pages to a holding page on said domain, will that preserve authority while I start the new business and build the new website and like-for-like pages?
Technical SEO | | Ant-Scarborough0 -
Blogs Not Getting Indexed Intermittently - Why?
Over the past 5 months many of our clients are having indexing issues for their blog posts.
Technical SEO | | JohnBracamontes
A blog from 5 months ago could be indexed, and a blog from 1 month ago could be indexed but blogs from 4, 3 and 2 months ago aren't indexed. It isn't consistent and there is not commonality across all of these clients that would point to why this is happening. We've checked sitemap, robots, canonical issues, internal linking, combed through Search Console, run Moz reports, run SEM Rush reports (sorry Moz), but can't find anything. We are now manually submitting URLs to be indexed to try and ensure they get into the index. Search console reports for many of the URLs will show that the blog has been fetched and crawled, but not indexed (with no errors). In some cases we find that the blog paginated pages (i.e. blog/page/2 , blog/page/3 , etc.) are getting indexed but not the blogs themselves. There aren't any nofollow tags on the links going to the blogs either. Any ideas? *I've added a screenshot of one of the URL inspection reports from Search Console alt text0 -
Google Search Console - Excluded Pages and Multiple Properties
I have used Moz to identify keywords that are ideal for my website and then I optimized different pages for those keywords, but unfortunately rankings for some of the pages have declined. Since I am working with an ecommerce site, I read that having a lot of Excluded pages on the Google Search Console was to be expected so I initially ignored them. However, some of the pages I was trying to optimize are listed there, especially under the 'Crawled - currently not indexed' and the 'Discovered - currently not indexed' sections. I have read this page (link: https://moz.com/blog/crawled-currently-not-indexed-coverage-status ) and plan on focusing on Steps 5 & 7, but wanted to ask if anyone else has had experience with these issues. Also, does anyone know if having multiple properties (https vs http, www vs no www) can negatively affect a site? For example, could a sitemap from one property overwrite another? Would removing one property from the Console have any negative impact on the site? I plan on asking these questions on a Google forum, but I wanted to add it to this post in case anyone here had any insights. Thank you very much for your time,
SEO Tactics | | ForestGT
Forest0 -
Solve Redirect Chains
I've gotten a few Redirect Issues that involve Redirect Chains, with the https:// version redirecting to the www. version and then redirecting to the right URL. Here is an example:
On-Page Optimization | | Billywig
Schermafbeelding 2021-12-07 om 11.04.32.png I've tried setting a direct redirect between the first and the last URL, but WordPress doesn't seem to allow that (it's overwritten). I've also tried checking the internal links to make sure that none of the links are the first one. They don't seem to be there. Does anyone have any tips on solving these Redirect Chains?0