Export Google Search Results
-
I would like to export a list of pages indexed by Google and other search engines. According to our site search with google, we have over 300,000 pages indexed. Anyway to export a list of all of these pages?
Tools like ScreamingFrog (which I own a copy of) can crawl your site but does not tell you the pages indexed by a search engine... I've tried using tools that will let you export each page of the results. However, this won't work for a 300K page website.
Thanks for your help!
-
Hi Shani! Do these responses help to answer your question or are you looking for more information? If you're good to go, please mark this as answered. Thanks!
-
Thank you Thomas and Logan! Thomas, GWT puts a cap at 999 pages, but it is also the search terms used to get to our site as opposed to the pages that are indexed. Logan, it appears your answer is accurate as I have been researching this for awhile with no luck.
-
There's no tool that will handle it in that kind of volume. I know of one tool called SERP Scraper that will do 100 URLs, but that's no good when you're evaluating 300k. I'm fairly certain Google makes it impossible for anyone to build this tool, as useful as it would be, it would exist already if this weren't the case.
-
I'm not aware of any tools that would be able to achieve what you need. However, would GWT not be able to help you here?
Search Analytics section would allow you to download all the pages that are indexed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Viewing search results for 'When searching in google we find our site in the first position but when some others search it is seen on the second page 1 st position why is this happening?'
Viewing search results for 'When searching in google we find our site in the first position but when some others search it is seen on the second page 1 st position why is this happening?'
Search Behavior | | Alyaauditors0 -
Looking for something better than Googles in page analytics
Buongiorno from 12 degrees C wetherby UK 🙂 I'm interested in measuring content engagagement, things like what gets clicked on mouse tracking beahaviour etc. Google Analytics in page analytics does a good job but is there another product able to guage user interactive behaviour with content (Yes ive added event tracking). Thanks in advance,
Search Behavior | | Nightwing
David1 -
Getting People to Leave Product Reviews, When they've reached our site in search of reviews
We have a product review website that has been showing a steady increase in organic traffic. However we seem to be having a disconnect between what happens when a visitor arrives. We essentially want a visitor coming to our website in search of a review, to LEAVE a review for another product they may have used... Any suggestions on ways we might encourage this? We are kind of newbies at this!
Search Behavior | | Sarah_Frantz0 -
Safari Has Top Browser in Google Analytics?
I find this hard to believe but it looks to me as though Safari has jumped ahead as our top browser in the month of January. At first glance I would think it was combining mobile safari with desktop safari numbers but then on the 5th spot I see Safari(in-app) which after looking into could be multiple things from safari mobile, bookmarked pages, or Chrome for ios. Just looking for thoughts on this, are others seeing a large uptick in Safari traffic for desktops?
Search Behavior | | sknott0 -
I need Help with Google!!!!
I am trying to have my picture on the first page just like SEOmoz when someone search just the name, I know have something to do with google plus, but I am so new doing that no luck or probably I am doing wrong, I have been looking in the internet but I haven't found anything. Is there anyone who can write a tutorial and post here. Maybe is already done and I don't know where. Please see the picture attached so you understand better what I want to do Again I want appear just like this but with my company http://www.sombras.co.uk/images/pic.jpg pic.jpg
Search Behavior | | teksyte0 -
Visitors via Android App in Google Analytics
Hey there Mozzers! For one of our clients, we've developed an application on both Android and iOS. In the app, there is a link to the regular website. Can I see these visitors from the IOS/Android app in Google Analytics? Can I find them via Refferals? Thanks a lot in advance! Partouter
Search Behavior | | Partouter0 -
How do you know if you have been penalized by search engines?
How do you know if you have been penalized by search engines? I logged into my Google and Bing webmaster accounts and all looks ok, but in the last 30 days, my organic traffic has decreased 33%. My traffic usually steadily increases and if there is ever a decrease, it's usually very small.
Search Behavior | | StreetwiseReports0 -
Google Location - Taking Away Our National Reach?
Hey, I was just noticing that we achieve #2 ranking on Google for one of our customers for one of their primary keyword phrases. But then I noticed the traffic analytics were not matching what we should expect from that keyword phrase. Then I noticed, in using "Chrome's Incognito Window", that our location was automatically selected for our main geographical city area. I then went and changed that location from Denver, to San Diego & Also Chicago, just to see what would happen, and I noticed we instantly dropped from #2 to #7 when changing our location. I don't know what my question is, but I guess I feel like that is preventing us from achieving the results we need to sell ecommerce products. Is there any info on this or suggestions anyone has on how to tackle this issue? It feels like Google is pulling the rug out from underneath our feet and trying to spread rankings more to localized areas, rather than offering someone the opportunity to capitalize on good rankings for a national audience. I understand why they would do it, and I don't say I disagree. But it just seems to affect our work as SEO's doesn't it? Since we can't be as effective for customers that have a global audience instead of strictly a localized one. I'm curious to see what people have to say about this issue. Thanks!
Search Behavior | | JerDoggMckoy0