List all URL's indexed by google
-
Hi all i need a list of all urls google has indexed from my site
i want this in excel format or csv
how do i go about getting this
thanks in advance
-
One method of doing this is following the first part of this YouMoz for using a crawler to see what Google has indexed.
http://www.seomoz.org/ugc/solving-new-content-indexation-issues-for-large-b2b-websites
-
Hi Will
I like using Google Webmaster Tools. Sign up and add your websites to it and you will get a wide range of information from that platform.
Or you can type site:www.domain.com in the search of each search engine.
easy
Have a look at using some of these tools :
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Variables in Google Tag Manager
Last week I got a great answer here on how to implement GTM and cross domain tracking, now that we have that in place I'm looking for some more advice 😉 I'm wondering that if we push variabels into the data-layer of GTM, can we actually use those variables in reports as well? Do they get recorded in Google Analytics? I'd like to use some data that we push to the data layer for segmentation purposes. Anybody an idea how to achieve this?
Reporting & Analytics | | jorisbrabants0 -
Submitting an 'HTTPS' sitemap.xml to Bing
I have been trying to submit my sitemap to Bing [via their webmaster tools] for well over a week and it continues to report 'pending' My site is HTTPS and the sitemap is accepted by Google. I questioned Bing about this and got this response: To set your expectations, our Sitemap fetchers use a different pipeline and because of this, we cannot crawl Sitemaps in HTTPS format. We require that you submit an HTTP version of sitemap in order for Bing to properly crawl the file. Please go ahead and delete the current Sitemap and resubmit a new one in HTTP. Currently I don't and can't have a HTTP version of my site & sitemap and my developers are telling me that 3hrs worth of dev time will go into coming up with a work-around which I'm not sure I want to invest in [I have more important things to concentrate my spend on!]. Has anyone been faced with this problem and is there any quick/cheap alternative or do I just accept that Bing won't crawl my site until they update their end?!
Reporting & Analytics | | cityxplora.com0 -
How do I fix 608's please?
Hi, I'm on the free trial and finding it very useful I've fixed all my 301's. but now I have a load of 608's. I don't no what this is! I feel like I've cured herpes only to get gonorrhea! can any one help. I have 41 608's which is more than the 301's I had. I hope they are non-related! I won't bore you with the whole list but some of the url's are: Error Code 608: Page not Decodable as Specified Content Encoding http://sussexchef.com/catering-at-mr-mrs-currys-50th-wedding-anniversary/guestsarrive----608 Error Code 608: Page not Decodable as Specified Content Encoding http://sussexchef.com/funeral-catering/picture4-2----608 Error Code 608: Page not Decodable as Specified Content Encoding http://sussexchef.com/wedding-venues
Reporting & Analytics | | SussexChef831 -
Difference between site: search and Total Indexed in Google Webmaster Tools.
This morning I did a search on Google for my site using the site: operator. I noticed that the number of results returned was significantly different than the "Total indexed" in Google Webmaster Tools. What is the difference and is it normal to have two very different numbers here?
Reporting & Analytics | | Gordian0 -
301 redirect visible on Google.
Hello, This is a strange one for me. We have 301 redirected our .com domain to the .co.uk domain. The strange thing is that if you gogole the .com domain (theloanengine.com) there is a result with the website description. If you click on the result you are redirected tot he website homepage. One other thing: I've discovered this 301 issue because Google Analytics started to show me a few days ago referrals from the .com domain. I don't know if these things are connected. Cornel
Reporting & Analytics | | Cornel_Ilea0 -
My first campaign identidied long URLs
Hello! 🙂 I've just created my first campaign, and the crawling proccess have detected posts with long URL (more than 70 characters). If I change it, i.e., alter the URL's, can some problem happens to my blog? Or do I have to disconsider this problem and just "work correctly" from now on? Thanks in advance for your help!
Reporting & Analytics | | Andarilho0 -
Webmaster Not found URLs
Dear All, I would really like help with this. Due to some unknown reason (another thread is open for this reason), my google webmaster is showing 7000 not found URL's. Now, when i try to find out the day these broken URLs were detected, webmaster is showing dates between November 2011 to December 2nd, 2011. I havent found a single not found error showing after 2nd December 2011. So does that mean that the mistake has been solved? Because daily webmaster is adding 200-300 not found URLs. Along with this, my traffic has dropped drastically since 12th December and has still not recovered. Are these not found URLs the reason for this sudden traffic drop? If so, then i m ready to find someone for paid seo to remove this error. I would love to have some concrete answers for these questions. Thanksss
Reporting & Analytics | | hith2340 -
Will Google start trimming 'stale' sites rank?
With the recent focus on Google to reduce rank of farms and low value sites, I am interested to get SEO view on if you think Google will start devaluing stale sites. I do find it a bit frustrating that in the top 5 for my main key phrase, there is one site that has NO content just an error and another blog that has not updated content in 2 years. How can blogs that do not blog be considered high enough value by Google to rank in the top 5? How can sites that just return 404 or 500 for ALL their pages be even considered a site let alone rank 2nd. I am interested so see others experiences and thoughts on 'user experience' clean ups by Google and why these types of sites get missed?
Reporting & Analytics | | oznappies0