Find archived sitemap of a website that no longer exists
-
I am trying to figure out the site structure of a website and the urls of all the pages. Normally this would be easy but a couple of months ago the website went down and I don't think it will ever come back. Any help would be appreciated.
-
Use the internet archive (wayback machine) which effectdigital mentioned above, to find the /robots.txt file from the desired date. In that file you should find the referenced sitemap file (assuming the site properly included its sitemap reference in its robot.txt file). Then you can use the same process to request the sitemap file which was referenced in the robots.txt file.
-
Hi Effect,
Does your second link automatically provider the sitemaps available, or does the user still need to "know" or be able to guess where they might be e.g /sitemap.xml?
Nick
-
You can use this site to see legacy site-maps for some websites (though they may be partial or incomplete):
For example, check these sitemap results:
For smaller sites, the results are much easier to look at.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What tool can I use to find the top traffic-driving keyword for a batch of pages from multiple sites?
I thought I might be able to use Ahref's Batch Analysis for this, but that just gives stats on backlinks. I have a several lists of thousands of pages scraped for particular sets of keywords, but what I need is some way to automate fetching the biggest keyword that each page ranks for - biggest being the keyword that is estimated to drive the most organic traffic. Is there a tool out there that does something like this?
Competitive Research | | helenlorettahasan0 -
Why my website's Alexa rank is dropping
My website Alexa rank has dramatically dropped in the past 3 months: Website: www.sastasundar.com End of July, 2012: 5217 Global, 265 Local(India) But Now: ** 29260** Global, 2274 Local(India) Definitely something big has affected my global ranking and I'm not able to figure it out. Please help.
Competitive Research | | foreseegame0 -
How does my competitor website jump from a natural ranking of 10 to 1 in a matter of days?
OK this has really got me baffled, and I really hope the MOZ community can give me the answers I am seeking.... My statements and queries below are my own opinions and thoughts. I am not an experienced SEOer and I am learning all the time, so if I have written something incorrectly, please feel free to correct me 🙂 I have been told that natural rankings take time, and I have read with the recent algorithm changes from Google, that purchased links do not have an effect like they used to. I have also read somewhere that google has clamped down on bogus article writing (MOZ please clarify) But what I am seeking some clarification on is, how does my competitors website, that has had a natural rank of #1 in the past, suddenly drop off; for a lengthy period of time, then suddenly re-appear at #10 on page #1, then some three/four days later, hit #1? I have read that natural rankings are purely natural. I.e. a domain owner cannot pay google to boost their natural ranking. But is this really true? Why is it when I speak to a close colleague of mine in the same industry, he mentions that his maps listing has dropped off, and after making a couple of phone calls to his SEO guys in India, maybe a day or two later, magically his maps re-appear? I really am starting to question is a "natural ranking" really "natural?" I have heard that some of these SEO agencies do have some tools that dramatically effect the natural ranking of a domain. What are these tools?
Competitive Research | | Giorgio680 -
How to Find Another Site's robots.txt File?
An SEO report, not by SEOmoz, says my top two competitors have robots.txt files that disallows spidering. I suspect that their robots.txt file doesn't disallow all spidering. How do I find out what is in their robots.txt files?
Competitive Research | | lbohen0 -
How do I know top pages (first on Google) for one website?
Hi! I'm SEOMOZ PRO user and I want to know which pages of other website are ranking on top on Google, Yahoo, Bing, ecc... with relative position (rank). Is there a features of SEOMOZ for it? Regards
Competitive Research | | jadlib0 -
Website analysis automation
I need a tool that will input a list of URLs and produce a spreadsheet with link metrics for all sites including PageRank, Domain Authority, Page Authority, Social data, etc..i know that OSE will do this for 5 sites at a time but I need to compare more. At lest 50 at a time... I bought Buzzstream thinking this would work, but it doesn't. does anybody have any good advice? Thanks!!
Competitive Research | | znotes0 -
Why do some websites that have hardly any backlinks, poorly optimised get placed above sites that are doing all they can with SEO?
For example take these two websites. www.rfasecurity.co.uk is a site that Iam working on and have had some success with yet snifferdogs.org.uk seems to be poorly optimised with little backlinks and still gets place higher. is it due to the key word in the domian name? The key search word is sniffer dogs New at this so any help would be appreciated regards Dave
Competitive Research | | ThePod0 -
Can i have chance to rank higher than official website in google local domains ?
Q : can i have chance to rank higher than official website in google local domains ? for example : rank higher than microsoft,kaspersky,nokia etc... in google italy or google germany or any other local domain for google
Competitive Research | | activeacts0