Find archived sitemap of a website that no longer exists
-
I am trying to figure out the site structure of a website and the urls of all the pages. Normally this would be easy but a couple of months ago the website went down and I don't think it will ever come back. Any help would be appreciated.
-
Use the internet archive (wayback machine) which effectdigital mentioned above, to find the /robots.txt file from the desired date. In that file you should find the referenced sitemap file (assuming the site properly included its sitemap reference in its robot.txt file). Then you can use the same process to request the sitemap file which was referenced in the robots.txt file.
-
Hi Effect,
Does your second link automatically provider the sitemaps available, or does the user still need to "know" or be able to guess where they might be e.g /sitemap.xml?
Nick
-
You can use this site to see legacy site-maps for some websites (though they may be partial or incomplete):
For example, check these sitemap results:
For smaller sites, the results are much easier to look at.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I can see competitors ranking for certain long-tail keywords but cannot find them on web pages. What am I missing?
Hi there. I'm pretty new to SEO and I've been doing a fair bit of training but there is one aspect I have yet to grasp. When I carry out keyword research, I get all these results and I understand the metrics. What I'm not getting is, when a competitor is ranking highly for say "where can I buy fresh turkeys", I assume that that phrase must appear somewhere on the page, but it doesn't. I realise I'm just not thinking about this in the right way. Can anyone offer clarification, please? Kind regards, Bruce
Competitive Research | | BruceBarbour0 -
I am confused/frustrated/surprised how bad my website is doing on google ranking
Hello, I am confused/frustrated/surprised how bad my website (flyhy.co) is doing on google ranking and I have no clue why even though I have been doing my homework regarding SEO. Just a bit of background, I have created a new website about 6 months ago for the paragliding community, the primary goal is to provide a platform for people to publish their ads (osclass), but also to provide some interesting reviews and tools to help paragliders chose their wing. We have been putting a lot of effort to provide a nice user experience and tio build the tools mentioned above. Our main channel to connect with the community is Facebook, and we have been quite active there. I have looked at many SEO articles and I made sure the website provides a good UX, the URLs are SEO friendly, good meta data, etc. Also have been using the google search console and analytics to monitor all of this. But here is the thing, all these does not seem to change anything in our ranking for important keywords such as "paraglider for sale", "paragliding equipment", etc. We seem to only rank (looking at Google’s keyword tool) for very specific wing model names that people have mentioned in their ads. I have ran out of ideas on how to improve our SEO !!!!!! I know the website is only 6 months old, but by now we should get some results. As an example, I will mention one our main website competitors: www.paraglidingequipment.org. OK the URL is pretty obvious and this website ranks in page #1 for "paragliding equipment" (but also for "paraglider for sale" and other paragliding related key phrases). OK there is the URL (paraglidingequipment.org), but I thought nowadays google bots are smarter than just that. The website is 1 year old (so not really much older than us, and was ranking high anyway even 6 months ago). The website looks like it was clearly made by one person and then quickly just left it running, so no content has been added (except for people putting their ads), there is almost no activity on the Facebook account. I have run some test such as "pagespeed insights" and we both rank the same. On "seositecheckup.com", we are clearly better with more 10 points. Is there anyone out there who can tell me what is going on? Have I missed a very important aspect of SEO? Is our website somehow compromising the robots crawling (although I can see about 80 pages have been already indexed in google search console)? I know content is king, but in paraglidingequipment.org the only content I see are ads, and we have ads and other interesting (ie reviews and tools) for paragliders. To conclude, I am basically completely clueless of what to do to rank at least on the first couple of pages of google for the key phrases above. I need help. Hichem. PS: in Moz bar our score is non existing (PA=1,DA=10), on paraglidingequipment.org (PA=23,DA=15). So it looks that essentially we are not apparent on the web! PSS: We have also tried to build some backlinks on few important paragliding community websites.
Competitive Research | | hichemboudali0 -
How can I track where visitors go after exiting my website?
I don't want to track external links. I just want to know where they go when they leave. Is that possible? Can I do this with a cookie?
Competitive Research | | Vacatia_SEO0 -
Why is our website ranked lower but beats most competitors in full SERP report?
We are analysing why our website ranks so low (currently position 8 in serp). We beat our competitors in most areas. Also we produce by far the most useful content. Do not buy links or do any other malpractice. We have been for a long time ranked in top 3 (well mostly 1-2) positions. In the last year we have seen a decline to position 8-10 and we are not sure why this is the case. Can anyone suggest what we should be focusing on? We are clueless. All the practices we used to "know" now seem obsolete. m8qIeyC,Kf1LLei#0
Competitive Research | | urkeman0 -
White branded website & SEO
Hi guys, We might have as a new project to create a white branded website for a big portal in our local market (which have a strongest domain than ours): the goal will be to reach their big mass of users and if possible, place this new site BEHIND us on the SERPs. Since the content of the new website will basically be the same, we are considering 2 solutions: to "noindex" the site on search engines, which is a "secure" way to not create ourself a competitor to allow the site to be indexed on search engines but using the "rel=canonical" strategy to not be affected by duplicate content penalties (For example, we plan to add rel='canonical' href='http://www.ourdomain.com/category1/product2' /> on their page http://newsubdomain.theirdomain.com/category1/product2) The main question is: can the white branded website rank better than our site even with the "canonical" strategy? (Of course we could "lower" the quality of the white-branded website pages to avoid that risk... but if somebody has better advices, we would be glad to hear them 😉 )
Competitive Research | | Kuantokusta0 -
Find competitors based on keywords
Are there any tools where I can enter say 5 keywords and it will look at the top 10 sites for each of those keywords then return which sites show up the most.
Competitive Research | | eyeflow0 -
Free tools to discover traffic for a website?
Are there any free tools to learn traffic data (especially organic traffic data) from a site or blog?
Competitive Research | | nicole.healthline0 -
Website analysis automation
I need a tool that will input a list of URLs and produce a spreadsheet with link metrics for all sites including PageRank, Domain Authority, Page Authority, Social data, etc..i know that OSE will do this for 5 sites at a time but I need to compare more. At lest 50 at a time... I bought Buzzstream thinking this would work, but it doesn't. does anybody have any good advice? Thanks!!
Competitive Research | | znotes0