Robot.txt and Sitemap
-
Simple question for today
What do you use to create a sitemap ( if your site is NOT Wordpress ) and do you mention your sitemap in your robot.txt?
Thanks in advance!
-
Hi Angelos, you've received some great responses. Did any of them answer your question?
-
I use xml-sitemaps.com for sitemap generation as its simple and easy to use!
If you are adding sitemap link in your robots.txt file its fine but not adding is not really a big problem!
-
I use GsiteCrawler all the time. Its free and gives you lot of options. You can get it here:http://gsitecrawler.com/
Best regards,
Devanur Rafi.
-
You could try one of the following listed here https://code.google.com/p/sitemap-generators/wiki/SitemapGenerators
Yes its a good practice to mention your sitemap in robots.txt and dont forget to submit it ( at the very least ) via Google/Bing WebMasterTools
-
Hi Angelos,
We use http://www.xml-sitemaps.com/ and yes we mention our sitemaps, among other things, in our robots.tct file:
http://www.ccisolutions.com/robots.txt
and
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you check my Quran site and its technical issues?
Hi, Hope you are doing good, I saw many Quran websites working fine. I have a blog that has online Quran classes of Arabic and Tajweed related courses as well. Can you please check that if all is okay in terms of technical issues? Here is the website: https://learnquran.academy
SEO Learn Center | | ShirleyLemons0 -
A simple link juice question!
lets say i have backlink from a site with dr 80 and ur 10 and no.of external links on that page are 5 on the other hand my competitor has same backlink with same dr and ur but on that page there are other 10 external links already and link is 11th who is getting the most juice
SEO Learn Center | | Sam09schulz0 -
When looking up a URL in GSC's URL Inspection tool has anyone ever come across a page that has the following page in "Referring Page" --> http://achadda-dev.us.archive.org/
Heres where the URL inspector is saying it found this xml sitemap page QAaLtfh
SEO Learn Center | | SFChronTeam0 -
I want to know how is looking my website?
Hi, I hope you are doing well. I want know how is looking my website? Is that attractive or boring for visitor. I really need your answer to improve my website. here is my website: https://www.myqurantutor.com/
SEO Learn Center | | Bigbrand1 -
How can I secure my website?
Hi, I hope you are doing well. I wanted to ask you how I secure my website whenever I have SLL but how can I make more secure my website? I hope I will like anyone's reply. thanks in advance This is my website: https://www.myqurantutor.com/
SEO Learn Center | | Bigbrand1 -
Blog Post Relevance vs Traffic
Hello Moz community, I have a website (lets say that I am selling tshirts), and want to create a blog (www.website.com/blog). With that blog, I would like to post about random topics including travel, leisure, events, new products etc. I want to do this for 1. to get affiliate commissions, 2. to increase web traffic, 3. to potentially increase my domain authority. I would like to know if this blog with random posts would negatively affect my website ranking and keyword ranking (for t-shirts), as it is not relevant to the products that I am selling (and the keywords that I am ranking for). As Google currently understands that my website is a webstore selling t-shirts (and I am ranking for the tshirt keywords), but the algorithm get confused if I start posting about travel and random topics? Your help greatly appreciated in the matter.
SEO Learn Center | | PTurbo1 -
For a parent blog on our website, what should we go for - Subdomain or Subdirectory?
We are a kids website with fun learning content dedicated to kids aged 6-14 yrs, now we want to start a blog page for parents with parenting tips and information useful to parents. For this, should we choose - a subdomain or subdirectory?
SEO Learn Center | | Mocomi0 -
Should you Move To HTTPS? Google Thinks So!
Just wanted to share some information that HTTPS is now a positive ranking factor. Here is an article about the fact that it can give a small boost in Rankings: http://searchengineland.com/explainer-googles-new-ssl-https-ranking-factor-works-200492 Here is also a video by Google Saying they think "every site should have a security certificate" and how to employ it. https://www.youtube.com/watch?v=cBhZ6S0PFCY People may already know about this; just thought it was important to share and couldn't find it thoroughly addressed anywhere on the site.
SEO Learn Center | | photoseo13