Will it be possible to point diff sitemap to same robots.txt file.
-
Will it be possible to point diff sitemap to same robots.txt file.
Please advice. -
In your sitemap you only have to link to the pages on your site - there is no need to add a link to the robots.txt file.
Your robots.txt is always on the root of the domain (yourdomain.com/robots.txt). In your robots.txt you can put the location of your sitemap(s) - just put the following line(s) at the end of the file. It is possible to add multiple lines if you have multiple sitemaps. Alternative would be to put a link to the index type of sitemap which contains the links to the different sitemaps you have.
Sitemap: http://www.yourdomain.com/name_of_your_sitemap.xml
It is not strictly necessary to add the link to the sitemap in your robots.txt - you can also just indicate the location in the Search Console (Google) / Web master tools (Bing)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hreflang in country specific XML Sitemaps?
Hello! I'm rolling out hreflang tags in my client's "main" XML Sitemap. My question is: do we need to implement these tags in the country level XML Sitemaps also? Thanks!
Technical SEO | | SimpleSearch1 -
XML Sitemap Generators
I am looking to use a different sitemap generator that can do 5 thousand or more pages at once. Any recommendations? Thanks guys.
Technical SEO | | Chenzo0 -
How to use robots.txt to block areas on page?
Hi, Across the categories/product pages on out site there are archives/shipping info section and the texts are always the same. Would this be treated as duplicated content and harmful for seo? How can I alter robots.txt to tell google not to crawl those particular text Thanks for any advice!
Technical SEO | | LauraHT0 -
GWT returning 200 for robots.txt, but it's actually returning a 404?
Hi, Just wondering if anyone has had this problem before. I'm just checking a client's GWT and I'm looking at their robots.txt file. In GWT, it's saying that it's all fine and returns a 200 code, but when I manually visit (or click the link in GWT) the page, it gives me a 404 error. As far as I can tell, the client has made no changes to the robots.txt recently, and we definitely haven't either. Has anyone had this problem before? Thanks!
Technical SEO | | White.net0 -
Robots.txt
Google Webmaster Tools say our website's have low-quality pages, so we have created a robots.txt file and listed all URL’s that we want to remove from Google index. Is this enough for the solve problem?
Technical SEO | | iskq0 -
Robots.txt & Mobile Site
Background - Our mobile site is on the same domain as our main site. We use a folder approach for our mobile site abc.com/m/home.html We are re-directing traffic to our mobile site vie device detection and re-direction exists for a handful of pages of our site ie most of our pages do not redirect the user to a mobile equivalent page. Issue – Our mobile pages are being indexed in desktop Google searches Input Required – How should we modify our robots.txt so that the desktop google index does not index our mobile pages/urls User-agent: Googlebot-Mobile Disallow: /m User-agent: `YahooSeeker/M1A1-R2D2` Disallow: /m User-agent: `MSNBOT_Mobile` Disallow: /m Many thanks
Technical SEO | | CeeC-Blogger0 -
Would this be possible for my research
Hi i am trying to find out if it is possible for me to put a search term down and found out what companies are using this. I know you can do this with google but i am finding that it is not bringing up the right results. What i am trying to do is, search for a search word that is being used on their content pages, so for example, if i put down the word holidays in spain, i want to find that word being used in their content and not the site being in google. Hope this makes sense, so i am trying to find how many companies out there use a certain search phrase in their content any help would be great
Technical SEO | | ClaireH-1848860 -
Submitting Sitemap File vs Sitemap Index File
Is it better to submit all sitemap files contained in a Sitemap Index File manually to Google or is it about the same as just submitting the Master Sitemap Index File.
Technical SEO | | AU-SEO0