Will it be possible to point diff sitemap to same robots.txt file.
-
Will it be possible to point diff sitemap to same robots.txt file.
Please advice. -
In your sitemap you only have to link to the pages on your site - there is no need to add a link to the robots.txt file.
Your robots.txt is always on the root of the domain (yourdomain.com/robots.txt). In your robots.txt you can put the location of your sitemap(s) - just put the following line(s) at the end of the file. It is possible to add multiple lines if you have multiple sitemaps. Alternative would be to put a link to the index type of sitemap which contains the links to the different sitemaps you have.
Sitemap: http://www.yourdomain.com/name_of_your_sitemap.xml
It is not strictly necessary to add the link to the sitemap in your robots.txt - you can also just indicate the location in the Search Console (Google) / Web master tools (Bing)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Recovering from Sitemap Issues with Bing
Hi all, I recently took over SEO efforts for a large e-commerce site (I would prefer not to disclose). About a month ago, I began to notice a significant drop in traffic from Bing and uncovered in Bing Webmaster Tools that three different versions of the sitemap were submitted and Bing was crawling all three. I removed the two out of date sitemaps and re-submitted the up to date version. Since then, I have yet to see Bing traffic rebound and the amount of pages indexed by Bing is still dropping daily. During this time there has been no issue with traffic from Google. Currently I have 1.3 million pages indexed by Google while Bing has dropped to 715K (it was at 755K last week and was on par with Google several months ago). I know that no major changes have been made to the site in the past year so I can't point to anything other than the sitemap issue to explain this. If this is indeed the only issue, how long should I expect to wait for Bing to re-index the pages? In the interim I have been manually submitting important pages that aren't currently in the index. Any insights or suggestions would be very much appreciated!
Technical SEO | | tdawson090 -
Good robots txt for magento
Dear Communtiy, I am trying to improve the SEO ratings for my website www.rijwielcashencarry.nl (magento). My next step will be implementing robots txt to exclude some crawling pages.
Technical SEO | | rijwielcashencarry040
Does anybody have a good magento robots txt for me? And what need i copy exactly? Thanks everybody! Greetings, Bob0 -
Is there any value in having a blank robots.txt file?
I've read an audit where the writer recommended creating and uploading a blank robots.txt file, there was no current file in place. Is there any merit in having a blank robots.txt file? What is the minimum you would include in a basic robots.txt file?
Technical SEO | | NicDale0 -
Robots.txt
Google Webmaster Tools say our website's have low-quality pages, so we have created a robots.txt file and listed all URL’s that we want to remove from Google index. Is this enough for the solve problem?
Technical SEO | | iskq0 -
How can I make Google Webmaster Tools see the robots.txt file when I am doing a .htacces redirec?
We are moving a site to a new domain. I have setup an .htaccess file and it is working fine. My problem is that Google Webmaster tools now says it cannot access the robots.txt file on the old site. How can I make it still see the robots.txt file when the .htaccess is doing a full site redirect? .htaccess currently has: Options +FollowSymLinks -MultiViews
Technical SEO | | RalphinAZ
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www.)?michaelswilderhr.com$ [NC]
RewriteRule ^ http://www.s2esolutions.com/ [R=301,L] Google webmaster tools is reporting: Over the last 24 hours, Googlebot encountered 1 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 100.0%.0 -
Duplicate Video Onsite - How do you treat this in Sitemap?
How would you handle multiple pages using the same video content? As sometimes it does not make sense to have new videos for every product so you re purpose. Will you still get the effects in search results if the thumbs and video location is duplicated for some product urls?
Technical SEO | | andrewv0 -
Use of Robots.txt file on a job site
We are performing SEO on a large niche Job Board. My question revolves around the thought of no following all the actual job postings from their clients as they only last for 30 to 60 days. Anybody have any idea on the best way to handle this?
Technical SEO | | WebTalent0 -
Video Sitemap Help
MOZ Helpers, Currently our clients videos are hosted on Viemo and that will not change as our client likes the creative/artist vibe and community via Viemo. That being said we need to create a video sitemap. BTW, Our site uses wordpress. When someone in house uploads a video in the future we want them to be able to enter the video title, description, and tags on the video and when they hit "update" the video and information will get added to our video site map. Wistia has this option here http://wistia.com/doc/video-seo , but like I mentioned above our client has all videos via Viemo. I found a Google XML wordpress plugin, but that said it only works for Youtube video's. The Jr. developer is not confident in creating one himself from the Google webmaster instructions and asked me to search for another solution. Check out the attached pic, that is what I am looking for. Is their a plugin or another option where we can use for easy sitemap updating and management for our Viemo videos? Thanks in advance! M video-seo-dialog.png?id=video-seo video-seo-dialog.png?id=video-seo
Technical SEO | | matthew-2202000