Robots file include sitemap
-
Hello,
I see that google, facebook and moz... have robots.txt include sitemap at the footer.
Eg: http://www.google.com.vn/robots.txtSitemap: http://www.google.com/sitemaps_webmasters.xml
Sitemap: http://www.google.com/ventures/sitemap_ventures.xml Should I include my sitemap file (sitemap.xml) at the footer of robots.txt and why should do this?Thanks,
-
Fore sure. The reason is that not all sitemap files are simply sitemap.xml it may be sitemap.gzip sitemap.zip or in my case, sitmap_index.gzip. Also, some people may not be able to include their sitemap at root.
Including sitemap exacts in robots.txt gives a clear directive to each of the search engines exactly where to find your sitemap. Google and Bing/Yahoo will have no issue finding it as you probably submitted it to them but crawlers like ask.com will usually look at your robots.txt and skip your site if no map is placed in it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hello, I've heard that the outbound links I provide in my content should have a high degree of relevancy to the topic I'm writing about or they aren't really worth including. Is this true?
Hello, I've heard that relevancy of the content between the source page and the target page of outbound links in my content matters greatly. The outbound links I provide in my content should have a high degree of relevancy to the topic I'm writing about, or they aren't really worth including. Example: Don't just link to the homepage of an organization mentioned in the article, link to a page on their site that is related to the topic you are writing about. Is this true? Would including less relevant links negatively impact SEO in any way?
On-Page Optimization | | DJBKBU0 -
[HELP!] File Name and ALT Tags
Hi, please answer my questions: 1. Is it okay to use the same keyword on both file name and alt tags when inserting an image? Example: File Name: buy-lego-online.jpg ALT tag: buy-lego-online Will it trigger Google Panda? Will I be penalized for that? Or the file name and alt tags should be different from each other? Because when inserting an image on Wordpress, the alt tags are always the same as the file name by default. 2. For example, I have 2 images in a page (same topic/niche) and I will put "cheap-lego-for-kids" and "best-lego-for-sale" as alt tags. Considering that I repeat the word "lego", is it considered keyword stuffing? Will I be penalized for that? Thanks in advance!
On-Page Optimization | | bubblymaiko0 -
Is it OK to include name of your town to the title tag or H1 tag on a blog to enhance local search results
I recently attended a webinar by ETNA Interactive on local search SEO. The presenter recommended including the name of your town in the title of the blog to increase local search SEO. Is this OK? Ive always been concerned that it is such an obvious attempt to rank locally that Google would consider it "spammy" ? black hat, "sketchy" or otherwise manipulative. Have the rules changed? Is it OK to do? Brooke
On-Page Optimization | | wianno1680 -
Sitemap.xml parameters
Hello, I confused about the parameters when create sitemap.xml. I don'n know how to use these parameters efficiently. Some parameters are:
On-Page Optimization | | JohnHuynh
- **Page changing frequency: **what is a good value? Can I choose "None"?
- **Last modified date: **what is a good value? Can I choose "Don’t specify"?
- **Page priority: **what is a good value? Can I choose "Don’t specify"? Thanks,0 -
Video Sitemap Generator
Do you have a favorite tool for generating video sitemaps? Best,
On-Page Optimization | | ChristopherGlaeser
Christopher0 -
If you were working on a wine site would you include the wine year in the URL?
I've come across a case where I'm asking myself what the best direction would be to go and while there is no right direction I would like to here some feedback from others. I'm working with some great content pages all about wine. As you probably know the difference between a 07 wine and a 95 is vastly different and up to this point I'm using the full year in the url much like this: grapesinyourtoesexample.com/2007-cellar-pod-viognier-adelaide-hills/. What I'm worried about is my use of the year in the URL. I feel it's very important for it to be used in the page title and on page but I'm concerned that it might be setting me back with my use of it in the url. My concern is that search engines might be interpretting it as a datestamp rather than as a informational piece of data describing the asset. Looking at my competitors, my content is one of the only sites using the year and in most searches for various wines my content is in the second half of the SERPs. If you were creating this content would you use the year? If you were working with current content would you drop the year across all of the site and implement to necessary redirects? Just to be clear this is a client related project so my use of "my site|my content" refers to the client's content.
On-Page Optimization | | DotCar0 -
No follow for html sitemap?
Hi, I have been working on an e-commerce site and have been wondering if i should add the meta robot tag with no follows, on pages like delivery , terms, returns and the html sitemap? From what I have read this seems a good idea, but i am a little confused as what to do with the html sitemap. I can understand that having this on the homepage will allow crawlers quick access to the deeper pages within the site. But would it be better to guide them down the natural route of the category navigation instead? Thanks
On-Page Optimization | | Bmeisterali0 -
I have a direct question about file structure.
This question is about a new file structure and SEO friendly URL's. Does a file name make a difference? I have a direct question about file structure. Our old site was formated with a URL of http://rousechamberlin.com/about_us.aspx our new site is structured http://rousechamberlin.com/AboutUs/ no file no extension. As the SEO guy of the company and not the programmer my feeling is this is killing us. Does anybody have any thoughts on this?
On-Page Optimization | | HeadWebChef0