Question about robots file on mobile devices
-
Hi
We have a robots.txt file, but do I need to create a separate file for the m.site or can I just add the line into my normal robots file.
Ive just read the Google Guidelines (what a great read it was) and couldn't find my answer.
Thanks in Advance
Andy
-
Hi Patrick
I thought that but just wanted to triple check before writing a nice list for my developers tomorrow.
Thanks
Andy
-
Hi there
Yes, if you have a mobile site, you should create a separate robots.txt and sitemap, among other things.
I would also create a Google and Bing Webmaster Tools profiles for your mobile site so you can watch it with more granularity.
Hope this helps! Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Meta robots
Hi, I am checking a website for SEO and I've noticed that a lot of pages from the blog have the following meta robots: meta name="robots" content="follow" Normally these pages should be indexed, since search engines will index and follow by default. In this case however, a lot of pages from this blog are not indexed. Is this because the meta robots is specified, but only contains follow? So will search engines only index and follow by default if there is no meta robots specified at all? And secondly, if I would change the meta robots, should I just add index or remove the meta robots completely from the code? Thanks for checking!
Intermediate & Advanced SEO | | Mat_C0 -
A Really Specific Question about 301 Redirect Strategies
Hi there: As part of a site redesign project, we've been doing a lot of 301 redirects, as we retire old URLs or rename them. My question is: is it necessary to redirect ALL old URLS? What about URLs with no links and low authority? Are these really necessary to redirect, since they're not referenced on the web and there's obviously a global redirect happening at the level of the root domain? Just curious; I'm not sure I've ever really understood this...
Intermediate & Advanced SEO | | Daaveey0 -
Does we need to add a canonical tag with the mobile url in each desktop version as a result of mobile first index?
Hi, Does we need to add a canonical tag with the mobile url in each desktop version as a result of mobile first index? Thanks Roy
Intermediate & Advanced SEO | | kadut0 -
Set Robots.txt file to crawl my website at specific times
Our website provider has stated that they can only 'lift' their block on our website in order for it to be crawled as specific times. Is there any way to amend a robots.txt to ensure that it crawls our website at a specific time of day/night in order to coincide with the block being lifted? Many Thanks, Charlene
Intermediate & Advanced SEO | | CharleneKennedy120 -
Search box within search results question
I work for a Theater news website. We have two sister sites, theatermania.com in the US and whatsonstage.com in London. Both sites have largely the same codebase and page layouts. We've implemented markup that allows google to show a search box for our site in its results page. For some reason, the search box is showing for one site but not the other: http://screencast.com/t/CSA62NT8 We're scratching our heads. Does anyone have any ideas?
Intermediate & Advanced SEO | | TheaterMania0 -
Robots.txt issue for international websites
In Google.co.uk, our US based (abcd.com) is showing: A description for this result is not available because of this site's robots.txt – learn more But UK website (uk.abcd.com) is working properly. We would like to disappear .com result totally, if possible. How to fix it? Thanks in advance.
Intermediate & Advanced SEO | | JinnatUlHasan0 -
301 redirect via htaccess question
So, I have www.sitea.com and www.siteb.com I want to redirect SOME pages from www.sitea.com to www.siteb.com (example: www.sitea.com/plugins/wp/ to www.siteb/wp/ etc ) and the rest of any pages left, to the homepage of www.siteb.com. It is something with conditions... any help, please?
Intermediate & Advanced SEO | | jasmin280 -
Crawl questions
My first website crawl indicating many issues. I corrected the issues, requested another crawl and received the results. After viewing the excel file I have some questions. 1. There are many pages with missing Titles and Meta Descriptions in the Excel file. An example is http://www.terapvp.com/threads/help-us-decide-on-terapvp-com-logo.25/page-2 That page clearly has a meta description and title. It is a forum thread. My forum software does a solid job of always providing those tags. Why would my crawl report not show this information? This occurs on numerous pages. 2. I believe all my canonical URLs are properly set. My crawl report has 3k+ records, largely due to there being 10 records for many pages. These extra records are various sort orders and style differences for the same page i.e. ?direction=asc. My need for a crawl report is to provide actionable data so I can easily make SEO improvements to my site where necessary. These extra records don't provide any benefit. IF the crawl report determined there was not a clear canonical URL, then I could understand. But that is not the case. An example is http://www.terapvp.com/forums/news/ If you look at the source you will clearly see Where is the benefit to including the 10 other records in the Crawl report which show this same page in various sort orders? Am I missing anything? 3. My robots.txt appropriately blocks many pages that I do not wish to be crawled. What is the benefit to including these many pages in the crawl report? Perhaps I am over analyzing this report. I have read many articles on SEO, but now that I have found SEOmoz, I can see I will need to "unlearn what I have learned". Many things such as setting meta keyword tags are clearly not helpful. I wish to focus my energy and I was looking to the crawl report as my starting point. Either I am missing something, or the report design needs improvement.
Intermediate & Advanced SEO | | RyanKent0