Mobile site - allow robot traffic
-
Hi,
If a user comes to our site from a mobile device, we redirect to our mobile site. That is www.mysite/mypage redirects to m.mysite/mypage. Right now we are blocking robots from crawling our m. site. Previously there were concerns the m. site could rank for normal browser searches. To make sure this isn't a problem we are planning on rel canonical our m. site pages and reference the www pages (mobile is just a different version of our www site).
From my understanding having a mobile version of a page is a ranking factor for mobile searches so allowing robots is a good thing. Before doing so, I wanted to see if anyone had any other suggestions/feedback (looking for potential pitfalls, issues etc)
-
Hi Jamie -- I noticed the mobile version of our website is appearing in the Bing search results (from a desktop) --- can you employ the same technique as Google and specify to the Bing/Yahoo! mobile bot to only crawl the mobile version of your site and restrict the normal bot from crawling mobile version? Thanks for additional guidance. I assume this would clear up this issue of our mobile site appearing in normal non-mobile search results.
- Matt
-
My preference is not to redirect to m.domain.com as there are potential issues with m.domain.com listings appearing in natural search. I would use the user agent detection, I think this gives the best user experience as well, rather than re-directing, what type of re-direct are you using anyway?
I would for best practice, still specify google mobile bot to crawl the mobile version of your website only ans restrict normal g bot form the mobile version.
Cheers,
Jamie.
-
Rather than just blocking robots from crawling your mobile site, you should configure your robots.txt file to allow the mobile googlebot to crawl that subdomain specifically and disallow it on your regular domain. This way users searching on mobile devices will be shown your mobile site. In your current set up you risk sending them to your main site from the SERPs.
If you need help getting that set up there is a tool in Google webmasters called "Generate robots.txt" located in the crawler settings under Site Configuration.
-
We don't block robots from our mobile site. Our site is designed to work well with iPad. We have found that our KWs perform slightly better when searched from a mobile device. It's a once little bonus for us. Duplicate content hasn't been a problem, I think the engines are smart enough to figure it out, even without a mobile sitemap.
-
For Google's opinion, you'll want to watch http://www.youtube.com/watch?v=mY9h3G8Lv4k, and read http://googlewebmastercentral.blogspot.com/2009/11/help-google-index-your-mobile-site.html.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Shopify robots blocking stylesheets causing inconsistent mobile-friendly test results?
One of our shopify sites suffered an extreme rankings drop. Recent Google algorithm updates include mobile first so I tested the site and our team got different mobile-friendly test results. However, search console is also flagging pages as not mobile friendly. So, while us end-users see the site as OK on mobile, this may not be the case for Google? I researched more about inconsistent mobile test results and found answers that say it may be due to robots.txt blocking stylesheets. Do you recognise any directory blocked that might be affecting Google's rendering? We can't edit shopify robots.txt unfortunately. Our dev said the only thing that stands out to him is Disallow: /design_theme_id and the rest shouldn't be hindering Google bots. Here are some of the files blocked: Disallow: /admin
Technical SEO | | nhhernandez
Disallow: /cart
Disallow: /orders
Disallow: /checkout
Disallow: /9103034/checkouts
Disallow: /9103034/orders
Disallow: /carts
Disallow: /account
Disallow: /collections/+
Disallow: /collections/%2B
Disallow: /collections/%2b
Disallow: /blogs/+
Disallow: /blogs/%2B
Disallow: /blogs/%2b
Disallow: /design_theme_id
Disallow: /preview_theme_id
Disallow: /preview_script_id
Disallow: /discount/*
Disallow: /gift_cards/*
Disallow: /apple-app-site-association0 -
Are mobile annotation in PC xml sitemaps a replacement for mobile xml sitemaps?
These two links confused me as to what I should do... https://developers.google.com/webmasters/smartphone-sites/details https://support.google.com/webmasters/answer/34648?hl=en
Technical SEO | | JasonOliveira0 -
Site not loading on Firefox
Hello guys, I can't get my website to be loaded on Firefox, why's that?
Technical SEO | | PremioOscar0 -
Micro-sites for Landing Pages?
We are working with a site that is difficult at best to update. The client intends to re-do the site in 18 months or so but needs to start generating more traffic (and sales) now. What are thoughts on creating landing pages as micro-sites that point to the current site conversion page as a stop gap? Beyond not sharing authority is there any known penalty? By the way they don't have tremendous ranking right now - often bottom of page two - and the micro-site won't duplicate any content.
Technical SEO | | InformaticsInc0 -
Traffic drop with no explaination
Hello Everyone, I made a post a few months ago about one of my sites, Poker On A Mac dropping off the face of google. I found out that it was hacked, and redirecting traffic from google to other, pharmacy websites. Doing some research, I found out it was the WordPress PharmaHack, and we ended up recovering from it. Now, since October 23rd traffic has went down again mainly organic traffic, I tried to see if it was hacked again, with the same thing and their was nothing I noticed. I was wondering if anyone could give me any insights as to why this happened? Thanks Zach CwFda.png
Technical SEO | | Zachary_Russell0 -
Site maintenance and crawling
Hey all, Rarely, but sometimes we require to take down our site for server maintenance, upgrades or various other system/network reasons. More often than not these downtimes are avoidable and we can redirect or eliminate the client side downtime. We have a 'down for maintenance - be back soon' page that is client facing. ANd outages are often no more than an hour tops. My question is, if the site is crawled by Bing/Google at the time of site being down, what is the best way of ensuring the indexed links are not refreshed with this maintenance content? (ie: this is what the pages look like now, so this is what the SE will index). I was thinking that add a no crawl to the robots.txt for the period of downtime and remove it once back up, but will this potentially affect results as well?
Technical SEO | | Daylan1 -
Site Architecture Trade Off
Hi All I'm looking for some feedback regarding a site architecture issue I'm having with a client. They are about to enter a re-design and as such we're restructuring the site URLs and amending/ adding pages. At the moment they have ranked well off the back of original PPC landing pages that were added onto the site, such as www.company.com/service1, www.company.com/service2, etc The developer, from a developer point of view wished to create a logical site architecture with multiple levels of directories etc. I've suggested this probably isn't the best way to go, especially as the site isn't that large (200-300 pages) and that the key pages we're looking to rank should be as high up the architecture as we can make them, and that this amendment could hurt their current high rankings. It looks like the trade off may be that the client is willing to let some pages be restructured so for example, www.company.com/category/sub-category/service would be www.company.com/service. However, although from a page basis this might be a solution, is there a drawback to having this in place for only a few pages rather than sitewide? I'm just wondering if these pages might stick out like a sore thumb to Google.
Technical SEO | | PerchDigital1 -
What are the pros and cons of moving one site onto a subdomain of another site?
Two sites. One has weaker sales. What would the benefits and problems for SEO of moving the weak site from its own domain to a subdomain of the stronger site?
Technical SEO | | GriffinHansen0