Why are these m. results showing as blocked?
-
If you go to http://bit.ly/173gdWK, you'll see that m. results are showing as blocked by robots.txt, but we don't have anything in our robots.txt file that specifies to block m. results. Any ideas why these URLs show as blocked?
-
Yeah, I was testing exactly the same when you posted the response. I even tried crawling a googlebot-mobile and still I get the 301 redirect. Which, for everything that I am seeing it is correct, as no matter what browser I use (desktop, mobile, spider) I always get a 301 to the www. version.
@michelleh, are you sure there's a mobile version not being redirected to the www. one?
-
(Using example.com instead of your domain in case you want anonymity later)
If you try to go to any of these m.example.com URLs on your desktop computer, you're redirected on the server to a www.example.com URL. I'm guessing Googlebot and Googlebot-Mobile cannot access m. pages (unless you're sniffing out Googlebot-Mobile specifically to serve it m.example.com pages). If you're looking at screen resolution for these redirects, you might not be catching Googlebot-Mobile, as I don't think Googlebot-Mobile gives a screen resolution in its user agent. I believe you want Googlebot indexing your www. content, and Googlebot-Mobile indexing your m. content, so you'll need to sniff out Googlebot-Mobile's user agent (see here), and redirect it to m. content.
Also of note is that I think these should be 302 temporary redirects, and not 301 permanent redirects between your www. and m. versions, as they're not really permanent, just getting a given user to the right version of the site. Also, you don't let me switch from the mobile version to the desktop version, which drives me bananas! Let users choose after the initial redirect. If you allow people to switch, but maintain 301 redirects, the browsers may cache some of the redirects which will lead to weird behavior if people hit a page that redirected before.
You don't have a robots.txt file at m.example.com/robots.txt, as that redirects to www.example.com/robots.txt even on my phone. I don't think this is the root of the problem, but once you figure things out, you can set up a robots.txt file on your m. subdomain.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is Moz Premium Show Authenticate Keyword Difficulty States???
Hi, I am a regular user of Moz premium. I was advised some of my friends about Moz benefits. 4 months ago I found a low competition micro niche longtail keyword and made a website on it. The keyword difficulty was 16 and I was confident that I will rank on the top page. but now after posting 15 articles of that micro niche, I am still ranking on 129 position on Google search results. Can anybody advise me what is the problem with my site. Was the keyword difficulty had manupulated states on authenticative. My site name and https://www.vixpr.com/. Please guide me how I can take further benefits of Moz Premium.
Intermediate & Advanced SEO | | Iuj873g0 -
UK version of site showing US Cache and meta description
Hi Fellow Moz'ers We seem to have an issue where some of our UK site is showing meta descriptions from our US site in the serp's and when you check the cache: of the site it's brining up the .com instead of the .co.uk site. example: cache:https://www.tinyme.co.uk/name-labels shows the US site We've checked the href lang tags and they look ok to me (but i'm not an expert) https://www.tinyme.co.uk/name-labels" hreflang="en-gb"/> https://www.tinyme.com/name-labels" hreflang="en-us"/> https://www.tinyme.com.au/name-labels" hreflang="x-default" /> https://www.tinyme.com.au/name-labels" hreflang="en-au"/> We've had a search around and seen people have similar issues, but cant seem to find a definitive solution.
Intermediate & Advanced SEO | | tinyme1 -
I'm in Canada and building a website for the US...approach?
Hi there - we already have a Canadian website for the company and we're building one for our American branch. From an SEO perspective what is the best approach here? We have already purchased a .com domain and the company is branded a little different in the US than in Canada. How do I tell Google that this site is American and should be served primarily to the American audience? Should I be tagging duplicate content with rel=canonical (for similar pages like the About us section for instance) or does that matter here? Hope you guys can help. Thanks!
Intermediate & Advanced SEO | | MelcorDev0 -
Search Results Pages Blocked in Robots.txt?
Hi I am reviewing our robots.txt file. I wondered if search results pages should be blocked from crawling? We currently have this in the file /searchterm* Is it a good thing for SEO?
Intermediate & Advanced SEO | | BeckyKey0 -
Blank Cart Pages Showing as Duplicate, HELP
Hi Everyone, I'm seeing a bunch of URLs that look something like this [ domain.com/cart?add&id_product=42&token=776d4a08721f3d8c920e287248797547] showing as duplicate content in my Moz crawls. I think these are just blank pages for the most part. Is there anything to be concerned with here? Is there a way to clean this up? Thanks! Ricky
Intermediate & Advanced SEO | | RickyShockley0 -
I'm invisible!
Dear all, Earl 2015 i launched www.taobao.nl. Its a platform that shows the products of chinese marketplaces in our local language (auto translated) and provides the option to pay in our local currency & methods. (i blocked all the auto translated product pages via http://www.taobao.nl/robots.txt) All content (category pages, general pages etc) are unique and written by us. In the past I've set up several websites and shops and i know it always takes time and effort to get a site to good positions, but i never experienced that my efforts did not pay of. Until now 😞 The url is not blocked by Google and i see a (very) limited amount of traffic in WMT & Analytics, but the site is >page 30 on its most important keyword 'taobao'. This, while the competition on this keyword is low in my country (google.nl). My main competitor (taobao.com.nl) seems to do worse on most SEO elements i was able to check, but they rank very well on 'taobao' and all related keywords. What could be the reason of this very poor ranking and what should i really adjust / improve?
Intermediate & Advanced SEO | | benhond
(besides backlinks: i'm working hard on that) Your input is deeply appreciated!! Sander1 -
Internal Search Results Appear in Google SERPS
My friend is running an ecommerce store selling apparels. How can we make internal search results to appear in Google SERPS and rank them? For example: the query is "peplum dress". You type the query into the internal search box and it returns a set of results. In this case, it's product listing. How can we optimize and rank it so it appears in Google SERP? Do we do it the traditional way in terms of links? Say URL is: http://www.asos.com/search/peplum-top?q=peplum+top&r=2 And we build links to it? Some of you may ask why not create a dedicated page for this, the reason being we'd have too many categories if we were to create one for each. Thoughts?
Intermediate & Advanced SEO | | WayneRooney0 -
Can I compete with these results? (Brand in Serp)
Hey, One quick question. Lets say im fighting for keyword "british airways" and i want to appear straight after first result in number 2 position. Is it possible to compete with stroked results. (See image attached) Thanks Stxct.png
Intermediate & Advanced SEO | | Marteen0