Why are these m. results showing as blocked?
-
If you go to http://bit.ly/173gdWK, you'll see that m. results are showing as blocked by robots.txt, but we don't have anything in our robots.txt file that specifies to block m. results. Any ideas why these URLs show as blocked?
-
Yeah, I was testing exactly the same when you posted the response. I even tried crawling a googlebot-mobile and still I get the 301 redirect. Which, for everything that I am seeing it is correct, as no matter what browser I use (desktop, mobile, spider) I always get a 301 to the www. version.
@michelleh, are you sure there's a mobile version not being redirected to the www. one?
-
(Using example.com instead of your domain in case you want anonymity later)
If you try to go to any of these m.example.com URLs on your desktop computer, you're redirected on the server to a www.example.com URL. I'm guessing Googlebot and Googlebot-Mobile cannot access m. pages (unless you're sniffing out Googlebot-Mobile specifically to serve it m.example.com pages). If you're looking at screen resolution for these redirects, you might not be catching Googlebot-Mobile, as I don't think Googlebot-Mobile gives a screen resolution in its user agent. I believe you want Googlebot indexing your www. content, and Googlebot-Mobile indexing your m. content, so you'll need to sniff out Googlebot-Mobile's user agent (see here), and redirect it to m. content.
Also of note is that I think these should be 302 temporary redirects, and not 301 permanent redirects between your www. and m. versions, as they're not really permanent, just getting a given user to the right version of the site. Also, you don't let me switch from the mobile version to the desktop version, which drives me bananas! Let users choose after the initial redirect. If you allow people to switch, but maintain 301 redirects, the browsers may cache some of the redirects which will lead to weird behavior if people hit a page that redirected before.
You don't have a robots.txt file at m.example.com/robots.txt, as that redirects to www.example.com/robots.txt even on my phone. I don't think this is the root of the problem, but once you figure things out, you can set up a robots.txt file on your m. subdomain.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to find Topics ? software or results and user intent
Hello, Is there a software that is better than an other to find to right topics to cover in my content. I am thinking about Moz, Marketmuse or Semrush or is it better to look at the search results because they match user intent and see what is covered and cover those in my content Thank you,
Intermediate & Advanced SEO | | seoanalytics1 -
I'm stumped!
I'm hoping to find a real expert to help out with this. TL;DR Our visibility in search has started tanking and I cannot figure out why. The whole story: In fall of 2015 I started working with Convention Nation (www.conventionnation.com). The client is trying to build a resource for convention and tradeshow attendees that would help them identify the events that will help them meet their goals (learning, networking, sales, whatever). They had a content team overseas that spent their time copy/pasting event information into our database. At the time, I identified several opportunities to improve SEO: Create and submit a sitemap Add meaningful metas Fix crawl errors On-page content uniqueification and optimization for most visible events (largest audience likely to search) Regular publishing and social media Over nine months, we did these things and saw search visibility, average rank and CTR all double or better. There was still one problem, and that is created by our specific industry. I'll use a concrete example: MozCon. This event happens once a year and there are enough things that are the same about it every year (namely, the generalized description of the event, attendees and outcomes) that the 2015 page was getting flagged as a duplicate of 2016. The event content for most of our events was pretty thin anyway, and much of it was duplicated from other sources, so we implemented a feature that grouped recurring events. My thinking was that this would reduce the perception of duplicate or obsolete content and links and provide a nice backlink opportunity. I expected a dip after we deployed this grouping feature, that's been consistent with other bulk content changes we've made to the site, but we are not recovering from the dip. In fact, our search visibility and traffic are dropping every week. So, the current state of things is this: Clean crawl reports: No errors reported by Moz or Google Moz domain authority: 20; Spam score 2/17 We're a little thin on incoming links, but steady growth in both social media and backlinks Continuing to add thin/duplicate content for unique events at the rate of 200 pages/mo Adding solid, unique strategic content at the rate of 15 pages/mo I just cannot figure out where we've gone astray. Is there anything other than the thin/copied content that could be causing this? It wasn't hurting us before we grouped the events... What could possibly account for this trend? Help me, Moz Community, you're my only hope! Lindsay
Intermediate & Advanced SEO | | LindsayDayton0 -
Block subdomain directory in robots.txt
Instead of block an entire sub-domain (fr.sitegeek.com) with robots.txt, we like to block one directory (fr.sitegeek.com/blog).
Intermediate & Advanced SEO | | gamesecure
'fr.sitegeek.com/blog' and 'wwww.sitegeek.com/blog' contain the same articles in one language only labels are changed for 'fr' version and we suppose that duplicate content cause problem for SEO. We would like to crawl and index 'www.sitegee.com/blog' articles not 'fr.sitegeek.com/blog'. so, suggest us how to block single sub-domain directory (fr.sitegeek.com/blog) with robot.txt? This is only for blog directory of 'fr' version even all other directories or pages would be crawled and indexed for 'fr' version. Thanks,
Rajiv0 -
No matter what the keyword, only the homepage shows in the SERP
Hi, wondered if someone could help. My clients website shows up well for terms but its always the homepage rather than the targeted landing page. For example, if you search for "teeth whitening anglesey" they appear http://goo.gl/ohJdua however, its the homepage rather than the tooth whitening page http://goo.gl/uVI8gK Thanks Ade
Intermediate & Advanced SEO | | popcreativeltd0 -
When I type link:mydomainname.com in Google I don't see any result, why?
My other website is 4 years old and Page Rank 3. We are into business of design and development for 5 years and still we don't have any result from Google Searches. When I type link:mydomainname.com I don't get any result. What's the reason?
Intermediate & Advanced SEO | | vikaspooja1 -
Google Preview not showing images
No matter which our pages i find in the SERP the Google Preview does not show images for www.spies.dk Try searching for "rejser til malaysia" on google.dk. The result www.spies.dk/malaysia is not showing images. Why is that? Using the "google preview tool" under labs in Google Webmaster Tools sys it found 13 errors on that page: | Ressource: | Information: http://images2.spies.dk/images/SiteID11/SiteLayout/logo-spies.png?v=4 | Indekseret http://images1.spies.dk/images/StaticLayout/Spies_hori_sRGB.png?v=1 | Indekseret http://images1.spies.dk/images/SiteID11/Button/search-button-text3.png?v=2 | Indekseret http://images2.spies.dk/images/Country/my1001_10_48.jpg?v=1 | Indekseret http://images2.spies.dk/images/SiteID11/Button/Opdater_bt.png?v=2 | Indekseret http://images1.spies.dk/images/SiteID11/Button/Sog_bt.png?v=3 | Indekseret http://images2.spies.dk/images/Country/my1002_2_22.gif?v=1 | Indekseret http://images2.spies.dk/images/Resort/bkikki1001_4_11.jpg?v=1 | Indekseret http://images2.spies.dk/images/SiteID11/Button/Vaelg_bt.png?v=7 | Indekseret http://images1.spies.dk/images/Resort/bkiskn1001_4_11.jpg?v=1 | Indekseret http://images1.spies.dk/images/Resort/kulkum1001_4_11.jpg?v=1 | Indekseret http://images1.spies.dk/images/Resort/lgklak1001_4_11.jpg?v=1 | Indekseret https://track.adform.net/serving/scripts/trackpoint/async/ | Indekseret |
Intermediate & Advanced SEO | | alsvik0 -
I'm afraid I may have messed up my site's organization
So I recently started working on an existing site for a company, and I'm afraid I may have done something to lose some backlinks. So to start off, say the website is www.domain.net and when I arrived domain.net and www.domain.net showed up as two separate sites so I changed my web.config file to direct all domain.net to www.domain.net The homepage was called default.asp, and I wanted the homepage to always show up as www.domain.net instead of www.domain.net/default.asp. Of course they both showed the same thing but I couldn't figure it out. So I removed www.domain.net/default.asp from indexing and changed the my internal links to the homepage to point at www.domain.net instead of simply pointing at the file default.asp. So now www.domain.net/default.asp still brings up the page, but I want it to revert to www.domain.net. I'm also a little worried because I noticed that one of my incoming links points at www.domain.net/default.asp and it doesn't get passed along to www.domain.net and I think i may have damaged my sites SEO I guess this is a very complicated and roundabout way of saying this, but how can I get www.domain.net/default.asp to take you to www.domain.net
Intermediate & Advanced SEO | | bcrabill0 -
Why do I have better results with absolute search
Hi, I completed the optimization of my website, and while my rankings have improved, i seem to have consistently better resultats with absolute search terms: ex: i'm better ranked on "word1 word2" than just word1 word2 without the guillemets. Do you have any idea why? As most people perform search without guillemets, i would like to improve my rankings with these searches too. thanks, cedric
Intermediate & Advanced SEO | | smartgrains0