Client wants to remove mobile URLs from their sitemap to avoid indexing issues. However this will require SEVERAL billing hours. Is having both mobile/desktop URLs in a sitemap really that detrimental to search indexing?
-
We had an enterprise client ask to remove mobile URLs from their sitemaps. For their website both desktop & mobile URLs are combined into one sitemap. Their website has a mobile template (not a responsive website) and is configured properly via Google's "separate URL" guidelines.
Our client is referencing a statement made from John Mueller that having both mobile & desktop sitemaps can be problematic for indexing. Here is the article https://www.seroundtable.com/google-mobile-sitemaps-20137.html
We would be happy to remove the mobile URLs from their sitemap. However this will unfortunately take several billing hours for our development team to implement and QA. This will end up costing our client a great deal of money when the task is completed.Is it worth it to remove the mobile URLs from their main website to be in adherence to John Mueller's advice? We don't believe these extra mobile URLs are harming their search indexing. However we can't find any sources to explain otherwise.
Any advice would be appreciated. Thx.
-
Hey Paul
Did you get any response after tweeting Google? Thx.
-
Paul
That was an excellent response. I also appreciate you going out of your way to hit up Google directly about this as well.Yes we believe that this it is completely unnecessary to employ valuable resources to resolve a very minor issue. However our client would is going to ask us to back our argument.
Thanks again
-
As usual, Mueller's answers can be problematic because they're actually kind of vague. (e.g. his use of "if you use one of the other methods, make sure to follow those instructions separately" in that seroundtable article) Because the question asked in that article is specifically about responsive sites, non m. separate URL versions.
Here's the best I can give you... On that guidelines page you ,inked, Google specifically provides instructions for how to either include the mobile-URL versions of pages in the rel-alternate tag or by annotating the desktop sitemap to include rel-alternate info for the mobile URLS.
It does not make any mention of saying "or you can simply include the mobile URLs in the sitemap as well." Google's usually pretty good about telling us when there is more than one alternate method, while indicating which one they prefer. in this vase, I have to assume the conspicuous absence of any mention of including mobile URLs separately means it shouldn't be done.
Still conjecture, but does that make sense?
I'd definitely say it's imperative that the rel-alternate/rel-canonical treatment must be in place. Beyond that, I suspect it's a crawl budget/crawl efficiency issue, not an actual "indexing will break if mobile URLs are in sitemap" situation. As such, I wouldn't want to prioritise an expensive solution to this over whatever other more high-impact projects might be awaiting funding.
Just for the hell of it, I'll tweet at the Google guys to see if I can get a direct response to "will it cause harm" and let you know if I hear back.
I know this is just another perspective, not anything definitive, but hope it helps?
Paul
-
-
Thanks Thomas. The challenge we have is providing our client with a reputable source (not saying your not credible..lol) that states this is a negligible issue.
-
I don't believe that having the mobile urls in the sitemap is causing any issue. Due to the fact that these urls presumably can be crawled anyway on the mobile subdomain. I can't see any negative for having these urls on a sitemap.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do internal search results get indexed by Google?
Hi all, Most of the URLs that are created by using the internal search function of a website/web shop shouldn't be indexed since they create duplicate content or waste crawl budget. The standard way to go is to 'noindex, follow' these pages or sometimes to use robots.txt to disallow crawling of these pages. The first question I have is how these pages actually would get indexed in the first place if you wouldn't use one of the options above. Crawlers follow links to index a website's pages. If a random visitor comes to your site and uses the search function, this creates a URL. There are no links leading to this URL, it is not in a sitemap, it can't be found through navigating on the website,... so how can search engines index these URLs that were generated by using an internal search function? Second question: let's say somebody embeds a link on his website pointing to a URL from your website that was created by an internal search. Now let's assume you used robots.txt to make sure these URLs weren't indexed. This means Google won't even crawl those pages. Is it possible then that the link that was used on another website will show an empty page after a while, since Google doesn't even crawl this page? Thanks for your thoughts guys.
Intermediate & Advanced SEO | | Mat_C0 -
Google Is Indexing My Internal Search Results - What should i do?
Hello, We are using a CMS/E-Commerce platform which isn't really built with SEO in mind, this has led us to the following problem.... a large number of internal (product search) search result pages, which aren't "search engine friendly" or "user friendly", are being indexed by google and are driving traffic to the site, generating our client revenue. We want to remove these pages and stop them from being indexed, replacing them with static category pages - essentially moving the traffic from the search results to static pages. We feel this is necessary as our current situation is a short-term (accidental) win and later down the line as more pages become indexed we don't want to incur a penalty . We're hesitant to do a blanket de-indexation of all ?search results pages because we would lose revenue and traffic in the short term, while trying to improve the rankings of our optimised static pages. The idea is to really move up our static pages in Google's index, and when their performance is strong enough, to de-index all of the internal search results pages. Our main focus is to improve user experience and not have customers enter the site through unexpected pages. All thoughts or recommendations are welcome. Thanks
Intermediate & Advanced SEO | | iThinkMedia0 -
Dealing with Penguin: Changing URL instead of removing links
I have some links pointing to categories from article directories, web directories, and a few blogs. We are talking about 20-30 links in total. They are less than 5% of the links to my site (counting unique domains). I either haven't been able to make contact with webmasters, or they are asking money to remove the links. If I simply rename the URL (for example changing mysite.com/t-shirt.html to mysite.com/tshirts.html), will that resolve any penguin issues? The link will forward to the homepage since that page no longer exists. I really want to avoid using the disavow tool if possible. I appreciate the feedback. If you have actually done this, please share your experience.
Intermediate & Advanced SEO | | inhouseseo0 -
Website.com/blog/post vs website.com/post
I have clients with Wordpress sites and clients with just a Wordpress blog on the back of website. The clients with entire Wordpress sites seem to be ranking better. Do you think the URL structure could have anything to do with it? Does having that extra /blog folder decrease any SEO effectiveness? Setting up a few new blogs now...
Intermediate & Advanced SEO | | PortlandGuy0 -
Duplicate Content http://www.website.com and http://website.com
I'm getting duplicate content warnings for my site because the same pages are getting crawled twice? Once with http://www.website.com and once with http://website.com. I'm assuming this is a .htaccess problem so I'll post what mine looks like. I think installing WordPress in the root domain changed some of the settings I had before. My main site is primarily in HTML with a blog at http://www.website.com/blog/post-name BEGIN WordPress <ifmodule mod_rewrite.c="">RewriteEngine On
Intermediate & Advanced SEO | | thirdseo
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]</ifmodule> END WordPress0 -
Removing dashes in our URLs?
Hi Forum, Our site has an errant product review module that is resulting in about 9-10 404 errors per day on Google Webmaster Tools. We've found that by changing our product page URLs to only include 2 dashes, the module stops causing 404 errors for that page. Does changing our URL from "oursite.com/girls-pink-yoga-capri.html" to "oursite.com/girlspink-yoga-capri.html" hurt our SEO for a search for "girls pink yoga capri"? If so, by how much (assuming everthing else on the page is optimized properly) Thanks for your input.
Intermediate & Advanced SEO | | pano0 -
No index, follow vs. canonical url
We have a site that consists almost entirely as a directory of videos. Example here: http://realtree.tv/channels/realtreeoutdoorsclassics We're trying to figure out the best way to handle pagination and utility features such as sort for most recent, most viewed, etc. We've been reading countless articles on this topic, but so far have been unable to determine what might be considered the industry standard. Two solutions seem to stand out... Using the canonical url on all the sorted and paginated pages. However, after reading many blog posts, it seems that you should NEVER use the canonical url to solve the issue of paginated, and thus duplicated content because the search bots will never crawl past the first page leaving many results not in the index. (We are considering ruling this method out.) Another solution seems to be using the meta tag for noindex, follow so that a search engine like Google will crawl your directory pages but not add them to the index themselves. All links are followed so content is crawled and any passing link juice remains unchanged. However, I did see a few articles skeptical of this solution as well saying that there are always better alternatives, or that there is no verification that search engines obey this meta tag. This has placed some doubt in our minds. I was hoping to get some expert advice on these methods as it would pertain to our site. Thank you.
Intermediate & Advanced SEO | | grayloon0 -
WWW vs Non-WWW/Moving a site to a new CMS/Redirect all of the previous URLs
We are working on a new design for a website, which is currently on a CMS that has non-seo-friendly URLs. There is no redirection of 'www' to non-www or vice versa, or handling of homepage redirection so there is only one instance of 'home'. To move the site in the future, all of these URLs will have to be redirected to their new, and I hope, seo-friendly counterparts. Is it prudent now to redirect the four home page links so there is only one? and to redirect all non-www to 'www' so there is only one instance of each page? Or should I leave it and redirect all of them when the time comes?
Intermediate & Advanced SEO | | haan_seo0