Www or no www
-
right now our organization has the a robot text file set to block search engines from indexing the non-www version of our URL - example.com
looks like this
#Dynamically generated robots.txt file for example.com
User-agent: *Disallow: /
Of course, we do allow search engines to crawl the www. version
Is this the best way to do this?
-
Likewise to you Simon, thanks a lot for the advice!
-
Thanks so much Kevin. Much appreciated!
-
I wouldn't recommend setting your robots.txt to disallow indexing of the site without www. The best method is to create a domain wide 301 redirect using .htaccess. You should also set it up in Google Webmaster tools under settings and select with www as your preferred domain. Below is an example of the rewrite in .htaccess.
RewriteEngine on
RewriteCond %{HTTP_HOST} ^mysite.com [NC]
RewriteRule (.*) http://www.mysite.com/$1 [L,R=301]
Hope that helps!
Kevin
-
Hi Kevin
That is a risk, you could end up blocking search engines from indexing your entire website. Usually, both www and non-www point to the same location, so one robots.txt file can apply to both (unless you've configured your site differently).
Check out http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=44231
I'd suggest removing that Disallow command within your robots.txt file and do the following:
Within Google Webmaster Tools, go to 'Site configuration' 'Settings' ~ from there you can set how you'd like your pages to be displayed in Google's search results, either with or without the www. (or can leave Google to decide).
Regards
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Please who can help me rank my website www.shopdocuments.com ?
Hi , i bought my website this month , i built it myself now i want it to grow seo on google pages and stuff tried google ads but they always block my account don't know why so somebody help me please any company? individual willing to work it for me ? even if paid company no problem
Technical SEO | | planetdocs1 -
Weird Google indexing issues with www being forced
IM working on a site which is really not indexing as it should, I have created a sitemap.xml which I thought would fix the issue but it hasn't, what seems to be happening is the Google is making www pages canonical for some of the site and without www for the rest. the site should be without www. see images attached for a visual explanation.
Technical SEO | | Donsimong
when adding pages in Google search console without www some pages cannot be indexed as Google thinks the www version is canonical, and I have no idea why, there is no canonical set up at all, what I would do if I could is to add canonical tags to each page to pint to the non www version, but the CMA does not allow for canonical. not quite sure how to proceed, how to tell google that the non www version is in fact correct, I dont have any idea why its assuming www is canonical either??? k11cGAv zOuwMxv0 -
Redirect our www.website.com to website. Com/target-keyword
Dear moz community I have been analyzing the websites that rank in top 20 for our target keywords. All of the top 20 sites except us have their websites re directing to websites.com/target keyword. This is due to probably because they have multiple city's and one of the target keywords term is cityname + word. My question is and idea - make a 301 to our www.website.com to /city-keyword and start linking to that page with new links. Would that bring any benefit? Seems that it's a very strong ranking signal. Any threats that I must take into account? We currently rank as #9
Technical SEO | | advertisingcloud0 -
Www & non-www URL Issue
Hi is there any tools out there which can enable me to find pages which both serve www and non-www versions of each URL
Technical SEO | | monster990 -
Www vs non-www which is better?
Is it better to have all your pages point to the www version or non www version.
Technical SEO | | bronxpad0 -
A rel="canonical" to www.homepage.com/home.aspx Hurts my Rank?
Hello, The CMS that I use makes 3 versions of the homepage:
Technical SEO | | EvolveCreative
www.homepage.com/home.aspx homepage.com homepage.com/default.aspx By default the CMS is set to rel=canonical all versions to the www.homepage.com/home.aspx version. If someone were to link to a website they most likely aren't going to link to www.homepage.com/home.aspx, they'll link to www.homepage.com which makes that link juice flow through the canonical to www.homepage.com/home.aspx right? Why make that extra loop at all? Wouldn't that be splitting the juice? I know 301's loose 1-5 % juice, but not sure about canonical. I assume it works the same way? Thanks! http://yoursiteroot/0 -
Www vs non-www
We just had our site redesigned. Previously, it was indexed under www.suss.net, but now the developer has it at suss.net with www.suss.net 301 redirecting to suss.net. Is this bad for SEO?
Technical SEO | | kylesuss0 -
Follow up from http://www.seomoz.org/qa/discuss/52837/google-analytics
Ben, I have a follow up question from our previous discussion at http://www.seomoz.org/qa/discuss/52837/google-analytics To summarize, to implement what we need, we need to do three things: add GA code to the Darden page _gaq.push(['_setAccount', 'UA-12345-1']);_gaq.push(['_setAllowLinker', true]);_gaq.push(['_setDomainName', '.darden.virginia.edu']);_gaq.push(['_setAllowHash', false]);_gaq.push(['_trackPageview']); Change links on the Darden Page to look like http://www.darden.virginia.edu/web/MBA-for-Executives/ and [https://darden-admissions.symplicity.com/applicant](<a href=)">Apply Now and make into [https://darden-admissions.symplicity.com/applicant](<a href=)" > onclick="_gaq.push(['_link', 'https://darden-admissions.symplicity.com/applicant']); return false;">Apply Now Have symplicity add this code. _gaq.push(['_setAccount', 'UA-12345-1']);_gaq.push(['_setAllowLinker', true]);_gaq.push(['_setDomainName', '.symplicity.com']);_gaq.push(['_setAllowHash', false]);_gaq.push(['_trackPageview']); Due to our CMS system, it does not allow the user to add onClick to the link. So, we CANNOT add part 2) What will be the result if we have only 1) and 3) implemented? Will the data still be fed to GA account 'UA-12345-1'? If not, how can we get cross domain tracking if we cannot change the link code? Nick
Technical SEO | | Darden0