Hi Rohit,

Just to follow up and add to what others have said: if you built links from third-party websites, pointing to your site, Google should crawl the third-party websites and see the links pointing to you within 15 days. If Google is not crawling the site more regularly than once every 15 days, the websites are probably lower in quality and not great sources of backlinks to begin with.

If you are using Moz / Ahrefs, etc. are slower in crawling and indexing than Google. Moz often updates once every 30 days, so could pick up these news links in its next crawl. Ahrefs tends to be slightly faster, but is still not instantaneous in picking up new links.

As others have said above, combating scrapers is a big reason, but you're relying on lazy scrapers not removing the tags.

Another reason is to prevent rogue content management systems from attaching unnecessary query strings to URLs, creating pages that can end up loading infinite times under different URLs. A canonical tag in the source file would mean that any number of duplicate pages point back to the original. The same reason goes for sites that have issues with redirecting www / non-www URLs to the correct version, or who deal with the same problem regarding secure / insecure URLs. In all these cases, the canonical tag is a bandaid, not a cure - it would be better to fix the underlying problem of the rogue CMS, incorrect redirection, etc. but the canonical tag (self-referencing) is there if you need it.

Google doesn't seem to have an issue with it, but if you're at all concerned about the other engines, use with care.

Hi Prashant,

Yes - any URLs that are different are different in Google's eyes, unless the modifier is a # symbol.

So if you have www.example.com/key#value12345 and www.example.com/key#valuexyzabc, then Google sees these as the same, i.e. www.example.com/key. They will ignore everything after the # character.

All other query strings, etc., mean that the URL has changed and if the pages on those URLs are the same, it's duplicate content.

I hope this helps.

Cheers,

Jane

Hi there,

Have you looked into the SEO guides available here on Moz? There are a lot of important on and off page factors to get right in order to gain good rankings. Have a look at the Beginner's Guide to SEO - it is not specifically about forums, but all the points included are important and count for what you're looking to do.

As Chris says, you need to look a off-page optimisation as well. It is very difficult to have a site rank well with little or no off-page work. Check out the link development category on the blog as well - there are years of great content about off-page work in there.

Cheers,

Jane

William and Gary sum it up really well here - directory listings can be useful, but they should be sought for other reasons than SEO. Noah and Eduardo are also right - the safest thing to do is to acquire these links with the nofollow tag.

If you want to take a risk or two, the highest quality directories probably won't hurt you if they're followed links. Google's stance on these things is pretty hard line though, so if you're working on behalf of a larger company or client, perhaps it's best to play it safe.

Hi Prashant,

This sounds like removal due to duplication rather than DMCA - the omission is usually noted as being because of DMCA notices if they are the reason, e.g. http://img.labnol.org/images/2008/07/googlesearchdmcacomplaint.png

Google likely sees these as duplicates, or near-dupes, as David has said,

Hi Conor,

Chris's answer here is perfect.

You definitely don't just want to link from the old site to the new; you want the old site to be "gone" and to serve a 301 redirect to tell search engines "this resource has permanently moved, and is now over here ----> [new site]".

301 redirects are vital; 302s do not pass authority from one URL to another (there is speculation that a 302 that has been in place for a very long time might be treated as a 301 by Bing, but let's go with the fact that they're next to useless in most cases for passing SEO value).

The 301 redirect should be implemented on a page by page basis, meaning that the home page at www.oldiste.com redirects to www.newsite.com. www.oldsite.com/page1.html redirects to that page's new location, e.g. www.newsite.com/page1.html, and so forth. As Chris says, you do not want www.oldsite.com/page23.html to redirect to the home page of www.newsite.com. The only case in which you'd want to do this is if /page23.html no longer exists on the new site.

How this is done depends on what type of server your site sits on. If you are using an Apache server, the web developers will do this in the site's .htaccess file. If you are using a Microsoft IIS server, this will be done in a control panel. The web development company will clearly know which server you're using and where this will be done.

Some servers and hosts still implement 302 redirects as standard and not 301s, which is very frustrating from an SEO point of view. There should be nothing more difficult about creating a 301 than a 302.

The short version is: 301 the old site to the new if you want to transfer the authority from the old site to the new one. This will ensure that you bring most of that authority with you.

Hi again,

Yep - your non-www and www pages are both resolving... e.g. http://cheaptubes.com/ and http://www.cheaptubes.com/ bring up the same content. Also, http://cheaptubes.com/default.htm and http://www.cheaptubes.com/default.htm is also a duplicate of the home page.

Internally, I am seeing the same thing, e.g. http://www.cheaptubes.com/carbon-nanotubes-prices.htm and http://cheaptubes.com/carbon-nanotubes-prices.htm - same page, one on the www subdomain ("www." is a subdomain like any other, just with an extremely common name) and one just sitting on the root.

The solution here is either to 301 redirect the non-www version of the site to the www version for every page, or to use the canonical tag to point from the non-preferred versions to the "canonical" versions. More information on this is available here.

You also have a situation where upper-case URLs will resolve as well as lower case ones, e.g. http://www.cheaptubes.com/SWNTs.htm and http://www.cheaptubes.com/swnts.htm (as well as http://cheaptubes.com/swnts.htm!).

URLs should only be allowed to resolve with one case, preferably lower. The upper / mixed case should 301 redirect to the proper version.

Essentially, the "two versions of the site" issue is the biggest problem, with all pages being available on at least two URLs - one with www and one without. There are other tidiness issues like /default.htm bringing up the home page as well.

Does this make sense? Let me know if this is not clear.

Best,

Jane

EGOL and Samuel raise some very good points here. I feel like you're also somewhat at risk of looking like you're trying to deceive the audience if you're deliberately divorcing the community from the brand site and image, but using it to drive the community to commercial action. I am not sure how subtle they are planning on making that connection, but subtlety in that manner rarely works: it's a lot more "honest" and appreciated to cultivate a community and openly market a product, as Moz has done.

Using the Moz example, this site did not start out as a tool / analytics provider, so the community actually came first. When I was a full time employee, we were largely an SEO consultancy. The blog was a labour of love as far as Rand and the staff were concerned. In 2007 when we first introduced Premium (now Pro) accounts, we clearly used the community (which was pretty substantial, even back then) to market the service. To all but a very few negative folks, this went down fine.

If Rand had started a separate company and website hosting and selling SEO tools and subscriptions, and had dropped hints and posts every now and again here to promote it, it would not have been met with nearly as much favour.

The SEO points about two sites being twice as hard to maintain, market and rank than one also apply, but I would question whether a smaller brand can build a totally separate community that is a) big enough, and b) independent enough not to annoy users if they figure out the commercial intent behind it. If there's one thing that's slightly worse than no community, it's an empty one.

Hi Damien,

So if I'm reading this correctly, the consultant is saying that due to the size of the site (tens of thousands of pages), and the need to categorise its content, that subdomains are the best choice.

I would say that there are far bigger websites using categories within subfolders, notably big retailers, e.g.

http://www.marksandspencer.com/c/beauty, http://www.marksandspencer.com/c/food-and-wine, http://www.marksandspencer.com/c/mands-bank

http://www.waitrose.com/home/inspiration.html, http://www.waitrose.com/home/wine.html, http://www.waitrose.com/content/waitrose/en/home/tv/highlights.html (<-- the last one being a crappy version, but a subdomain nonetheless)

and so do websites that deal with providing content for very different audiences:

http://www.ncaa.com/schools/tampa, http://www.ncaa.com/championships/lacrosse-men/d1/tickets, http://www.ncaa.com/news/swimming-men/article/2014-03-29/golden-bears-and-coach-david-durden-earn-third-national-title, http://www.ncaa.com/stats/football/fbs

Has the consultant provided examples of other websites doing this that would take on the same structure?

There are hundreds of examples of websites whose structure / categories are properly understood despite existing in subdirectories, so I'm still sceptical that this is a necessity.

This is not to say that a subdomain approach wouldn't work and is definitively bad or anything, I'm just not really convinced that the reasoning is strong enough to move content away from the root domain.

I disagree about user experience - from a user's perspective, the only difference between subfolders and subdomains is the URL they can see in the address bar. The rest is aesthetic. You can do or not do everything you'd do with the design of a website using subdirectories that you'd do with a website(s) employing subdomains. For example, just because content sits on www.icy-veins.com/wow/, its navigation wouldn't have to link to www.icy-veins.com/hearthstone/ or mention the other brand in any way if you don't want to. You can still have separate conversion funnels, newsletter sign-ups, advertising pages, etc.