I've seen similar situations, but never in bulk and not with adult sites. Basically what's happening is somehow a domain (or multiple) are linking to your site with inaccurate URLs. When bots crawling those sites find the links pointing to yours, they obviously hit a 404 page which triggers the error in Search Console.

Unfortunately, there's not too much you can do about this, as people (or automated spam programs) can create a link to any site and any time. You could disavow links from those sites, which might help from an SEO perspective, but it won't prevent the errors from showing up in your Crawl Error report.

David,

Everything John just said in Point 2 is exactly what was running through my mind as I read your question. As the person responsible for the SEO strength of your website, you should have full control over as much of your SEO activity as possible. If your blogging platform is concerning as described, you really need to reevaluate whether that's the best thing for your site.

No problem. Your best bet would be to avoid subdomains and have separate entities via ccTLD as you've mentioned is the plan. You could do a sub-directory structure (.com/au, .com/us, etc..) but honestly that gets pretty messy. I've got a client that has some countries on different ccTLDs and some that share a TLD, and the ones that share the .com are the messy ones.

Greg,

Have you tried creating multiple XML sitemaps by section of the site, like by folder or by product detail pages? 70,000 is a huge amount of URLs and even if you could get them all on one sitemap, I wouldn't recommend it. Nesting sitemaps into an index sitemap can help Google understand your site structure and make it easier for you to troubleshoot indexing problems should they arise.

You can specify English as the language for each one. The tag's main purpose is language, but it has deeper implications than that. The hreflang tags and Search Console specification will still help your .co.uk URLs in google.co.uk, your .com URLs in Google.com, and your .com.au URLs in google.com.au.

Regarding your .eu TLD, you might need to rethink that strategy. The primary language in Aus, UK, and US is English, so the plan you've laid out will be fine for those countries. But when you get into a .eu domain, there is no Google.eu and there's no single primary language spoken throughout. Serving up English content to everyone in Europe is sub-par UX since there's a plethora of languages in the region.

Happy to help!

Hi,

There's a couple things you'll need to do. First is to set up a Search Console account for each TLD version of your domain. Once you've setup and verified, you'll want to specify the target country for each. To do this, go to Search Traffic on the left nav, then click International Targeting, then head to the Country tab. Set each for its respective domain.

Next, you'll need to implement hreflang tags. They work similar to canonical tags, but instead of telling search engines the preferential version of the content, these tags point them to the other country/language versions. Everything you need to know about this tag can be found here: https://moz.com/learn/seo/hreflang-tag.

These two actions will help the country-specific versions of Google serve up the appropriate versions of your content and avoid getting dinged for dupes.

Hope that's helpful!

Glad to help!

Yea, I wouldn't worry too much about those. You should however, specify the purpose of those parameters in Google Search Console - it'll help Google better understand their function. Here's a good guide on how to do that: http://www.shoutmeloud.com/google-webmaster-tool-added-url-parameter-option-seo.html

Hi,

I wouldn't recommend the noindex approach here. Three of your four URLs are parameter'd versions of the homepage; meta tags are applied at the page level, not by parameters. This means your noindex tag would apply to the homepage - no good!! You could put a nofollow attribute on the links that point to those parameter URLs, but don't expect it to have a huge impact on your SEO efforts. Your /members/ folder might be worth de-indexing via a noindex tag, depends on what is contained in that folder. It sounds like it's probably account-level pages, if that's the case, you can and should keep those pages out of the index.

Hope that's helpful!

Stephanie,

YouMoz is a section of the blog completely dedicated to user generated content. Though it looks like they've paused submissions while they reconfigure some things regarding YouMoz and the Moz community.

Skyler,

You're close, give this a shot:

Disallow: /*?utm_

This will be inclusive of all UTM tags regardless of what comes before the tag or what element you have first.

I just noticed that clicking on the entire slider, even out to the sides where it appears to be just white space, takes you to another page. At first I didn't realize what I was clicking that got me to the next page. When I do Crtl+A on the page, the full width of the slider images shows highlighted in blue, but to the side of those images outside of those bounds is linked. I'm wondering if Google sees this as cloaking and kicked out the homepage as a result.

*I did see that AGM pointed out it's indexed now, but that's not to say this wasn't the cause of original de-index.

You're not kidding, this does defy explanation. When did it drop out of the index?

In all honesty, I don't have a solution, you've already checked everything I would have. I'm mostly commenting so I can keep up with this issue and see how it unfolds. Very curious to see if anyone can identify what's happening here.

Google will crawl that content, but it will be devalued greatly. Any content that you find valuable to your visitors should be readily available, and preferably above the fold (which of course is not always viable). Amazon and REI both do a great job with this, the content appears to be tabbed (for reviews, descriptions, Q&A, etc), but when you click the link, it takes you further down the page via anchors.