Welcome to the Q&A Forum

What an aggravating mess to have to clean up!

You're going to need to give Google stronger and less ambiguous directions for what to do with the offending domain. Right now, with the HTTPS version not redirecting properly, G is getting contradictory signals that will cause it to leave the HTTPS version in the index and prolong your issues.

The first step is to add an SSL certificate for the spam domain to the client's primary domain. I know this sounds counterintuitive, but it is required procedure in order for the redirect of the HTTPS version of the spam domain to work.

When a browser (or the Google crawler) tries to follow that HTTP redirect, the very first thing it must do is negotiate the connection. Because it's an HTTPS URL, that connection is going to fail because there is no SSL certificate present. Which means Googlebot can never get to the point where it can even see the 301 redirect. So Google thinks the spam domain is still active and leaves it in the index.

So step one is to get an SSL certificate for the www and non-www versions of the spam domain installed on the primary domain so the redirect will work.

The next step is to get the spam domain verified in its own Google Seach Console so that you can give Google direct instructions regarding what you want it to do with that domain.

Since you've deleted the spam domain's hosting, you'll need to Verify the spam domain in Google Search Console by using the DNS verification method where you add a TXT record to the spam domain's DNS records.

Once you have the spam domain verified and the HTTPS redirect is working properly, you can use the Change of Address tool in Search Console to point the spam domain to the primary domain. (The CoA tool must see a working 301 redirect in order to process the change.)

Note! At this point, I would use the Change of Address rather than the Remove URL tool in order to push any relevance signals of the spam domain to the primary instead of just dropping them altogether.

The additional benefit of the Change of Address is that the old spam domain will almost immediately drop out of the SERPs.

As a final step, it would be beneficial to get the individual spam URLs redirecting to their equivalents on the primary site. If the spam site used exactly the same URL structure, your domain redirect will take care of this. But otherwise, it would be beneficial to harvest as many of the spam URLs as possible and add 301 redirects for them into the htaccess file of the primary site. Again, we're trying to give Googlebot as many unambiguous signals as possible about what to do with the authority of those spam URLs.

Does that all make sense?

The short answer is - you can track up to 10 websites on your Medium subscription, @kassandrasharr .

That's the staging URL for GoDaddy-hosted sites, Ryan. It likely means that when the site was being developed under the hosting account's temporary URL, it got indexed by the search engines.

It's also possible there's a domain resolution issue on the hosting server. Are there any other pages showing this problem?

The reason its display is broken is because the CSS and JavaScript on the page will be getting called from the wrong URL, so they can't resolve and format the page content.

Paul

Knowing that with a large body of documentation like this, the chances of being able to rewrite it all to combine into a single page are pretty slim (and knowing that might be a very negative user experience) you're really only left with the canonical tag option - assuming the older docs need to be maintained.

You're right to be concerned, as Google has been clear that canonical only applies to pages that have substantially identical content. Unfortunately, they do a really poor job of explaining just how much variation would be allowed.

Is it okay if the canonical is not an exact duplicate of the content?
We allow slight differences, e.g., in the sort order of a table of products. We also recognize that we may crawl the canonical and the duplicate pages at different points in time, so we may occasionally see different versions of your content. All of that is okay with us.
~ https://webmasters.googleblog.com/2009/02/specify-your-canonical.html

My impression is that they would honour canonical in your use case.

Really, the only way to know is to select a couple of products' documentation pages and conduct a test. Canonicalise all old version to the current version and request re-indexing for each page. Then monitor the results (The new index monitoring tools in the new GSC are useful for this). You'll want to choose at least one test case that involves featured snippets - it would be incredibly useful to know if the FS transfers across to the new canonical page!

Do note that you'll need an ongoing process for managing the canonicals s each new iteration of documentation is added - all related pages will need to have their canonicals updated to point to the newest each time new docs are published.

Interesting conundrum. Please let us know the results if you decide to try a test!

Is that useful?

Paul

Please don't hijack someone else's question, zecase. Much better if you start your own question as it's completely unrelated to the current one.

Paul

Absolutely do NOT change from the www just for the sake of saving four characters in the URL. It would mean that every single page on your site would now have to go through a 301-redirect.

You should only change URLs when absolutely necessary, and the reasoning for this change certainly doesn't qualify!

Also worth noting - a www URL has more technical flexibility. Those reasons may not apply to you now but could in the future.

Paul

Your understanding of canonical tags is correct, GhillC.

If Tools are showing errors for those canonical tags you've listed, then the tools are wrong.

As long as the protocol and subdomain prefix (or not) exactly match and the only difference is the exclusion of the parameters (the "?" and the stuff after it) then the canonicals are correct.

Any tool's reports have to be filtered through your own understanding and knowledge. They often get things wrong. That's on eof the key differences between experienced SEOs and less-experienced. They kow when to question what an automated tool is telling them. So good on ya for questioning the results!

Paul

Direct traffic isn't a ranking factor, VT. There's no way it can be, as it would far to easy a signal to fake and abuse.

Paul

A proper canonical tag does a lot more than "just be telling Google not to rank it" When used properly (i.e. pages that truly do contain the same content), the canonicalised page passes its ranking signals back to the canonical source.

I agree with Kristina - while a 301 would be preferable (it's a directive, while canonical tags are taken as suggestions), a canonical tag would be vastly better than not doing anything about the issue. At least until the dev can get the problem with the 301-redirect properly resolved.

Paul

My suspicion would be that those URLs are accessible somewhere without the second variable. (I.e. That second variable isn't always present when the first variable is.)

Great to hear you've been able to fix it. Nice job tracking down the extra tag in GTM.

If you haven't also done so, it'll be worth looking at the Behaviour > Site Content > All Pages report to check out the actual bounce rate for a wide selection of pages. Just to make sure there aren't any problematic tags triggered for individual pages as well.

(And thanks for the Good Answer vote )

Paul

You have an error in your Analytics configuration in Google Tag Manager. It is causing two pageviews to be fired for each visit. (Note - all you pageview data and related - pgs/visit, conversion rate etc - will be incorrect and unreliable from that date, not just your bounce rate.)

If you click on the 1st Analytics entry in the Result of Tag Analysis in your Tag Assistant, you'll see the double pageviews that are messing your data.

Easiest way to track it down is to use the history feature of GTM to look at what changes were deployed in GTM on the date the issue started.

Places to look - duplicate triggers, or a tracking event that is misconfigured to create a new pageview.

Hope that helps you find it!

Paul

hlfykgN

I'd absolutely agree. If you can get it fixed properly in a couple of months, then leaving the status quo would e better than making interim changes that just trade one set of redirects for another.

P.

Quick thing to check - have a look at your server logs. Those kinds of random load time increases look typical of the load spikes that are typical of bot overloads - where spambots may be hitting your pages in bursts, causing the server to overload and slow (or even reset, as Vijay mentions).

Those bot hits aren't recorded in GA - you'll have to look at the actual server access logs to find them.

Hope that helps?

Paul

Honestly? I'd spend the time to get the Custom CMS fixed to allow trailing slashes in the navigation links. That would eliminate the redirect issue, Instead of just trading it off to another set of links that would have to redirect.

It sounds like a code sanitising issue in the CMS. Worth spending a couple of hundred dollars to fix the root cause of the issue instead of spending that money to apply bandaids that cause other problems elsewhere. (And bonus, maybe you can get proper canonicalisation built at the same time.)

Of course, yea, this does depend on having/finding a competent developer and having a test environment that doesn't endanger the live site.

Any chance you could push for this option?

Paul

Yup - as Ruth says - this is exactly what UTM parameters are for.

It's critical that you taq ALL links in the email and not just the first ones (as can happen when in a hurry). Any untagged links will result in a significant amount of the traffic from that link being recorded as Direct traffic on GA and will pollute your results. (And some will record as referral, still messing up your goal tracking.)

Those comment links are almost certainly already no-followed, Nikki. So not much point in worrying about disavow at this point.

Also - the fact that AHREF seems to be rediscovering them just means their own crawler data may have changed. Doesn't likely reflect any change in Google's awareness and understanding of those links.

Hope that helps?

Paul

My answer above applies specifically to your situation too, @Elchanan. You are not being penalised by Google for duplicate content in your example.

As I mentioned, the root of a domain is a special case. The version with the slash at the end is considered the same URL as the one without the slash by both browsers and search engines. So much so that it is impossible to redirect from one to the other because that creates a redirect loop - the page is redirecting to itself.

If that SEO tool has warned you of that issue, it's because the tool isn't properly programmed to handle this unique situation correctly.

Hope that helps.

Paul

You can do this in Google Search Console too, without needing any other tools. It will give you the average ranking position, which has some advantages and disadvantages over the other tools' reports.

Screaming Frog also has a basic, useful site visualisation capability built into it.

That nofollow link only applies to that one specific link, dk. The site is using a special service to help create a printer-friendly version of the page for those that might want to print out the content. Quite correctly, they've nofollowed that one particular link.

The website's own page URL only appears in the external link to the printfriendly site because it is a parameter that needs to be passed so the print site will know what page to provide the printout of.

So no, that nofollow definitely does NOT apply to the whole article.

Make sense?

Paul

Users have been begging for this functionality for many, many years and It's been recognised and promised as a "highest priority" many times, but nothing's ever come if it, unfortunately,

I'd recommend assuming it will never be provided.

(And yea, this has been a sore point for a lot of users).

Paul

Google's really clear that they expect canonicalised pages to be "essentially identical" in content - for example, the same list of products, but sorted differently by price, size etc.

In the situation you're describing, the content is far too different to be considered an equivalent page and so the canonical tags will almost certainly be ignored (which is what search engines do with canonical tags that aren't used correctly) Canonicalisation is very literal about the actual content on the page - it doesn't consider similar contexts, only actual words and content. As such, it probably wouldn't' be worth the time to add the tags to pages where it was going to be ignored anyway. But no, they've never indicated there's any penalty beyond simply ignoring inappropriate usage.

Hope that helps?

Paul

The solution to what you want to accomplish requires setting up what is called a reverse proxy, Sharon. It is not an automatic thing to set up and requires a high level of customisation on your server that is not generally available on inexpensive shared hosting.

Paul

There's no automated way to use Fetch and Render for a word count, it's just a visual representation you'd have to estimate for yourself. (Not really sure there's much use to that kind of counting in most cases anyway.)

Screaming Frog does a decent job of crawling and imitating the crawler if set to imitate Googlebot with JavaScript rendering.

Paul

Meta-keywords in the head of pages have no SEO value, and in some instances may even be used as spam signals. It's been a very long time since they ever had any value. So Drupal did the right thing by removing them by default.

I understand there may be a few smaller European search engines that might still use meta-keywords, and there are some internal site search engines that use them for internal indexing purposes. But if neither of those cases applies, there's no point in wasting any time implementing them.

Hope that helps?

Paul

The majority of the configuration needs to happen on the server hosting the main website, Alex. Because essentailly, the visitor isn't even going to be aware of the existence of/URL for the other infrastructure. They're going to go to example.com/blog, and example.com's server configuration is going to deliver them the content from someotherserver.com which hosts the WordPress install.

In addition to the proxy configurations, you may need to deal with cookies, SSL configurations and potentially other server header information that needs to be maintained between the requests passing back and forth between the different servers as well.

This is a pretty common requirement in enterprise configurations though - to keep the software running the blog from potentially interfering with or compromising the security of the main site infrastructure. So like I said - eminently do-able, but not trivial to implement.

Does that answer your question?

Paul

This is an incorrect implementation in the BeamUsUp tool. The hostname (the basic root URL) is a special case. Both the version with the ending slash and without the ending slash are considered by browsers and search engines to be exactly the same.

In fact, you cannot redirect one to the other. Because the browser is programmed to consider them the same, you'll create an infinite loop. So not only is there nothing you should do, there's nothing you can do.

This is the only case where this is true though! For all other internal URLs. the version with the slash is considered to be a completely different URL than the one without the slash. So unless you redirect one version to the other for internal pages, you'll have duplicate content issues.

Hope that helps.

Paul

Yup, using a reverse proxy is the way to do it. Not trivial to set up - you have to have a high level of server administration permissions to do it.

Paul

This is built-in functionality in Wordpress, matching closest existing URL.

Paul

So much this ^^

This is a question for which the only answer is "test your own visitors".

Something else to keep in mind - so much of getting visitors to complete longer forms is about setting expectations and demonstrating WIIFM (what's in it for me) up front.

"A minute and half of your time for these questions helps us give you more useful customised recommendations. We don't want to waste your time." Can go an awful long way to getting buy-in from visitors to complete the form. You've told them why giving the info will help them, and told them how much time is required. So even if they see a multi-step form, they won't get worried it's going to drag on.

Something else to think about.

Paul

Yea, you need to believe those tools, Nikki. In particular, if you click on the Page Loading Issues View Details in the mobile-friendly test, you'll see that there are errors with pretty well all the scripts on the page. And GSC's Fetch and Render is literally what the search crawler can see and process, so if it's not happy...

When I first visited the page, all I saw was the header and the footer as well.

Here's what I think is happening. Pretty well all the content on the page is being loaded with animations, which are JavaScript-based. You have RocketLoader enabled in CloudFlare which is known to cause havoc with JavaScript. (It takes over how JavaScripts get loaded on the page, but often doesn't handle it well). So I suspect that RocketLoader is interfering with the scripts that the page is depending on to display the animated content blocks.

Try turning off RocketLoader from within your Cloudflare settings, clear ALL caches for the site (CloudFlare's cache and any site/server-level caches), and check again.

Good luck! And let me know if that resolves it.

Paul

P.S. I'd recommend seriously reconsidering all that content animation on the page. All that "dancing macaroni" really interferes with the efficient reading of the page. Animation is supposed to be used to draw extra attention to one specific important thing on a page. When everything is wiggling and moving, nothing gets emphasised, and it's all just a major distraction and makes scanning the content nearly impossible.

As far as Google Search Console is concerned - make certain you've verified a new Property in GSC for specifically the HTTPS version of your domain. Google (and other search engines) consider the HTTPS version a completely separate site, and so accurate data will only exist in the new HTTPS property.

If multiple monitoring and crawling tools are at least occasionally reporting improper redirects or are blocked, that's a huge potential issue and must be thoroughly researched. First place to start is to make certain that the SSL certificate encompasses both the www and non-www versions of the domain. If you provide the URL, we may be able to provide more info.

Hope that helps?

Paul

P.S. If the developer's version of the site has carelessly been allowed to get indexed, that's going to have to be dealt with too.

Nope. The use of page speed as a (minor, and very specific) ranking factor doesn't work that way. Never has.

Page speed only applies to a small percentage of search queries and it's used as a disqualifier for really slow pages (think over 15 or 20 seconds). Meaning if your page is really slow compared to your competitors, their page will likely be preferred over yours. But small incremental improvements in page speed won't result in ranking boosts. (And Google's on record as saying that even slower pages will earn SERP results if they're content is significantly better for the user's query.)

Improve page speed for the usability of your site, not because a small incremental speedup will change rankings.

Hope that helps?

Paul

One more sidenote - you can also use inbound link auditing tools like Moz Open Site Explorer, AHREFS, or especially Cognitive SEO to collect as many of your incoming links as possible, then filter them for which ones 404 and then you'll know which external pages contain the broken incoming links. CognitiveSEO is especially well set up to do this, but it's not a cheap tool (Does have a free trial though).

Hope that helps?

Paul

While essential to maintaining the quality of the website, remember this process will only show missing URLs as detected from within the website. It can't show any info about all the broken links that may exist externally on other websites.

P.

Agree with Stephan - the dupe H1s, while not ideal, are not going to be a significant SEO issue.

But what will be the issue is the extra page bloat and therefore load speed of all those pages. Not necessarily for SEO, but most certainly for usability. There are much more efficient ways to get your pages to scale for different devices than to run three sets of headers.

Paul

These tools are designed to report very different things, Eric. SF and Moz are only able to tell you what they can discover about what's happening within your site, since that's all they have access to. (Moz's link index isn't tied into their site crawl reports). So your own pages that are linking to other pages which no longer exist.

Search Console is also showing what's happening on the rest of the web outside your website. So it's picking up and reporting on links on other websites that it's trying to follow but finding 404 at the end on your site.

So, for example, Search Console is the only one that's going to be able to find and report on URLs that you used to have, but that haven't been correctly redirected after the URLs changed, like in a redesign. Assuming you cleaned up your internal linking after the redesign, the only way to know that these other URLs still haven't been properly redirected is via GSC's external visibility.

If GSC is finding a lot of these, and you want to proactively find as many more as possible without waiting for GSC to find them. there are a couple of methods available to try to address in bulk. One way is to run a Screaming Frog crawl of the previous version(s) of the site in the Wayback Machine, then rerun those as a list to discover which ones now 404. Or if you have an old sitemap, you can upload that into SF and run that and check for 404s.

A partial alternative is to run a custom report in GA that shows the referrer to the 404 page. But it will only track actual clickthroughs, whereas GSC actually reports on external broken links, whether they've been clicked on recently or not. So much broader and a bit more proactive. Eoghan at Rebelytics has a great post on assembling these useful reports, including links to directly import from the Solutions Gallery. Do note though, this data can often be pretty "dirty" with direct/(not set) referrers, or very vague ones like "Facebook"

But bottom line - use your Screaming Frog crawl to find and fix the broken links within your own site. USe the GSC list (downloaded) to write appropriate redirects for the URLs it finds that haven't' been covered by your internal fixes. Since you can't do anything about the broken links on third-party sites, the only control you have of them is writing redirects inside your own system. Put another way - it's not worth the effort to find where on the external webs the broken inbound links are from, since you can't do anything about changing them anyway.

Hope that all makes sense?

Paul

P.S. Marking the 404s as fixed in GSC will suppress them for about 90 days, but if the broken links still exist out on the web or in the index and are re-crawled, they will again start to show up in your GSC list until they've been crawled often enough that G decides to trust the 404 and may stop crawling them. Once you apply redirects, G will stop reporting them as 404s. Do note though that if the equivalent content really is gone, there's nothing wrong with letting the broken links 404. Trying to artificially redirect them to unrelated pages (like the home page) will just result in G treating them as (soft)404s anyway.

Since this is essentially a duplicate of your earlier question, I've answered it there.

https://moz.com/community/q/text-hidden-in-tabs-on-desktop#reply_381047

Google's been quite clear that once a site is in the mobile-first index, the content that only becomes visible on user interaction will no longer be discounted the way it is now. But remember, the mobile-first index is slowly being rolled out on a site-by-site basis. So you'll need to do some specific investigation of your own site to know whether it has in fact been moved to the new indexing method.

If your site hasn't been moved to mobile-first index (and the vast majority haven't) then your mobile rankings are being based on your desktop site - which still gets hidden content devalued.

Hope that helps?

Paul

Yes, Google's been quite clear that once a site is in the mobile-first index, the content that only becomes visible on user interaction will no longer be discounted the way ti is now. But remember, the mobile-first index is slowly being rolled out on a site-by-site basis. So you'll need to do some specific investigation of your own site to know whether it has in fact been moved to the new indexing method.

Hope that helps?

Paul

You have a defective link coded for the FAQ link in your site's footer, Andrew. It's currently coded as

[Which then gets parsed as a relative link. This means the link gets parsed as www.example.com/currentpageurl/URL

Because that link is in the sitewide footer, it means every page is generating that link as a link to itself with the /URL added to the end. Which you're seeing detected as all those 404s.

This is very important to fix not only because the FAQ link is broken in the footer, but you've doubled the number of pages of the site for the search crawlers, meaning they're wasting time following thousands of useless links instead of focusing on real pages.

So fortunately it's a quick fix - find that link in your footer template and correct it.

Make sense?

Paul](URL)

Search engines don't "punish" for duplicate content. But what does happen is you make it very hard and confusing for them to decide which version of the duplicated content is more important and more valuable. As  a result, often neither does very well in rankings as they end up competing against and diluting each other..

What is the thinking around why having exactly the same content on two different sites for two different towns is useful and beneficial for a search visitor?

Paul

There's no reason to keep the old hosting just for the redirects. And that introduces a real weak point as at some future date, someone who doesn't realise it's still running only an htaccess file will delete what they think is unnecessary hosting.

Instead, use DNS of the old domain to point it to the new domain and then write the redirects in the new domain's htaccess.

Paul

P.S. And yes, you'll need to write redirects for each old URL that changes. If there are patterns of URLs, it's worth using regex to cut down the number of redirects written.

This is the kind of question that cause a lot of wasted tail-chasing if we can't inspect the affected URL, Nadav. Can you give us a link to the site in question?

Paul

Canonical and noindex are contradictory, Yael. It's either.or, never both. And in the case you describe, I doubt you could no-index the versions with parameters without doing it to the main URL as well (since technically they are all the same page code).

What you are describing is the classic use case for canonical tags - the exact same page referred to by multiple different URLs.

Hope that makes sense?

Paul

First thing to confirm - did you recently migrate to HTTPS?

If the crawler is returning 404 errors, it's because it can't find those pages when following links. Since those pages don't exist, you won't find them in your WP dashboard either, Myflgreen.

The next step is to determine if they should exist (i.e they've been deleted accidentally), in which case you need to rebuild them.

If in fact they don't exist on purpose, you then need to look at the pages that contain the links to the non-existent pages and fix those pages so they don't contain the broken link in their content. (You can find the linking pages by downloading the 404 error report.)

You'll also want to consider adding a redirect from the non-existent pages to another page on the site that provides essentially the same info, if such a page exists. (This is in case other websites might have been linking to those pages at some point)

Final note - sometimes, these links show up as 404s because the link itself is written slightly incorrectly - double check for that too, if the link seems to be pointing to a real page but is 404ing.

Hope that helps?

Paul