Welcome to the Q&A Forum

If your current pages can be accessed by http and by https, and if you don't have canonicals or redirects pointing everything to one version or the other, then one very significant "con" for that approach is that you are splitting your link equity. So, if the http page has 50 inbound links, and the https has another 50, you would do better to have one page with 100 inbound links.

Another difference is how browsers show/warn about non-secure pages. As well as any ranking factor they may associate with secure. Again, in favor of redirecting http to https. The visual handling can also impact conversion rates and bounce rates, which can in turn impact ranking.

As far as cons to redirecting, one would be that you might expect a temporary disruption to rankings. There will likely be a bit of a dip, short term. Another is that you will need to remove and then be careful about accidentally adding any non-secure resources (like images) on the https pages, which will then issue a warning to visitors as well as possibly impacting ranks. There is some consensus that redirects (and canonical links) do leak a very small amount of link equity for each hop they take. So, that's another "con". But my recent experiences doing this with two sites have been that after the temporary "dip" of a couple of months, if done properly, the "pros" outweigh the "cons".

In preparation for Core Web Vitals becoming a ranking factor in May 2021, we are making efforts to reduce our Cumulative Layout Shift (CLS) on pages where the shift is being caused by images loading. The general recommendation is to specify both height and width attributes in the html, in addition to the CSS formatting which is applied when the images load. However, this is problematic in situations where responsive images are being used with different aspect ratios for mobile vs desktop. And where a CMS is being used to manage the pages with images, where width and height may change each time new images are used, as well as aspect ratios for the mobile and desktop versions of those.

So, I'm posting this inquiry here to see what kinds of approaches others are taking to reduce CLS in these situations (where responsive images are used, with differing aspect ratios for desktop and mobile, and where a CMS allows the business users to utilize any dimension of images they desire).

You are missing the namespace for xhtml. There are multiple ways to format a sitemap, but you are using xhtml format for your hreflang tags. You can do it differently without using xhtml. But if you do it the way you are doing it, you need to declare the namespace up in the URLSET tag.

So, where you have:

<urlset <span="" class="crayon-h">xmlns</urlset>="http://www.sitemaps.org/schemas/sitemap/0.9">

It would instead need to be like

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">

Primarily, the requirement is to be in the section. However, I have seen cases where a long-load-time resource in the above meta tags can cause meta tags to be ignored. These were fairly extreme cases, where the resource took multiple seconds to load, synchronously. But moving the meta tags (and Title) above those resources fixed the issue. Also, in another case, we had a snippet from a CDN provider which included an iframe, and in that iframe there was a section and a closing . It turned out that Google was ignoring all of our tags after the which was injected in that iframe, even though it was only supposedly closing the section opened within the iframe. Once we moved that iframe to the end of our own section, below all our meta data, the issues resolved. So, with all that, I do recommend putting meta data at the top of the section, but depending on what else is in there, it might not be an issue for you.

If all of the pages you are interested in are linked internally from somewhere in your site which can be reached through navigation or page links, you can run a simulated crawl with a tool like ScreamingFrog, whcih will discover all the "discoverable" pages.

The site you referenced is built with a platform called "Good Gallery", whcih generates a sitemap. This is at  www.laskeimages.com/sitemap.xml. I'm not sure what criteria it might use to include/exclude pages, but that would likely be a good list. You will need to view the page source of that page to see the data in a structured way to extract it.

Another method is to use Google Analytics. Assuming that each page of your site has been viewed at least once in its history, you could extract the list from Google Analytics. Especially from an unfiltered view which includes visits by bots.

Yes, your understanding of x-default is correct. The purpose of including it everywhere you have alternate HREFLANG links, is to handle any locales you don't explicitly include (to tell the search engine which is the default version of the page for other non-specified locales). And it should be included on each version of the page, along with the other specified alternate links for each locale. Alternatively, you could collect all of these centrally into the sitemap file, rather than inserting into each page. Both types of implementation are valid (but anecdotally I've had better luck with on-page tags instead of sitemap implementation).

This is very interesting, and I see from the threads you linked that multiple businesses are having the same problem and the same difficulty navigating both the Google or Facebook support communities. Out of curiosity, are you able to inspect one of your Facebook pages whcih still has the video, and see if any schema for the type "VideoObject" is included in the page, and if so, paste the markup here (redacted as necessary)? I don't think I'll probably be able to help much on this, but perhaps something in the schema data might give some clues to the community here to work with.

We currently use geoip detection to present a dialog when customers access one of our country-specific sites other than the one matching their location, asking them to select whether to continue to that requested site, or else click a link to the site matching their IP location's country.The purpose is to diminish the likelihood of users having a bad experience if they accidentally shop a region's site only to discover when they get to checkout that they cannot complete the purchase and need to switch sites.

We are considering adding some logic to prevent this dialog from appearing if the user agent is a known search engine bot. The dialog serves no purpose to bots, and we are worried about its impact on crawling of our sites from servers outside the country-site's location.

That said, we don't want to incur any negative impact of perceived cloaking.

Is user agent-specific logic acceptable practice in this scenario?

Although the impact of keyword match in domain names isn't as high as it once was, my current experience is that it still is a very significant ranking factor. I've recently (last year, and also about 4 years ago) completed two domain name changes, and the impact on searches where the query term is/was matched in the domain name definitely has an impact. That said, after an initial "honeymoon" period, you're likely going to see some negative ranking impact of a domain name change, regardless of the specific domain names. My recent experience has been that things get crazy for a week or so, then look really good for 1-3 months, then the negative impact hits, and then it takes quite a while (sometimes more than a year) to get everything back to where it was. So, if you do change domain names, it needs to be seen as a long-term strategy, not a "this year" one.

The alternate links with hreflang tags should be present on all the alternate pages. In the example you gave above, there is a self-referencing tag on the English page, and there are a bunch of alternate tags on the French page. But the English page is missing the alternate links to point to the French (and other) alternate pages for those various languages.

To simplify, if you only had English and French pages, then the English page should include a self-referencing tag for English, and an alternate tag for French. And the French page would include a self-referencing tag for French, and an alternate link for English. You might also include an "x-default" on the English page, if it's intended as the primary version for any unspecified other users.

I currently manage a site which is localized per region, as opposed to country. For some regions, like US and Australia, it is 1:1 with country, so we do not have issues there. But for Europe, that is where we do have some issues currently. We took the following approach (below), but I have to first say that it is quite problematic and has not performed very well so far (implemented about 1 year ago).

The approach we took was to implement HREFLANG within our sitemap, and for Europe, we generate specific alternate locations for each of the countries where we do business in that region, all with the same URL. Here (below) is a redacted version of one page's LOC node in our sitemap (I've only included a partial list, and only showing English, as the full list of alternate URLs for this one LOC has 150 alternate links to cover every EU country x 5 languages we support). But, the general approach is that for Europe, we create one alternate link for each EU country, in each of our supported languages (we support 5 languages). So, we don't assume, for example, that German speakers are only in Germany, or that English speakers are only in the UK. We cover every country/language combination and point many of these to the exact same alternate link.

Again, as I mentioned, this hasn't achieved all we had hoped. But sharing the approach for a reference point here, as an option, and open to any other ideas from the community. We also struggle with EU in terms of Google Search Console geographic targeting. Unfortunately, Google does not allow a property to be targeted to "Europe". And they only allow one single country per property. In our case, we really need to target a single domain to "Europe", not to a specific country. But we can't, and that is a problem currently.

Here is the example from our Sitemap (partial cut-and-past of the first few entries from one URL node):

<loc>https://www.example.com/example-page-path</loc>

<priority>1</priority>

... remainder of alternate links removed to shorten list here

|   |

I'm not sure how the search engines look at 302 redirects which are in place for a prolonged time. I'll be interested to see if anyone else on this thread has additional insights about that. What I can say is that I've used 302 redirects in some cases for prolonged period of time (although not as long as 12-18 months, perhaps more like 4-6 months) and have not experienced issues from that approach. But others on this forum may have more experience with 302 redirects over that period of time.

The other thing I'll mention is that some tools like Moz Pro will report 302 Redirects as "issues". My perspective is to look through these because some might be unintentional, and then to ignore when they are inentional/strategic.

I have not personally seen the use of hreflang inside of the canonical link annotation, as you've done it on the English pages outside of your homepage. I see this on one of those pages:

rel="canonical" href="https://www.eginnovations.com/cloud-monitoring" hreflang="en"/>

That might be the culprit. I don't know for sure that this is wrong, or hurting anything, But it's not something I have ever used (embedding the hreflang inside the canonical tag). You might try removing that fragment. HREFLANG is only on a page-to-page (not site-to-site) level, so I would only include it on pages which have an alternate page available.

Even though Google does have an international targeting function in GSC, I've had more success with hreflang tags on pages. Technically, hreflang tags can be implemented on each page or in the sitemap, but having experimented with both, I've seen better results from the on-page implementation.

So, I would recommend that you try to implement hreflang tags, in which case you can specify both the country and the language (Ireland would be "en-ie").

Make sure to check the following, when you do:

• include a self-referencing hreflang tag on each page (in other words, a tag for the page's intended locale)
• include an "x-default" tag on each page
• verify that each page referenced by another page's hreflang tags has a reciprocal hreflang tag referencing back in the opposite direction.
• verify that the hreflang tag URLs are also the canonical URL for those pages, and that no redirect occurs when accessing.

In Search Console (new version) go to Coverage > All Known Pages. Then click "Valid" above the chart, Then click on "Submitted and Indexed" and download to a spreadsheet. Do the same for "Indexed, not Submitted in Sitemap". Combine these two downloaded lists into one list in a tab in excel, and in another tab you can put the URLs of all the pages you think should be getting indexed (unless you think all of them are in your sitemap anyway). And then use vlookup formulas to find whether each of the pages you know are on your site are actually indexed (by looking up in the tab containing the downloaded lists).

Right, that's what I'm suggesting to change. Although I am not 100% sure. But I would try removing the hreflang which is inside the canonical, like you have copy and pasted above. Just keep the canonical normal. There's also a slight inconsistency between your use of EN and EN-US, but if you remove the hreflang from the canonical tags, that will take care of that too. Again, I'm not 100% sure, but that's what I'd try next anyway.

Yes, that looks correct now. And in your specific case, x-default might indeed handle the rest since Europe is your default, and that's where the unspecified combinations are most likely to be for you.

I wouldn't be too concerned about site speed. These are just links. They don't load any resources or execute any scripts. For most intents, it's similar to text. The main difference is that they will be links that may be followed by the bots. But really, even though you'll have many lines, you only really have two actual links among them. So, I wouldn't be too concerned about this part.

Good luck.

Alan,

Spot checking, I looked at a few of your redirected old domain pages which are still showing up in the index. Here are my thoughts:

• For the ones which are redirected properly, you might be experiencing a ranking dip because the old domain name contained keywords better matching search terms. for example, if a search contains "office" or "space", it could be that your old domain was perceived by Google as a stronger match. Although Google doesn't place as much weight on exact or partial match domain names as was once the case, it is still a ranking factor, from my recent experience.
• URLs such as http://www.nyc-officespace-leader.com/neighborhoods seem to be properly redirected. From what I can tell, the redirects look fine.
• Other URLs such as http://www.nyc-officespace-leader.com/Chelsea are being redirected to a non-existent page, with a 404 error. Probably because you are using a pattern redirect which is preserving the requested file path. For any of these which previously had rank equity, I would put some effort into redirecting to some relevant existing page.
• I do see that your old URLs here http, and the new ones were https. The redirects are working fine, but you might want to check that when you did the change of address, it was on the http version of the old property (to the new domain). Even though we're not supposed to use change of address to go from http to https migrations, it is still necessary to change the address from the specific version of the old property. So, if you previously had both http and https versions of your old property, you want to make sure you've done the change of address tool for the http one.

If I understand your question properly, the majority of the content on each page is unique, and the page titles and descriptions are unique, but just one section of each page contains some common content which is the same sitewide. So, if this is a correct interpretation of what you wrote, then I don't think the search engines are going to consider the pages as duplicate. This scenario is no different than a common footer or a header navigation section, which are generally the same across all pages of a site.

Also, my perspective is that duplicate content is not a penalty as some people may think it is. Rather, it is a chance that the search engine will choose to not index some of your pages if it thinks that other pages have the same content and that those pages should be indexed instead. In cases where the pages are really just versions of the same thing, this can actually be a good thing. If you had 10 extremely similar pages, all targeted at the same exact visitor queries and intent, you would likely want only one of those indexed. And then, to make sure your preference of "which one" is followed, you would implement a canonical link on all 10, pointing to the preferred one. But, if you did nothing, the search engine might just pick one, and that might also be okay for you (not as good, but okay).

In any case, the situation you described, where there is a sitewide element visible on each page, I don't think is likely to cause the search engines to consider your pages as duplicates, unless there is very little or no other unique content differing on each of those pages.

If I understand the OP's intent, it is to target countries, not languages. Hreflang can specify alternates for a language, or a language-country combination, but unfortunately not just for a country. So, as the OP has proposed, yes you do need to specify the language and the country. And that does bring up a dilema faced by many of us in terms of what language to use. If your content is in all English, then yes you should use like "en-FR". BUT, you might also want to include an "fr-FR" as well, pointing to the same alternate URL. Because there are going to be a lot more France-based visitors on Google whose browser settings are for French language than English. For sure, both do exist (there are native English speakers in France too), but you don't have to choose one. You can include both. Google may not completely respect your directives since the content is in English (assuming that's the case), but it's what I would recommend. So, for each country (assuming the content is in English), include both an English and a language-specific hreflang tag (pointing to the same destination) for that country.

Since your last example uses "es-ES", I assume maybe that you're planning to also publish some content in Spanish language. But if not, again, realize you can include multiple hreflang tags for a single country, and pointing to the same page.

I also don't know where you are based. But if the business is US-based, I wouldn't duplicate US also as a localization. Rather, I would make that the default. Or, if you are based somewhere else, same thing, but with that country.

On question 2, you can set up a GSC property for folder paths (www.example.com/fr/), and target those. I would not target the root level (www.example.com) in your case, because that would also apply to all the subfolders. That's one of the advantages of using subdomains instead of subfolders, is that you can target each independently. But with subfolders, you can target all except the root (because it would cascade downward).

On question 3, you should do the same as you do in number 1, as long as you are duplicating those pages in each subfolder. Otherwise, if you don't give a directive of which page to index, since they are duplicates, Google is going to choose for you. And might not choose the one you prefer.

This is not a definitive response, but my approach has been to canonically relate each paginated review page to the product detail page. My rationale is that I get enough keyword relevance in the first 5 or 10 reviews which show up prior to pagination, to contribute effectively to the ranking of the product page.

Some product review solutions add semantic markup for the reviews, and even though the reviews themselves are paginated, the whole set of reviews is included in the JSON table of reviews on the product page. So, in this case, if yours does this, it might already be getting all the content when it crawls the product page.

I have not been a proponent of separately indexing the review pages themselves. I know there are proponents, and they might have solid arguments why to do that. I have preferred for my product page to get all the page relevance, inbound links, and visits.

The common solution is to overlay the text on the image, rather than producing the image with text in it. The overlay text can then be given an H1 element.

A few thoughts:

• Install the browser extension Ayima, which will let you see if this is actually the result of multiple redirects. There are other ways to see this same info, but the Ayima extension makes it really simple to see the multiple hops, when there are.
• You might try to sequence the existing redirect that is in your htaccess file (A to B) all the way up, or all the way down. There is a sequence followed (most specific to most general).

Screaming Frog does have a custom search, which allows you to search for specific text strings in the page source code. You might be able to leverage that function for finding images. They might not be as cleanly inventoried as you want, but at least it would find them and you could use Excel to parse into a neater list. The function is within Configuration > Custom >Search. And in there you use a "contains" filter. Please note that I haven't tried it out for your specific purpose, but I have used this quite often to search the page source for other strings.

The meta description is not always used for snippets, depending on what keywords/phrases are searched. When the search term is not in the snippet, but is on the page (and "page" does include elements such as breadcrumbs, top navigation menus, etc.). The search engines will often show the verbiage on the "page" surrounding the search terms. But also, your meta description is pretty lengthy. So you might have better luck with a description in the 100-150 character range.

Ah. So, then I might try one of the following:

• My preferred approach would be to set up a redirect for that URL to a valid new URL. That way, you would make the best use of the traffic coming from the Sitelink, for whatever time it might remain there. After a while, I suspect Google will either update the sitelink title and description with those from the new redirected page, or perhaps drop that sitelink eventually in favor of another page.
• If you can't do the above (maybe you are not able to set up redirects from the old URL), then I might go the route of using the Search console (old version) to request removal of the old URL (Google Index > Remove URLs). If it really does give a proper 404 response code, then this should work. It doesn't do the job on its own if the URL still gives a valid response code. But a 404 plus a removal should get rid of it. That said, then you are rolling the dice with whatever Google decides to promote as a replacement sitelink. So, I would prefer the first approach, if I thought I could make the best of the traffic coming from that link.

One of the advantages of migrating a blog into a site's structure is outside of the realm of SEO, but can be helpful in making a case to do this. I have migrated several blogs into sections of existing e-commerce sites, as "resource centers". These were not pure migrations, but rather restructuring of the content into pages whcih co-exist with merchandising on the e-commerce site. Here, the advantage is that once a visitor is attracted to a content page on the site, they are more effectively converted into an e-commerce prospect. This is especially true if the blog articles are restructured to include merchandising elements. But even if not, if the only difference is that the visitor is now on a site whose navigation structure includes all of the e-commerce merchandising, they are more likely to convert on such a site. So, the benefit is more about converting customers than attracting visitors.

One way I've tested this is to pick a relatively valuable single page, and migrate just that to an on-site page, with a one-to-one redirect. You will have to wait past an initial period where both of your pages may co-exist in the SERPs. But once the old blog page drops out and the new page comes in, then you can test the longer-term impact. Again, the number of visitors may actually be smaller, but you might more than compensate for that by converting more of those visitors.

From my experience, there is not an issue with too many redirects in parallel. However, too many redirects in sequence, or a "redirect chain" definitely is problematic. If a page redirects too many times, the search engine might not follow until the very end.

With regard to redirects in general. On occasion, Google has stated that some small amount rank equity is lost with each redirect. So, we do tend to assume that a direct link is preferred over a redirected link. When we restructure our URLs, we generally clean up all the internal linking to point directly at the new links, rather than rely on redirects. With internal links, that's feasible. With external links, while you can reach out and request link updates, you do also need to rely on redirects.

You would want the exact same hreflang tags on both versions of each page. So, that means each has a self-referencing tag, plus an alternate tag pointing to the sister page in another locale, plus an x-default tag.  The hreflang tags basically tell the search engines whcih version of the page is appropriate for which locales, and when they are on a page in one locale, it tells the search engine where it can find the equivalent pages for other locales, as well as which one is the "x-default" for any locales you haven't specified.

This is commonly done intentionally, when launching a site on a new domain. Once the disallow is removed, the general practice is to request reindexing of the root domain page (and possibly some key pages with paths not likely to be found through navigation) in GSC, and also submitting (or re-submitting) your sitemaps directly in GSC (even though they also may/should be in your robots.txt file).

I'm not sure how long you can expect the search engines to take, since your situation is a bit unique where the site was indexed, and then disallowed temporarily. Just guessing based on launching brand new domains, the process should be quick to be indexed (perhaps a few days) but might be slower on regaining previous ranking positions (unsure on timing of this).

While there is some level of uncertainty about the impact of multiple H1 tags, there are several issues about the structure you describe. On the "sub-pages", if you have an H1 tag on the site name, that means the same H1 tag is used on a bunch of pages. This is something you want to avoid. Instead, develop a strategy of which pages you would like to target to rank for which search queries, and then use the page's primary query in the H1 tag.

The other issue I see in your current structure is that it sounds like you have heading tags potentially out of sequence. Accessibility checker tools will flag this as an issue, and indeed it can cause frustration for people with vision disabilities accessing your pages with screen readers. You want to make sure that you preserve a hierarchy where an H1 is above the H2 is above the H3, etc.

Bryan,

If I understand your intent, you want your pages indexed. I see that your site has 5 pages indexed (/, /help, /influencers, /wholesale, /co-brand). And that you have some other pages (e.g. /donations), which are not indexed, but these have "noindex" tags explicitly in their HEAD sections.

Not having a robots.txt file is equal to having a robots.txt file with a directive to allow crawling of all pages. This is per http://www.robotstxt.org/orig.html, where they say "The presence of an empty "/robots.txt" file has no explicit associated semantics, it will be treated as if it was not present, i.e. all robots will consider themselves welcome."

So, if you have no robots.txt file, the search engine will feel free to crawl everything it discovers, and then whether or not it indexes those pages will be guided by presence or absence of NOINDEX tags in your HEAD sections. From a quick browse of your site and its indexed pages, this seems to be working properly.

Note that I'm referencing a distinction between "crawling" and "indexing".  The robots.txt file provides directives for crawling (i.e. access discovered pages, and discovering pages linked to those). Whereas the meta robots tags in the head provide directives for indexing (i.e. including the discovered pages in search index and displaying those as results to searchers). And in this context, absence of a robots.txt file simply allows the search engine to crawl all of your content, discover all linked pages, and then rely on meta robots directives in those pages for any guidance on whether or not to index those pages it finds.

As for a sitemap, while they are helpful for monitoring indexation, and also provide help to search engines to discover all desired pages, in your case it doesn't look especially necessary. Again, I only took a quick look, but it seems you have your key pages all linked from your home page, and you have meta directives in pages you wish to keep out of the index. And you have a very small number of pages. So, it looks like you are meeting your crawl and indexation desires.

The SKU property doesn't need to be an official/registered identifier. It can be anything the merchant chooses. It should ideally be unique in your product catalog, but doesn't need to be unique across merchants. So, you can map to it whatever product ID your catalog has assigned. Can even be a simple database row number if needed.

Canonical reference links are the preferred technique for this.

If you do nothing, very likely the search engines will decide for you which variations of your pages to index, and the selection may not be ideal. If an index page can be filtered many different ways, the unfiltered version should be referenced as the canonical on each, and a self-referencing canonical link should also be specified on the unfiltered version.

You don't really yet want to disallow the crawling of the refinement paths, because without canonicals implemented, you might very well do more harm than good, finding important pages getting de-indexed. If at some point in the future you find that all the URLs from the refinement paths have been disappeared from the index, and your desired pages are all indexed properly, then at that future date you might want to disallow crawling of the refinement paths (in your robots.txt file). But, not yet, IMO.

In answer to your questions:

1. You can place the JSON-LD in either location. If you are making use of data which may not be available at the beginning of the page load, then you might prefer to place at the very bottom of your body tag. But both locations work if you are simply inserting static content.

2. If you are using a CMS, you can simply insert the script as static content (as long as your CMS allows you to insert a script). This is assuming your page is a static page, not one which dynamically changes based on some server-side application. So, for example, if you have an article page on your site, and you want to insert schema to mark up that article, then this would be straightforward to do by simply inserting a script. But if your site is an e-commerce site and you are wanting to mark up all of your product details pages with product information, then this might be better done as part of the application. For static markup, if you search for "json ld generator" you will find some handy tools to simplify creating static scripts. Just be careful that you do not use static markup on a dynamic page where the page content might change based on application data, then causing inconsistency between your markup and the actual page content.

3. If you insert through your CMS, then yes you should see your script in the source code. If you use an ansynchronous script to inject your JSON-LD, then you might not see it in your source code, but could still see it by using dev tools inspect, after the page finishes loading. Either way, then use the Google Structured Data Testing Tool on the live URL to confirm that Google is able to parse the markup properly, with no errors. When you do that, look at its results to make sure the hierarchy of the data looks appropriate to you based on the data you know is on the page.

To my understanding, a redirect and a canonical are treated very similarly from an SEO standpoint. With either of these, only the end URL (either the one to which you are redirecting, or the one linked in the canonical reference) is the one which, if all directives are honored, gets indexed. So, unless I'm missing something, there is no benefit at all of having the category paths in the URLs if you are either redirecting from those to the flat one, or if you are pointing a canonical to the flat one. The benefit would be there if those keywords were in the final URL (redirected or canonical). But if the final URL is flat, then I don't think you get any benefits from the non-canonical URLs having keywords in their paths. So, if the flat URL is the final one, from either method, I would ensure that the "product name" is fully descriptive with the desired keywords.

An interesting dilemma. Often, UGC like forums or reviews is considered as a boost for SEO. But it seems you have concluded that in your case, it is a detriment.

I'm not completely certain of what would be the best strategy in your case, but I suggest you might start with exploring the main reasons why the forum is hurting your rankings (assuming it is). One possibility is that the forum content might contain links to spammy sites. To combat that, you might nofollow those links which are inserted into your user generated content. Perhaps even use the new-ish rel=ugc tag whcih Google introduced (https://webmasters.googleblog.com/2019/09/evolving-nofollow-new-ways-to-identify.html)

Another strategy might be to leave the UGC in place, but to make it not crawlable. For example, reading it into the page using a script from another URL which is disallowed in your robots.txt file. If I were to explore this method, I woudl probably start with some experiements on just a few pages, to measure the impact (out of caution in case the assumption that UGC is the culprit might be incorrect).

Lastly, the forum does not need to be on the same root domain as the website. Again, I'm not sure this would be a positive move or not, but you could register an entirely different domain, and even disallow crawling. Or, it could be another subdomain on your root domain, but again with disallowed crawling. If you want people to be able to find the forum, but are worried about the content being indexed, you could only allow crawling of a few key pages on it whcih are non-UGC.

But mainly, I think your challenge seems to be a very good candidate for running some experiments, rather than relying on opinions from those of us (myself included) who haven't been in your exact scenario before.

For Wordpress, it depends if you have it set to filter out script tags from content. So, if you toggle content editing mode to html, and put in a script tag, whether it allows this or not. That is a setting in Wordpress. If it is filtering out your script tags, and if you don't want to (or can't) disable the script tag filtering, then using a plugin to administer the schema markup would be your best bet. There are generic plugins to insert script tags. And there are more specific plugins for schema markup, like Yoast (and others).

If you are inserting the scripts through GTM, then you may not see them when you simply "view source".  With tag managers, you will see their container in the source code, but not the injected scripts. But if you instead use Chrome Developer Tools (right click and choose "inspect"), on the elements tab you will most likely find the injected scripts.

The benefit of the directory paths approach is the additional keywords, if your product name (or ID) is not in itself descriptive enough. For example, if you have a sofa style named "Diana", you wouldn't want your URL to be domainname.com/diana.html. Something like domainname.com/furniture/sofas/diana.html would be better.

But, you can accomplish that with more descriptive product IDs. So, in the example above, if you could make your product name "furniture-sofas-diana", then your URL would be domainname.com/furniture-sofas-diana.html, which accomplishes the same keyword targeting.

And then that solves the issue of when products are in multiple categories, since it's a flat URL regardless of how the visitor arrived to the page.

But if your products are really almost entirely in a single category each (keeping in mind temporary categories like "sale", "new", etc.), and they will be that way forever, then there is an argument to be made for the paths. Because it does help the search engine to parse up your site, and to provide nice breadcrumbs on your listings.

This is really a perennial debate. And there's no one answer. For most of us, we do have to live with products being in multiple categories, as the norm (especially when considering categories like sale, new, best sellers, etc.). Canonical reference links help this issue, but aren't necessarily ideal.

But, what really struck me in your question was that you said the URL changes when you click on the product. Ideally, you don't want all your internal links to be redirects. That's something I would try to avoid.

I also suggest to double check that the actual title tag (the first one you included in your post) is in the section of the page, and not the . And preferably high up in the section before any scripts or other elements which could take significant time to load. Check this in "view page source" and not just using the "inspect" function of the browser's dev tools, to ensure that some scripts on the page aren't altering the title tag after the document is loaded, since the search engines may instead use whatever is initially set in the actual of the document before scripts execute.

Yes, that looks correct. However, I would suggest adding two more hreflang tags to each page. One for "en-gb" (pointing to your UK desired version) and one for "x-default" (pointing to whichever version you would prefer for any other nonspecified locales. You will want all 4 of these on each of the two pages (so each page would include a self-referencing tag. These 2 additional ones are optional, but I think would provide a bit more clear direction to the search engines about which page to present for which locales.

This is because Moz hasn't updated their crawling tool to consider hreflang in the equation of reporting "duplicates". They've acknowledged that. They might update it in the future. But for now, you just have to ignore pages being reported  as duplicate if you know that they are properly linked by hreflang to distinguish countries or languages.

Self-referencing canonical tags are a best practice, and will give an important correct signal to the search engines, which is more important than cleaning up reported warnings in the Moz crawl.

In your original post, you wrote that you are a "UK-centric business". I would think you would want to be as specific as possible to make sure the search engines know to serve the most relevant pages to the UK audience. So, yes, I would definitely include a self-referencing hreflang tag. It also simplifies your process because you can just copy-and-paste the hreflang tags between the versions of the page, with no editing (you put the same exact set of tags on the British page as you do on the American).

However, I'm really only responding to your "how" question. There is also an implicit question of whether you should be doing this. I have to say that from recent experience, no matter how thorough we are with hreflang tags, the search engines inevitably serve up pages across the desired locales. For my brand, I have Australian links showing up in US searches, and US links showing up in British searches, etc. This is even with correct hreflang implementation. In our case, it is a necessity to have multiple localized sites, because we carry different inventory, at different prices, and with different policies in each region. But if that wasn't the case, I would not localize my site between US, British, and Australian English just for language variances. That is a subjective decision, but I have so many problems coming from the wrong pages being served in the wrong geographies, despite thorough hreflang tagging, that I would be very hesitant to create more localizations than absolutely necessary. This wasn't your question, I realize, and also this is purely subjective, but passing along for consideration.

To my understanding, no. Duplicate content generally occurs when two or more pages have very similar content, and the search engine decides that only one of those should be indexed (after crawling each). The concept of "duplicate content" as a "penalty" I think is much misunderstood. There's not actually a penalty, like one would see for a manual action due to spam or hacked content. But rather, if those multiple pages really needed to be separately indexed, then the "penalty" is that only one will be indexed. And, if you don't make smart use of techniques like canonicalization, then the one the search engine chooses to index may not be the one you want indexed. But anyway, none of that really applies in the case you're put forward. As best I can tell from very quickly looking at your site, the pages are unique, and yes, multiple pages might include the same thumbnail(s). But as long as they also include enough unique content besides the thumbnails (and/or don't include exactly the same set of thumbnails as another page), they are likely to be treated as unique pages by the search engine and each indexed. But, you don't need to guess. Just look to see whcih of your site's pages are being indexed. If all the pages you want indexed are being indexed, and none that you don't want indexed, then you don't have a duplicate content issue to worry about.

For one, the GSC functionality is much easier to use for dealing with URLs having multiple query string parameters. robots.txt processes the statements in order, so you often have to set up a broad disallow, followed by more specific allows, to achieve the same result which can be more easily managed in GSC.

Also, GSC is useful for the "representative URL" setting, if your pages don't necessarily get crawled without the parameter present at all, but you only want one version of the page indexed if the crawler encounters multiple versions. So, this is a little like a dynamic canonical, except you are not specifying which version.

If you are keeping them, rather than redirecting them, I assume that means you have a reason for people to be able to find those pages so that they can get some information abot the discontinued product, or at least understand that it was discontinued. If that's the case, then I don't think you would want to noindex or 404 them. On the other hand, if there is no reason for those pages to still exist, from a visitor standpoint, then usually I would redirect them to a category page (generally the parent category the product belonged to), to preserve any link equity, even if the number of links are low. Especially if you have a lot of discontinued products from a category, even if each product had let's say on average 0.1 links, then if you have 1,000 of those pages you would end up with 10 backlinks to your category page, which could be valuable. Again, this is assuming that you don't want/need to preserve the pages for your users to be able to find the info.

So, regarding NOINDEX vs. DISALLOW, there is a significant difference there.

If you disallow in robots, you are asking the search engine to not even crawl that page. Whereas if you NOINDEX in the page head, then the search engine may still crawl the page but should not index it.

There are a few impacts of this difference. For one, if you use NOINDEX but still allow the search engine to FOLLOW, then it may discover pages which otherwise might not have been discovered (if that page has unique links, for example). So in this case, you might prefer to use (NOINDEX, FOLLOW) if you want that discovery to happen. On the other hand, if you have many pages and you are trying to wisely use the search engine's crawl "budget", then you might in some cases prefer to disallow some paths in the robots.txt file.

It's also common to use robots.txt to disallow some files where you do not have control over the response. Non-html files, where you might not be able to easily administer noindex directives. Or dynamic pages your web application may serve but not allow you to administer head tags for.

All of that said, robots.txt files have been shrinking ever since the search engines began to render javascript, since now they need access to a lot of resource files which they previously did not. Much of the old advice of disallowing scripts and admin folder paths may be obsolete now, if those files are needed to properly render pages.

Yes, I meant 301, server-side redirects.

Regarding performance, I currently have a little over 50,000 entries in my redirects file with no discernable performance impact. But, different platforms handle differently, and also we have a CDN which caches redirects too, so that could make a difference. I guess the safest approach would be to insert a hundred thousand or more dummy redirect entries into your redirect file, temporarily, and stress test it.

Yes. As I understand it, hreflang is a page-specific directive. Each page can have different hreflang tags, and this is a very common approach. You could also just tag those single language pages with a self-referencing hreflang tag (EN) and x-default. Those two tags could be on all your US pages, and then add additional tags to the pages whcih have siblings in other languages/countries.