Welcome to the Q&A Forum

Hi Gemma, interesting question! I'd consider a few things;

1. While the product pages rank well for long-tail keywords, are they driving much organic traffic or, more importantly, organic revenue from people landing on the page?
2. If it's possible to reuse the pages for new products - what are the downsides?
3. What would the best user experience be for out-of-stock products? How similar are the new ones to the old ones?

In terms of question 1, if these product pages are numerous enough to be a source of concern, I'd want to know if you're getting any benefit out of them being indexed. If not then removing them from the index could be a simple solution and would help avoid things like searchers landing on an out-of-stock product. E-commerce clients of mine have often found that organic conversion rate for sessions landing directly on product pages tends to be worse because it's relying on the visitor wanting pretty much exactly that product to be interested whereas category pages can show off more of the range.

In terms of 2, if it's an option to reuse the existing product pages, why have you shied away from doing that until now? If you have new products which are similar enough to the old products that means users coming to the page are more likely to get what they want (rather than just being redirected to the category page, or hitting an out-of-stock or 404 page). Also, if each product is keyword researched and the products are similar enough, presumably new products will be competing with old ones for similar long tail keywords?

If neither 1 or 2 work, I'd focus on what I'd want as a user. It can be frustrating to land on a 404 page, either through search or on the website, but it can also be frustrating and confusing to be redirected straight to a category page or similar product. Maybe the user would want to see the out-of-stock page with the option of being taken to similar products? Again for me it'd come down to how much you think each of these unique products could fulfil similar criteria for the visitor.

Hope that helps, as you may have picked up from my response I don't think there is one universal right answer but there is likely a best for your site. Happy to discuss further

Hi there, so just to confirm - you have a page that is redirected to your homepage (I'm assuming not with a JavaScript redirect) and your Google Analytics is reporting that page is getting traffic from 12-reasons-for-seo.com?

This sounds like Measurement Protocol spam, I'll explain why I think that based on the way GA works but to reassure you - if that's the case then all it's messing with is your analytics. Google doesn't use Google Analytics data to inform search rankings - in terms of "bounces" Google just uses data about whether a searcher, on a Google page, clicks one result and then comes back to the search results page and clicks on another.

• Google Analytics records page views based on receiving a message from some JavaScript code that runs on (ideally) every page of your website
• If the page is redirected before it loads (as in not a JavaScript redirect) that code won't have time to run, so Google Analytics won't see you having a page view on that page, even if someone tried to access it
• Measurement Protocol is a way you can manually send hits to Google, it's a way of recording all kind of things that are difficult to reflect in terms of pageviews and events on your site (i.e. someone bought a product over the phone or someone viewed the same page on your app)
• Anyone can get your GA ID by looking at your page code and once they have that, they can send fake pageviews to your Google Analytics.

If it's true that all of these pageviews are landing on a page that doesn't even load before it's redirected (that's important, because if it's a JavaScript redirect your GA code might have time to run) then the best solution for this is probably to find what all of these hits have in common (sounds like you've already found a couple things) then create a Google Analytics filter for your reporting view which excludes this traffic as specifically as possible (to reduce risk of accidentally dropping real traffic)

When you say 1300 landing pages are coming online every night that doesn't mean 1300 new pages are being created does it? Based on the rest of your comment I'm taking it to mean that 1300 pages, which were already live and accessible to Google, are being updated and the content is changing if appropriate.

In terms of the specific situation I describe above, that should be fine - there shouldn't be a problem with having a system for keeping your site up-to-date. However, each of the below things, if true, would be a problem;

• You are adding 1300 new pages to your site every night

• This would be a huge increase for most sites, particularly if it was happening every night, but as I say above I don't think this is the case

• You are actually scraping key information to include on your site

• You mention an API so it may be that users are submitting this content to your site for you to use but if you are scraping the descriptions from some sites, and reviews from others that is what would be viewed as spammy and it seems like the biggest point of risk I've seen in this thread.

Hi, I think you've already got a couple of good answers here but just to throw in my thoughts; to me this would all come down to how much value you're getting for the volume of content you're creating.

It sounds to me like you have 1.3k product landing pages, and you're producing 80 articles a month, plus maybe you're indexing the review pages too?

I think frequency here becomes secondary to how much each of these things are adding. If you are indexing the reviews pages for specific products, those pages could just be diluting your site equity. Unless they are performing a valuable function I'd consider canonicalising them to the product pages. As the others have said, having product pages that regularly update with new reviews shouldn't be a problem but with all the content you're adding to the site you could be relying on Google indexing these changes far more quickly than it actually is.

If you're adding a large number of articles every month - are those articles cannibalising other pages, or each other? The way I'd try to gauge if it's too much is whether the pages are getting traffic, whether you're having a lot of flip-flopping in the keywords you're targeting, and whether you're starting to get issues with Google indexing all of your pages. Similar to the review pages, if the articles are providing value to your readers, getting you links or getting you a decent amount of traffic then grand, if they aren't generating much I'd consider producing less or removing/redirecting poorly performing articles after a little while to preserve site equity and help focus Google's crawl.

On the note of posting frequency, I would agree with Gaston that it's about what's right for your readers. If a lot of article-worthy content comes out at the same time, I'd post about it then and there, if this is just content you're coming up with and adding and timing doesn't matter, spreading it throughout the month makes sense in terms of staying fresh, getting the articles indexed, and honestly not having to rush deadlines/delay release.

Yea that's definitely tricky. I'm assuming you haven't taken out any load balancing that was previously in place between desktop and m. meaning your server is struggling a lot more? The Page Speed Insights tool can be good info but if possible I'd have a look at that user experience index to get an idea of how other users are experiencing the site.

A next port of call could be your server logs? Do you have any other subdomains which are performing differently in search console?

In terms of getting Google to crawl more, unfortunately at this point my instinct would be to keep trying to optimise the site to make it as crawl-friendly as possible and wait for Google to start crawling more. It does look like the original spike in time spent downloading has subsided a bit but it's still higher than it was. Without doing the maths, given that pages crawled and kilobytes downloaded have dropped, the level of slowdown may have persisted and the drop in that graph could have been caused by Google easing back. I'd keep working on making the site as efficient and consistent as possible and try to get that line tracking lower as an immediate tactic.

Hi there, thanks for posting!

Sounds like an interesting one, some questions that come to mind which I'd just like to run through to make sure we're not missing anything;

• Why do you have Crawl-delay set for all user agents? Officially it's not something Google supports but the reason for that could be the cause of this
• Have you changed any settings in search console? There is a slider for how often you want Google to crawl a site
• Have you had the Search Console notification that you're now on the mobile-first index?
• When you redirected the mobile site, was it all one-to-one redirects? Is there any possibility you've introduced redirect chains?
• After the redesign - are the pages now significantly bigger (in terms of amount of data needed to fully load the page)? Are there any very large assets that are now on every page?
• When you say responsive, is it resizing based on viewport? How much duplication has been added to the page? Is there a bunch of content that is there for mobile but not loaded unless viewed from mobile (and vice versa)?
• When you moved the images, were they the same exact image files or might they now be the full-size image files?

This is just first blush so I could be off the mark but those graphs suggest to me that Google is having to work harder to crawl your pages and, as a result, is throttling the amount of time spent on your site. If the redesign or switch to responsive involved making the pages significantly "heavier" where that could be additional JavaScript, bigger images, more content etc. that could cause that effect. If you've got any sitespeed benchmarking in place you could have a look at that to see whether things have changed. Google also uses pagespeed as a ranking factor so that could explain the traffic drop.

The other thing to bear in mind is that combining the mobile and desktop sites was essentially a migration, particularly if you were on the mobile-first index. It may be that the traffic dip is less related to the crawl rate, but I understand why we'd make the connection there.

Glad to be of help, good luck!

Hi Harrison, thanks for posting - good question!

I think the common understanding is that Google doesn't necessarily treat .co.uk as less relevant in the US per se but things like the domain can hurt clickthrough for example, which can end up having pretty much the same rankings impact anyway (I mean, we're making assumptions here, it would be hard to divorce the two).

I'd usually be all-for making decisions that are as future-proof as possible and you probably wouldn't want to have to go through a US migration in future so I'd lean towards getting one of those neutral ".com" domains you mention which you can use now and in the future.

That being said, you'd be starting from absolutely nothing in terms of trust signals to the new domain so if this business venture is reliant on getting quicker feedback about how you do if you optimise for the US, I think it's also legitimate to take the subfolder approach in the short term, it's just a bit less common and may force you in to a folder migration later if you think you are being limited by the domain.

What do you think?

Hi there! Great to hear you're baking this in from the start.

Open Graph is good to include, Twitter also has its own markup and schema could be a real help if you haven't already considered it. Have you read this post by Cyrus Shepard? It has a pretty comprehensive list of different tagging and some good examples.

Hope that helps!

Hi Rachel, I think István makes a good point about the translated urls but just as a quick follow-up to your original question - it should not cause technical problems to have the page names the same while they are in different directories, because the total file path should be different, as long as you have hreflang properly set up.

Regarding your question about canonical tags - I would not canonicalise some of these language variants to other language variants, even if you do decide to make the page names the same. Hreflang is saying "these two pages are different language variants of the same thing" whereas canonical tags are saying "this page is just the same as this other thing" - the canonical tag doesn't have the language component so could conflict with your hreflang and cause errors with things like return tags. At the very least it could confuse Google as to which page should rank in which country, for instance, how can the /de/ page rank in Germany if we're telling Google it's not the canonical version of the page, but the other one from root is?

Hope that helps!

For the sake of balance, probably worth mentioning that I'm with David in that I've seen a robots.txt noindex work. It has been relatively recently used by a large publisher when they had an article they had to take down but which Google was holding on to. That's irrelevant nuance in this situation but I think David deserves more credit than he got here.

In terms of this specific fix I agree with Nigel - remove the Disallow and add a noindex (prompt Google to crawl the pages, with a sitemap if they don't seem to be shifting). You can re-add the Disallow if you think it's necessary but once all of the appropriate pages have a noindex tag they should stay out of the index and if they are heavily linked to on the site disallowing them could result in a loss of link equity (it'll stop with the link to the disallowed pages). So if you think you can achieve this with just a noindex you might want to leave it at that.

Hi, I agree with Joseph that what you're seeing is probably due to the style breakpoints you have on that page.

**However - **while fetch and render gives us some insight into how Google might interpret a page, it is not what Google is "seeing". Google is viewing the page code and, while there is some evidence it will deprioritise content which is hidden with methods like display:none I really don't think your site will suffer because your break point is different from the one in fetch and render. Changing the break point could conceivably involve quite a lot of front end dev or working with your site templates to avoid causing issues. I understand your concern but I honestly wouldn't put this to the top of your priority list.

Hope that helps!

I wouldn't do that just to push keywords closer to the top of the page. What I'd aim for is what's most useful for users - are there a lot of elements in the nav that don't add to user experience? Navigation can help with link equity flow around the site because it's a way to have links from every page to certain other pages, but putting everything in the nav dilutes link equity and can have a negative impact on user experience.

Essentially, as I say, it's about prioritising the important stuff and being ready to cut the elements that aren't helping.

Forgive me if I'm misinterpreting but if your concern is that your top navigation is pushing things like your H1 further down the page, I wouldn't worry about that. Google understands that web pages have a number of different elements to them and it's very common for a page to have the top navigation above the H1.

I definitely wouldn't remove the navigation or move it further down the page - it's a key part of the user experience and if the site is difficult to navigate that's likely to have a negative impact on the bottom line, and search ranking as a secondary effect.

Hope that helps!

So I'm with James in that you should leave it a few days and see what happens. Google quite often tests results in different positions to see whether the site is a good result. It may be that your optimisations caused Google to start testing the pages for that term, Google may have decided the site isn't a good result or may still be testing.

Without knowing the brand of the site (which I understand you likely can't share) I couldn't comment on what level of competition you might be getting for the brand name but if it's something generic, or something which could be used to describe a whole industry or product, a new site might struggle.

It is also worth considering what James said about the links. How did you get them? Google may think that they are low value links. That could mean the links become devalued (don't have any real impact) or the site might be penalised, in which case it's quite possible not to rank for the brand name. It might be worth doing a quick backlink audit on the site and getting rid of dodgy links regardless of what happens with this specific ranking.

Hi jfishe1988, could I ask - do you have analytics on these pages? In your initial question you suggest that the pages don't have any organic value but in responding to Alick you say that lots of them get organic rankings.

As a couple of broader points:

• Google has given conflicting advice about how low-quality links are treated, Gary Ilyes has said that low-quality links are just ignored (which makes sense as a way to negate negative SEO tactics) so this may not be something you need to worry about if you aren't relying on these 100 odd domains to channel link equity to the site you want to rank. However, John Mueller has said that the disavow file still has value, as I say - conflicting advice.
• If the only purpose of these landing pages is for hyper-targeted PPC and you don't want any link equity that is coming from them in case it poisons the main site, you could consider adding a nofollow tag to those pages. That should mean that Google ignores all links from those pages to your main site. It would insulate your main site from any link-network based penalties but it would also remove any benefits the main site might currently be getting due to any backlinks those domains have.

This thread suggests that the quality score you see in the interface is only activated by searches which exactly match your broad match keyword, but that anything which activates your broad match keyword can impact Google's back end calculation of QS.

I initially agreed with Andrew, above, and the thread doesn't make me change my mind. Using a lot of broad match terms will result in lower control over what the ads are showing for, which is likely to reduce relevance and impact QS. This is just compounded by the fact that apparently the interface won't show accurate QS data in this scenario so you don't know how much you're being impacted. I'd lean towards closer matches wherever possible.

I know you said above you're not interested in account structure advice but that seems the most important outcome of this discussion. Do feel free to correct me if I'm wrong!

Good luck! Happy to help

HI resourcerep, it's great that you're asking questions on the community.

Have you been using this help article? I'm just going to check some assumptions here:

• Have you set up a search console property for the new site so you can select it during the site move operation?
• What errors (if any) do you get from search console?
• What happens when you try to access the old pages, do you just get a warning?

It sounds like the SSL cert is the issue. Unfortunately due to the way HTTPS works it needs a SSL certificate before you can send and receive data.

However, you could find the cheapest SSL cert you can buy, and put that on the old page to allow the redirect to work. To be honest that'll be valuable for more than just this, as having working 301s will help both searcg crawlers and users trying to access the old site.

Hope that helps!

Hi SvenRi, that's an interesting one! The message you're getting from Google suggests that, rather than not finding the canonical tag, the system has reason to believe that the canonical is not representative of the best content.

One thing I'd bear in mind is that Google doesn't take canonical tags as gospel, but rather guidance, so it can just ignore them without there necessarily being a problem in how you've implemented that tag. Another is that while Google says that their crawlers can parse JavaScript, there's evidence that it doesn't parse the page content perfectly.

What happens when you fetch and render the pages in question using Search Console (both the page you want to rank and the page Google is selecting)? Can you see all of the content? Google uses the same JavaScript rendering as Chrome 41 (see here) have you tried accessing with that? You could also try a tool like Screaming Frog with JavaScript rendering switched on to see what kind of page content comes back. It could be worth making sure the canonical is generated properly but I'd also be checking that the page content is being rendered properly to make sure Google is seeing the pages as different as you describe. I'd also check to make sure there isn't a second, conflicting, canonical tag on the page. I know some SPA frameworks can have issues with double-opening HTML tags when one page is accessed after another, that could be something that would confuse a crawler so you could double-check that.

As ever, there are the rumours that Google will start giving much more weight to mobile in terms of indexing. Given your question about things changing recently - does your site have desktop and mobile parity?

If it looks as though everything is kosher, is it possible that the page Google is suggesting is much more heavily linked to internally or externally? If internally you could consider reviewing your internal linking (Will wrote a post about ways to think about internal linking here). You could use a tool like Majestic to look at who is linking to these pages externally, it may be worth double checking that all the links are genuine.

TL;DR I would start with the whole page content, not just the search directives, to make sure that's always being understood properly, then I would look in to linking. These are mainly areas of investigation and next debug steps, hopefully they'll help narrow down the search for you!

Great news, glad you could solve it, thanks for the update David

Hi David, I'm assuming there are no common structural differences between the urls you want to individually redirect and the ones you want to bulk redirect? And what is your set up? It may be that the simplest solution is updating your htaccess file with a list but htaccess needs to be treated with care.

Hi Alan, I see Roman has already given a pretty detailed answer to this question which has some really good points. The only reason I'm not marking it as "good answer" is that, I would add the caveat that you should be building with your audience in mind. Therefore jargon etc. is acceptable if you know your audience talk that way.

Hotjar and Crazyegg are good tools to gather some information, as Roman suggests. I would also consider something like Optimisely or VWO to test variations of the page against each other, that being said I absolutely feel your pain about LinkedIn prices and the cost of testing. I'd recommend experimenting with Google Adwords to try to get some cheaper clicks through, you could also use usertesting.com or usabilityhub to gather specific feedback about the pages, from your target demographic, without having to wait for them to click through naturally.

It'd be worth considering what tracking you can add to the page. If you have Google Tag Manager installed on the site you can add scroll tracking to see how far down the page people actually scroll, Wistia should be able to tell you whether people are watching your video and tracking all the link clicks could give you information on whether they are clicking elsewhere. As a side note, have you considered adding links to supporting pages on the site? It could help reassure people that they are giving contact details to a reputable firm.

Hope that helps!

William and Ramon have good answers here. Roman also has good points  but some of it may be a bit confusing for this thread because it's not fully explained. If the pages haven't been published they shouldn't be indexable to Google based on standard Wordpress functionality. Having a bunch of messy unfinished pages could slow down use of the cms (if only because you have to paginate through them) but as Will and Ramon have said they're unlikely to slow down the site front end unless they are truly huge amounts.

Most relevant is the opportunity cost for not having all those pages published, if they aren't published they can't rank. So if you're posting this to get reasoning to finish the work I'd start by considering the keywords those pages are targeting; use semrush, stat, or similar and pull out the search volume for the keywords. Then you can point to what you're accumulatively missing out on.

Hope that helps

Hi there Bryce, what an unusual problem to have, Facebook is normally all too happy to take our money!

So I don't know all of the details, forgive me if I'm off the mark here, but is the budget incredibly high? Is it daily or lifetime? And what action are you being charged for? Is it cost-per-impression? Otherwise (no offence meant) could it be that the audience aren't being swayed to action by your creative?

I know there is a way of capping frequency on Facebook ads, if it is CPM, there's no way you have that switched on is there? Is it possible that some aspect of your advert is causing Facebook to limit reach (text-heavy image or low quality score for example)? Also, how long have you left each of these tests running?

Apologies if this is all retreading old ground, I would expect the rep to come up with a lot of these, but got to go through the old off-and-on-again routine with these things first.

Hi there I just want to make sure I'm interpreting the question right so do let me know if any of the below is false;

• You're launching a website subdomain to target a different industry niche
• You cannot target this niche with your main site
• Your subdomain will be duplicating the content on your main site and you are concerned about the drawbacks of that duplicate content.

One of my questions would be - if you can't target this niche on the main site, why is the strategy to copy main site content over? It seems as though if duplicate content will successfully target this niche on a subdomain, surely it can do the same on the main domain? If it is the case that there needs to be differentiation then it'd be a good idea to consider how to create content specifically targeting that niche. Search engines are doing everything they can to make it so that their definition of a good site aligns with users' definition of a good site, it's worth considering why it's so difficult to figure out how to get these pages ranking - is it because we're trying to create something that actually isn't targeted to users in a way that'll be successful even if it does rank?

You're right, if you canonicalise the new pages to the old pages they are duplicating, then by design the new ones are unlikely to rank. If you don't canonicalise them, you aren't giving either the old pages or the new pages a fair shot because they aren't offering anything new for the search terms they do target, and you're ignoring the opportunity to cover off new and more specific search terms you couldn't before.

Without knowing the exact details of the situation I would say; don't duplicate main site content onto this subdomain, start from scratch, find out what people in this niche want, what they are searching, what matters to them, design the subdomain to fit that need and only consider the main site in terms of avoiding competing with it.

Re: the issue of whether to go subdomain at all, there is evidence that subdomains don't share as much authority with the site overall as subdirectories do, it depends on how different this new niche is from your main offering. Does it make sense for a user to find this whole other niche in a subdirectory of your main site, or is the niche dissimilar enough that you should differentiate it with a subdomain?

I hope that helps, I think I've probably given more questions than answers but I think they are important questions for your business to consider. If I've misunderstood or if there's anything you'd like to discuss, do message back.

Hi Brit,

Interesting question! So I'd frame my answer in two parts:

1. It would be an idea to deploy Google Analytics code to both domains to give a better idea of what people are actually doing on your site, rather than seeing lots of bounces. If the login portal isn't being tracked then there is the potential that those are actual bounces and not log ins. Tom Capper wrote a post about setting up multiple domain tracking here.
2. If what you are seeing _are _logins then that shouldn't be affecting search traffic volume. Google monitors for users going to your site, then immediately going back to search results and clicking on something else, as a sign that they weren't satisfied with your site as a result. Google can't monitor what users do on non-Google sites and even if they did the fact that some users are going straight to somewhere else isn't necessarily a sign that your site isn't giving them what they want (in this case you're potentially giving them exactly what they want). On the other hand, if what you are seeing as bounces are actually users leaving your site and going back to search results then that could impact search traffic for the reasons above. So rather than considering domain moves, a good first step would probably be tying up those Analytics properties so you're tracking movement the same way your users are experiencing it, that way you can tell if you actually have a problem.

I hope that helps, do come back if you'd like more information

Hi there beritv, could you share some more information about this question? It would be good to get confirmation on a few points:

• What is the main purpose of the event page? I'm picturing a listings page which frequently changes because events are added and taken away, and includes either scroll loading or pagination for the latter results but someone I've chatted with about this thinks it could also be a static page which has changing details for a specific event.
• Which pages are we talking about archiving? Expired individual event pages?
• What is the nature of the archiving we're discussing? How would it work? I can see that Martin is asking whether it's a separate site (in which case I wouldn't recommend it particularly from an SEO perspective because if it's indexable it might start to compete with your live site). The person I've chatted with has asked whether this is a process of deindexing or taking offline the pages. If you could clarify that, it would help a lot.
• Are your IT dept. recommending this as an information store or are they citing SEO reasons?

If the concern is having lots of out-of-date event pages there may be file management and UX considerations to take into account - if having lots of defunct event pages makes your CMS unmanageable then it seems reasonable to have a way of refiling them there but that doesn't necessarily have to be reflected in the live site. On the other hand, are you seeing users landing on event pages for which the event has passed? If so, that's not fantastic UX and is likely to lead to them leaving your site immediately and clicking on another result which would have an SEO impact because it's sending consistent messages to Google that you're not fulfilling the search intent.

Donald is right, here's an article on the Search Engine Journal website that might be helpful in getting you started, it was published about a year ago but should still hold true. It talks mainly about the potential secondary benefits of adopting IA, in terms of having a cleaner setup and therefore potentially ranking better, but those improvements also give some clue as to the hoops you may need to jump through.

To echo MichaelAMG - it is possible to install wordpress on a subdirectory rather than subdomain so you'd be able to host your blog at mysite.com/blog  instead of blog.mysite.com. For SEO reasons, subdirectory is the recommended way to go.

On https://moz.com/learn/seo/domain, Moz explain that search engines view subdomains differently than they do subfolders, which means that it can be harder for successful content in a subdomain to benefit the site as a whole. Companies who host their blogs on subdomains have started moving them to subdirectories and have quite quickly seen positive results, for instance Salesforce moved their blog from a subdomain to a subdirectory and their traffic doubled pretty immediately.

If you are installing wordpress instances on one domain it might be a good idea to keep an eye on your permalinks because if you have url rewriting turned on then the different installs can sometimes confuse each other. Just something to watch out for.