Welcome to the Q&A Forum

Still no root cause detected. Also strange, starting May 21, sudden and steady decline in # of product detail pages with valid product schema. No recent production changes that would impacted that. From ~3,000 to ~300 pages with valid product schema. Seems suspicious timing with sudden lack of cached pages. Again, only impacting product detail pages.

The way the site works, any googlebot is directed to the html version. The only way this wouldn't fly is if google has some new user agents with very strange names. Could that be the case?

@blueprintmarketing Hi Tom. Thank you - email sent!

Absolutely no idea what is going on. All of our category / subcategory and other support pages are indexed and cached as normal, but suddenly none of our product pages are cached, and all of the product / offer schema snippets have been dropped from the serps as well (price, review count, average rating etc).

When I inspect a product detail page url in GSC, I am either getting errors or it is returned as a soft 404. There have been no recent changes to our website that are obvious culprits. When I request indexing, it works fine for non-product pages, but generates the "Something went wrong
If the issue persists, try again in a few hours" message for any product page submitted.

We are not SEO novices. This is an Angular 7 site with a Universal version launched back in October (new site, same domain), and until this strange issue cropped up we'd enjoyed steady improvement of rankings and GSC technical issues. Has anyone seen anything like this?

We are seeing rapid deterioration in rankings overnight for all product detail pages due to this issue. A term / page combination that ranked for over a decade in the top 10 lost 10 places overnight...

There's just no obvious culprit. Using chrome dev tools to view as googlebot, everything is kosher. No weird redirects, no errors, returns 200 and page loads.

Thank You

Hi

Thanks. Yeah, the thing here is not just a rank decline, it is the page dropped from the index altogether. Google instead choosing a different page from our site to rank for the same terms, but at a lesser position. Confirmed  page dropped using site: www.domain.com searches. This is an ecommerce site with some 5,000 distinct product pages. We are not new, we've been around with an ecommerce site since 1997. The current site has existed in current for ~14 years. Again, it's seemingly random product detail pages being dropped. No manual action, no site errors (500 etc), no redirects or other problematic factors.

Roughly 1/3 of all tracked keywords in our long-running Moz pro campaign experienced either rank declines and a different page from the site being returned for the search, or suddenly not ranking whatsoever due to this matter of pages disappearing from the index. We are talking about product detail pages with >100 customer reviews  and other UCG, tons of backlinks etc.

Interestingly, a handful of pages and terms seem to have corrected overnight. Not all - but a handful. We updated sitemaps yesterday and added some sketchy domains to our disavow file last night.   None of the "usual suspects" types of issues explain this phenomenon.

Hello

We've been around a very long time, and I have a long running pet set of core terms and pages tracked using Moz and other tools.  With no changes to the content or site or htaaccess or robots.txt or sitemap, insignificant backlink changes etc, we saw a ton of important product pages drop out of the index the week ending December 22 2019.  We are still ranking for many of the terms associated, but at far worse positions since the pages G is choosing instead for those terms are not as focused. I need to be clear that this has not happened across the board, but seemingly at random.

When I look in G Search Console, the pages are  submitted and indexed (last crawl yesterday), mobile friendly, have breadcrumbs, and the only warning are product level for lack of optional fields under offers (nothing new, not particular to the dropped pages in question here).

So, what happened the week ending December 22???? Should I expect the dust to settle and the pages to return? Extremely strange.

Thx

Hello

Thank you for the detailed, helpful response. I should note that most of our SEO traffic does NOT come from forum pages. The overwhelming amount of natural search traffic we receive is from product detail, category, subcategory, and related (how-to articles etc) pages on the main www.xxxx.com site.

I am concerned mainly with the potential fallout of Google seeing 3000 or more 404 pages if we just delete the forum and kill the server, and am looking for the best way to handle that. I am ok with returning a 410 or redirecting anything that tries to hit forum.xxxx.com to www.xxxx.com.

What do you think?

Thanks

Hello All

Our site www.xxxx.com has long had a forum subdomain forum.xxxx.com.

We have decided to sunset the forum. We find that the 'Ask a Question' function on product pages and our social media presence are more effective ways of answering customers' product & project technical Qs.

Simply shutting down the forum server is going to return thousands of 404s for forum.xxxx.com, which I can't imagine would be helpful for the SEO of www.xxxx.com even though my understanding is that subdomains are sort of handled differently than the main site. We really tremendously on natural search traffic for www.xxxx.com, so I am loathe to make any moves that would hurt us.

I was thinking we should just keep the forum server up but return 410s for everything on it, including the roughly ~3,000 indexed pages until they are removed from the index, then shut it down.

The IT team also gave the option of simply pointing the URL to our main URL, which sorta scares me because it would then 200 and return the same experience hitting it from forum.xxxx.com as www.xxxx.com, which sounds like a very bad idea. (Yes, we do have canonicals on www.xxxx.com).

In your opinion, what is the best way to handle this matter?

Thank You

No because I thought Moz followed GoogleBot rules. Is this not the case?

I noticed that most of the errors would not be occurring if Moz's tool followed the rules implemented in sites robots.txt. Has anyone else seen this problem and do you know if Moz will fix this?

Oleg

Thank you for the reply. I am going to submit to G as well. What's really interesting is that for some of those ancient pages that have somehow resurfaced, you can view the cache dates.  Those pages seem to have cache dates from late nov and dec 2012. But for others, attempting to view the cached version yields a google 404!

IMO, this suggests to its a bug.

As an aside, you are certainly correct about canonical and pagination issues on our site. We have implemented canonical thus far only on product pages (over 10k prod pages), and I've had getting next/prev for pagination of subcategories as a top priority for months now.

Thanks