Welcome to the Q&A Forum

Still no root cause detected. Also strange, starting May 21, sudden and steady decline in # of product detail pages with valid product schema. No recent production changes that would impacted that. From ~3,000 to ~300 pages with valid product schema. Seems suspicious timing with sudden lack of cached pages. Again, only impacting product detail pages.

The way the site works, any googlebot is directed to the html version. The only way this wouldn't fly is if google has some new user agents with very strange names. Could that be the case?

@blueprintmarketing Hi Tom. Thank you - email sent!

Absolutely no idea what is going on. All of our category / subcategory and other support pages are indexed and cached as normal, but suddenly none of our product pages are cached, and all of the product / offer schema snippets have been dropped from the serps as well (price, review count, average rating etc).

When I inspect a product detail page url in GSC, I am either getting errors or it is returned as a soft 404. There have been no recent changes to our website that are obvious culprits. When I request indexing, it works fine for non-product pages, but generates the "Something went wrong
If the issue persists, try again in a few hours" message for any product page submitted.

We are not SEO novices. This is an Angular 7 site with a Universal version launched back in October (new site, same domain), and until this strange issue cropped up we'd enjoyed steady improvement of rankings and GSC technical issues. Has anyone seen anything like this?

We are seeing rapid deterioration in rankings overnight for all product detail pages due to this issue. A term / page combination that ranked for over a decade in the top 10 lost 10 places overnight...

There's just no obvious culprit. Using chrome dev tools to view as googlebot, everything is kosher. No weird redirects, no errors, returns 200 and page loads.

Thank You

Hi

Thanks. Yeah, the thing here is not just a rank decline, it is the page dropped from the index altogether. Google instead choosing a different page from our site to rank for the same terms, but at a lesser position. Confirmed  page dropped using site: www.domain.com searches. This is an ecommerce site with some 5,000 distinct product pages. We are not new, we've been around with an ecommerce site since 1997. The current site has existed in current for ~14 years. Again, it's seemingly random product detail pages being dropped. No manual action, no site errors (500 etc), no redirects or other problematic factors.

Roughly 1/3 of all tracked keywords in our long-running Moz pro campaign experienced either rank declines and a different page from the site being returned for the search, or suddenly not ranking whatsoever due to this matter of pages disappearing from the index. We are talking about product detail pages with >100 customer reviews  and other UCG, tons of backlinks etc.

Interestingly, a handful of pages and terms seem to have corrected overnight. Not all - but a handful. We updated sitemaps yesterday and added some sketchy domains to our disavow file last night.   None of the "usual suspects" types of issues explain this phenomenon.

Hello

We've been around a very long time, and I have a long running pet set of core terms and pages tracked using Moz and other tools.  With no changes to the content or site or htaaccess or robots.txt or sitemap, insignificant backlink changes etc, we saw a ton of important product pages drop out of the index the week ending December 22 2019.  We are still ranking for many of the terms associated, but at far worse positions since the pages G is choosing instead for those terms are not as focused. I need to be clear that this has not happened across the board, but seemingly at random.

When I look in G Search Console, the pages are  submitted and indexed (last crawl yesterday), mobile friendly, have breadcrumbs, and the only warning are product level for lack of optional fields under offers (nothing new, not particular to the dropped pages in question here).

So, what happened the week ending December 22???? Should I expect the dust to settle and the pages to return? Extremely strange.

Thx

Hello

Thank you for the detailed, helpful response. I should note that most of our SEO traffic does NOT come from forum pages. The overwhelming amount of natural search traffic we receive is from product detail, category, subcategory, and related (how-to articles etc) pages on the main www.xxxx.com site.

I am concerned mainly with the potential fallout of Google seeing 3000 or more 404 pages if we just delete the forum and kill the server, and am looking for the best way to handle that. I am ok with returning a 410 or redirecting anything that tries to hit forum.xxxx.com to www.xxxx.com.

What do you think?

Thanks

Hello All

Our site www.xxxx.com has long had a forum subdomain forum.xxxx.com.

We have decided to sunset the forum. We find that the 'Ask a Question' function on product pages and our social media presence are more effective ways of answering customers' product & project technical Qs.

Simply shutting down the forum server is going to return thousands of 404s for forum.xxxx.com, which I can't imagine would be helpful for the SEO of www.xxxx.com even though my understanding is that subdomains are sort of handled differently than the main site. We really tremendously on natural search traffic for www.xxxx.com, so I am loathe to make any moves that would hurt us.

I was thinking we should just keep the forum server up but return 410s for everything on it, including the roughly ~3,000 indexed pages until they are removed from the index, then shut it down.

The IT team also gave the option of simply pointing the URL to our main URL, which sorta scares me because it would then 200 and return the same experience hitting it from forum.xxxx.com as www.xxxx.com, which sounds like a very bad idea. (Yes, we do have canonicals on www.xxxx.com).

In your opinion, what is the best way to handle this matter?

Thank You

No because I thought Moz followed GoogleBot rules. Is this not the case?

I noticed that most of the errors would not be occurring if Moz's tool followed the rules implemented in sites robots.txt. Has anyone else seen this problem and do you know if Moz will fix this?

Oleg

Thank you for the reply. I am going to submit to G as well. What's really interesting is that for some of those ancient pages that have somehow resurfaced, you can view the cache dates.  Those pages seem to have cache dates from late nov and dec 2012. But for others, attempting to view the cached version yields a google 404!

IMO, this suggests to its a bug.

As an aside, you are certainly correct about canonical and pagination issues on our site. We have implemented canonical thus far only on product pages (over 10k prod pages), and I've had getting next/prev for pagination of subcategories as a top priority for months now.

Thanks

We recently noticed google is showing approx 23,000 indexed .jsp urls for our site. These are ancient pages that haven't existed in years and have long been 301 redirected to valid urls. I'm talking 6 years.

Checking the serps the other day (and our current SEOMoz pro campaign), I see that a few of these urls are now replacing our correct ones in the serps for important, competitive phrases.

What the heck is going on here?

Is Google suddenly ignoring rewrite rules and redirects?

Here's an example of the rewrite rules that we've used for 6+ years:

RewriteRule ^(.*)/xref_interlux_antifoulingoutboards&keels.jsp$ $1/userportal/search_subCategory.do?categoryName=Bottom%20Paint&categoryId=35&refine=1&page=GRID [R=301]

Now, this 'bottom paint' url has been incredibly stable in the serps for over a half decade. All of a sudden, a google search for 'bottom paint' (no quotes) brings up the jsp page at position 2-3.

This is just one example of something very bizarre happening. Has anyone else had something similar happen lately?

Thank You

<colgroup><col width="64"></colgroup>
| RewriteRule ^(.*)/xref_interlux_antifoulingoutboards&keels.jsp$ $1/userportal/search_subCategory.do?categoryName=Bottom%20Paint&categoryId=35&refine=1&page=GRID [R=301] |

Point taken on the multiple H1s on the home page. Good call.

Digging deeper into the matter, it appears a backup version of the site was up over the weekend and on monday. It is possible that a different robots.txt file was up.

Looking at crawl results for 1/16/2012, Google is showing some really strange things. Thousands of allowed/regularly crawled/.indexed pages are listed as 'URL restricted by robots.txt' for 1/16/2012.

We immediately made sure the latest robots.txt file (the one that should be up) had these pages as allowed, and it does, so we made sure that is the one in use. It may or may not have been over the weekend.

I resubmitted both regular and mobile sitemaps last night and both are showing with a checkbox today.

Hopefully this all sorts itself out in a few days. We certainly can't afford any meaningful long term issues with our natural search visibility.

Thank you.

Of course, I welcome additional feedback.

DMG

We are a very stable, time tested domain (over 15 yrs old) with thousands of stable, time tested inbound links. We are a large catalog/e commerce business and our web team has over a decade's experience with coding, seo etc. We do not engage in link exchanges, buying links etc and adhere strictly to best white hat seo practices.

Our SERPs have generally been very stable for years and years. We continually update content, leverage user generated content etc, and stay abreast of important algorithm and policy changes on Google's end.

On Wednesday Jan 18th, we noticed dramatic, disturbing changes to our SERPs. Our formerly very stable positions for thousands of core keywords dropped. In addition, there is no snippet in the SERPs and no cached page for these results.

Webmaster tools shows our sitemap most recently successfully downloaded by Google on Jan 14th. Over the weekend and monday the 16th, our cloud hosted site experienced some downtime here and there. I suspect that the sudden issues we are seeing are being caused by one of three possibilities:

1. Google came to crawl when the site was unavailable.
However, there are no messages in the account or crawl issues otherwise noted to indicate this.

2. There is a malicious link spam or other attack on our site.

3. The last week of December 2011, we went live with Schema.org rich tagging on product level pages. The testing tool validates all but the breadcrumb, which it says is not supported by Schema. Could Google be hating our Schema.org microtagging and penalizing us? I sort of doubt bc category/subcategory pages that have no such tags are among those suffering.

Whats odd is that ever since we went live with Schema.org, Google has started preferring very thin content pages like video pages and articles over our product pages. This never happened in the past.

the site is: www.jamestowndistributors.com

Any help or ideas are greatly, greatly appreciated.

Thank You

DMG