Welcome to the Q&A Forum

No problem with stating the obvious...A fetch within GWT was done, but they would start from the homepage and work their way down from what I understand. How would they crawl these 'dead' pages which have been 301'd and 410'd?

Hi Patrick,

The sitemap only shows the active products; therefore, the older, out of stock items, are not in there (definitely a good thing to check).

If you try to go to one of these pages, the header does show a 301 or 410, respectively. But, does Google recrawl all of the pages in their index? How will they see that these are gone if there are no links to many of these pages?

All product descriptions are unique, but unfortunately, a large site scraped them for a few years and recently stopped. That's another big piece to the puzzle as Google gave them credit when in fact, it was coped from the penalized site.

Here's a quick background of the site and issue. A site lost half of its traffic over 18 months ago and its believed to be a Panda penalty. Many, many items were already taken care of and crossed off the list, but here's something that was recently brought up.

There are 30,000 pages indexed in Google,but there are about 12,000 active products. Many of these pages in their index are out of stock items. A site visitor cannot find them by browsing the site unless he/she had bookmarked and item before, was given the link by a friend, read about it, etc. If they get to an old product because they had a link to it, they will see an out of stock graphic and not allow to make the purchase.

So, efforts have been made about 1 month ago to 301 old products to something similar, if possible, or 410 them. Google has not been removing them from the index. My question is how to make sure Google sees that these pages are no longer there and remove from the index? Some of the items have links to them and this will help Google see them, but what about the items which have 0 external / internal links?

Thanks in advance for your assistance.

In working on a site which has about 10,000 items available for sale. Looking in G

To respond, I don't think it was an EMD or PMD (partial matching domain) issue as the domain is not relative to any keywords, industry, etc.

If the 301s are removed from these uppercase urls and sites link to them, would the canonical do enough to inform the crawlers to pick up the lowercase version where the canonical tag points to?

Would this cause link juice to be split between the uppercase urls and lowercase urls, or would the canonical take care of that? Note: there are plenty of links going to the uppercase urls because they were in existence for several years.

Thanks for the other suggestions.

We have a medium size site that lost more than 50% of its traffic in July 2013 just before the Panda rollout. After working with a SEO agency, we were advised to clean up various items, one of them being that the 10k+ urls were all mixed case (i.e. www.example.com/Blue-Widget).

A 301 redirect was set up thereafter forcing all these urls to go to a lowercase version (i.e. www.example.com/blue-widget). In addition, there was a canonical tag placed on all of these pages in case any parameters or other characters were incorporated into a url.

I thought this was a good set up, but when running a SEO audit through a third party tool, it shows me the massive amount of 301 redirects. And, now I wonder if there should only be a canonical without the redirect or if its okay to have tens of thousands 301 redirects on the site.

We have not recovered yet from the traffic loss yet and we are wondering if its really more of a technical problem than a Google penalty. Guidance and advise from those experienced in the industry is appreciated.

Sure. Try going to http://tools.seobook.com/general/spider-test to run a test on:

http://www.maxandchloe.com/bridal-jewelry

http://www.maxandchloe.com/nashelle

These are just two example where the meta tags nor text on the page seems to be getting picked up.

I am analyzing a site which has several thousands of pages, checking the headers, meta tags, and other on page factors. I noticed that the spider tool on SEO Book (http://tools.seobook.com/general/spider-test) does not seem to recognize the meta tags for various pages. However, using other tools including Moz, it seems the meta tags are being recognized.

I wouldn't be as concerned with why a tool is not picking up the tags. But, the site suffered a large traffic loss and we're still trying to figure out what remaining issues need to be addressed. Also, many of those pages once ranked in Google and now cannot be found unless you do a site:// search.

Is it possible that there is something blocking where various tools or crawlers can easily read them, but other tools cannot. This would seem very strange to me, but the above is what I've witnessed recently.

Your suggestions and feedback are appreciated, especially as this site continues to battle Panda.

I don't think you understood the question as it didn't have much to do with links. This is related to a content management system which has an ugly url which a developer then took to make it more user friendly. While a user browsing the site sees clean URLs, webmaster tools reported 500 errors which are essentially server errors.

It doesn't seem those errors are in the console anymore. But, I was wondering if anyone has seen sites receive penalties because of a handful of 500 errors even though the site looks pretty good from a user perspective. (Note: the site is fairly large and a few 500 errors appeared).

Hi. I was curious to know if you were able to find all the urls that needed to be removed, if they were effectively removed, and if your traffic came back. I'm assuming you were dealing with Panda issues.

On a .NET site, there was a url rewrite done about 2 years ago. From a visitor's perspective, it seems to be fine as the urls look clean. But, Webmaster tools reports 500 errors from time to time showing /modules/categories... and /modules/products.... which are templates and how the original urls were structured. While the developer made it look clean, I am concerned that he could have set it up incorrectly. He acknowledged that IIS 7 on a Windows server allows url rewrites to be set up, but the site was done in another way that forces the urls to change to their product name. So, he has believed it to be okay.

However, the site dropped significantly in its ranking in July 2013 which appears to be a Panda penalty. In trying to figure out if this could be a factor in why the site has suffered, I would like to know other webmasters opinions. We have already killed many pages, removed 2/3 of the index that Google had, and are trying to understand what else it could be.

Also, in doing a header check, I see that it shows the /modules/products... page return a 301 status. I assume that this is okay, but wanted to see what others had to say about this.

When I look at the source code of a product page, I see a reference to the /modules/products...

I'm not sure if any of this pertains, but wanted to mention in case you have insight. I hope to get good feedback and direction from SEOs and technical folks

In glancing quickly at your first site, I see that Google isn't indexing it nor are any inbound links found. You need to have legitimate links pointing to the domain, not spammy links, to give it some creditability. If you get too many links too fast, this can also hurt you.

In regards to "How did they connect my sites together..?", Google has its own domain registrar which can see who owns what sites.

Hope this helps.

Try running your queries in Bing and see how it compares. If you see the terms you ranked for earlier ranking well in Bing, then you know you've been hit by a penalty in Google.

That's what made it clear for a site I'm involved with and have been trying to recover for many months now.

Hi Matias. Now that it's been a few months, I was curious to see if you were able to get your traffic back. Let us know if there has been some progress and what actions helped cure the curse.