Welcome to the Q&A Forum

No problem with stating the obvious...A fetch within GWT was done, but they would start from the homepage and work their way down from what I understand. How would they crawl these 'dead' pages which have been 301'd and 410'd?

Hi Patrick,

The sitemap only shows the active products; therefore, the older, out of stock items, are not in there (definitely a good thing to check).

If you try to go to one of these pages, the header does show a 301 or 410, respectively. But, does Google recrawl all of the pages in their index? How will they see that these are gone if there are no links to many of these pages?

All product descriptions are unique, but unfortunately, a large site scraped them for a few years and recently stopped. That's another big piece to the puzzle as Google gave them credit when in fact, it was coped from the penalized site.

Here's a quick background of the site and issue. A site lost half of its traffic over 18 months ago and its believed to be a Panda penalty. Many, many items were already taken care of and crossed off the list, but here's something that was recently brought up.

There are 30,000 pages indexed in Google,but there are about 12,000 active products. Many of these pages in their index are out of stock items. A site visitor cannot find them by browsing the site unless he/she had bookmarked and item before, was given the link by a friend, read about it, etc. If they get to an old product because they had a link to it, they will see an out of stock graphic and not allow to make the purchase.

So, efforts have been made about 1 month ago to 301 old products to something similar, if possible, or 410 them. Google has not been removing them from the index. My question is how to make sure Google sees that these pages are no longer there and remove from the index? Some of the items have links to them and this will help Google see them, but what about the items which have 0 external / internal links?

Thanks in advance for your assistance.

In working on a site which has about 10,000 items available for sale. Looking in G

To respond, I don't think it was an EMD or PMD (partial matching domain) issue as the domain is not relative to any keywords, industry, etc.

If the 301s are removed from these uppercase urls and sites link to them, would the canonical do enough to inform the crawlers to pick up the lowercase version where the canonical tag points to?

Would this cause link juice to be split between the uppercase urls and lowercase urls, or would the canonical take care of that? Note: there are plenty of links going to the uppercase urls because they were in existence for several years.

Thanks for the other suggestions.

We have a medium size site that lost more than 50% of its traffic in July 2013 just before the Panda rollout. After working with a SEO agency, we were advised to clean up various items, one of them being that the 10k+ urls were all mixed case (i.e. www.example.com/Blue-Widget).

A 301 redirect was set up thereafter forcing all these urls to go to a lowercase version (i.e. www.example.com/blue-widget). In addition, there was a canonical tag placed on all of these pages in case any parameters or other characters were incorporated into a url.

I thought this was a good set up, but when running a SEO audit through a third party tool, it shows me the massive amount of 301 redirects. And, now I wonder if there should only be a canonical without the redirect or if its okay to have tens of thousands 301 redirects on the site.

We have not recovered yet from the traffic loss yet and we are wondering if its really more of a technical problem than a Google penalty. Guidance and advise from those experienced in the industry is appreciated.

Sure. Try going to http://tools.seobook.com/general/spider-test to run a test on:

http://www.maxandchloe.com/bridal-jewelry

http://www.maxandchloe.com/nashelle

These are just two example where the meta tags nor text on the page seems to be getting picked up.

I am analyzing a site which has several thousands of pages, checking the headers, meta tags, and other on page factors. I noticed that the spider tool on SEO Book (http://tools.seobook.com/general/spider-test) does not seem to recognize the meta tags for various pages. However, using other tools including Moz, it seems the meta tags are being recognized.

I wouldn't be as concerned with why a tool is not picking up the tags. But, the site suffered a large traffic loss and we're still trying to figure out what remaining issues need to be addressed. Also, many of those pages once ranked in Google and now cannot be found unless you do a site:// search.

Is it possible that there is something blocking where various tools or crawlers can easily read them, but other tools cannot. This would seem very strange to me, but the above is what I've witnessed recently.

Your suggestions and feedback are appreciated, especially as this site continues to battle Panda.

I don't think you understood the question as it didn't have much to do with links. This is related to a content management system which has an ugly url which a developer then took to make it more user friendly. While a user browsing the site sees clean URLs, webmaster tools reported 500 errors which are essentially server errors.

It doesn't seem those errors are in the console anymore. But, I was wondering if anyone has seen sites receive penalties because of a handful of 500 errors even though the site looks pretty good from a user perspective. (Note: the site is fairly large and a few 500 errors appeared).

Hi. I was curious to know if you were able to find all the urls that needed to be removed, if they were effectively removed, and if your traffic came back. I'm assuming you were dealing with Panda issues.

On a .NET site, there was a url rewrite done about 2 years ago. From a visitor's perspective, it seems to be fine as the urls look clean. But, Webmaster tools reports 500 errors from time to time showing /modules/categories... and /modules/products.... which are templates and how the original urls were structured. While the developer made it look clean, I am concerned that he could have set it up incorrectly. He acknowledged that IIS 7 on a Windows server allows url rewrites to be set up, but the site was done in another way that forces the urls to change to their product name. So, he has believed it to be okay.

However, the site dropped significantly in its ranking in July 2013 which appears to be a Panda penalty. In trying to figure out if this could be a factor in why the site has suffered, I would like to know other webmasters opinions. We have already killed many pages, removed 2/3 of the index that Google had, and are trying to understand what else it could be.

Also, in doing a header check, I see that it shows the /modules/products... page return a 301 status. I assume that this is okay, but wanted to see what others had to say about this.

When I look at the source code of a product page, I see a reference to the /modules/products...

I'm not sure if any of this pertains, but wanted to mention in case you have insight. I hope to get good feedback and direction from SEOs and technical folks

Try running your queries in Bing and see how it compares. If you see the terms you ranked for earlier ranking well in Bing, then you know you've been hit by a penalty in Google.

That's what made it clear for a site I'm involved with and have been trying to recover for many months now.

Hi. I would be curious to know if anyone else has experienced something similar and recovered from Panda. How long did it take you? Did you manually remove the pages, set up 410s or 404s, or create 301s?

I've been working on a site for sometime now which has lost a great of traffic since July 2013. Over the past 2 months, a process has gone underway to manually remove the URLs from the index. The index has been cut in half, but still not at what it was pre-penalty. About 20,000 more pages to figure out what needs to be removed before it reaches the level it was before the massive traffic drop.

Any recovery or insight would be helpful.

That's correct. A 301 was placed to point to the lowercase urls, and then a canonical tag on the same page to try to clean out the parameters in the URL.

Mike,

To answer your question above, the product url without parameters (i.e. www.domain.com/product123) should be what the SERPs pick up. The parameters that were there for a short period are no longer there and haven't been. I've also heard that in time, the crawlers will notice it and index correctly. But, 3 months after canonical tags were implemented and still no updates from what the index had as they are still being shown.

Also, a tool was used to show what crawling the site would look like to a spider. The uppercase urls (i.e. www.domain.com/Product123) have the 301 redirect being picked up. However, the canonical tag didn't seem to be picked up according to the tool. On other pages of the site where the canonical tag was implemented, without the 301, the tool shows detection which is what led me to this post.

A site lost a great deal of traffic in July, which appears to be from an algorithmic penalty, and hasn't recovered yet. It appears several updates were made to their system just before the drop in organic results. One of the issues noticed was that both uppercase and lowercase urls existed. Example urls are:

www.domain.com/product123
www.domain.com/Product123

To clean this up, a 301 redirect was implemented a few months ago.

Another issue found was that many product related urls had a parameter added to the url for a tracking purpose. To clean this up, the tracking parameters were removed from the system and a canonical tag was implemented as these pages were also found in Google's index. The tag forced a page such as www.domain.com/product123?ref=topnav to be picked up as www.domain.com/product123.

So now, there is a 301 to address the upper and lowercase urls and a canonical tag to address the parameters from creating more unnecessary urls.

A few questions here:

-Is this redunant and can cause confusion to the serps to have both a canonical and 301 redirect on the same page?

-Both the 301 and canonical tag were implemented several months ago, yet Google's index is still showing them. Do these have to be manually removed with GWT individually since they are not in a subfolder or directory?

Looking forward to your opinions.

Hi Matias. Now that it's been a few months, I was curious to see if you were able to get your traffic back. Let us know if there has been some progress and what actions helped cure the curse.

I'm associated with a site that ranked fairly well. Earlier in the summer, the blog was moved from a subfolder to a subdomain for various reasons. While the reasons seemed valid at the time, the site's traffic plummeted about 1-2 weeks later. We've still been trying to analyze as many other changes were made a few weeks prior; however, the arrows are pointing to the subfolder to subdomain change which may have really caused this plague. We're now looking into moving it back to see if it will resolve the problem.

In glancing quickly at your first site, I see that Google isn't indexing it nor are any inbound links found. You need to have legitimate links pointing to the domain, not spammy links, to give it some creditability. If you get too many links too fast, this can also hurt you.

In regards to "How did they connect my sites together..?", Google has its own domain registrar which can see who owns what sites.

Hope this helps.