Google indexing thousands crazy search results with %25253

[BoulderJoe]

In GWT I started seeing very strange pages indexed a few weeks, and Google is no reporting over 21,000 of pages (blocked by robots.txt) with weird URLs like this:

http://www.francesphotography.com/?s=no-results:no-results%25252525252525253Ano-results%2525252525252525253Ano-results%252525252525252525253Ano-results%252525252525252525253Ano-results%252525252525252525253Ano-results%252525252525252525253Ano-results%25252525252525252525253Ano-results%25252525252525252525253Ano-results%2525252525252525252525253Adanna&cat=no-results

http://www.francesphotography.com/?s=no-results:no-results%2525253Ano-results%25252525253Ano-results%25252525253Ano-results%25252525253Ano-results%2525252525253Ano-results%25252525252525253Ano-results%25252525252525253Ano-results%25252525252525253Adanna&cat=no-results

The current robots.txt looks like this:

User-agent: *
Disallow: /wp-content

Disallow: /wp-admin

Disallow: /wp-includes
Disallow: /data
Disallow: /slideshows
Disallow: /page/*/?s=
Disallow: /?s=
Disallow: /search

This website is running an up to date WP install with Yoast's Google Analytics and SEO plug-in.  I can't point to anything specific that happened with the site when these URLs started appearing even after I modified the robots.txt.

What can be done to try and stop Google from creating and indexing these goofy URLs?

I see lots of sites having this issue when I search in Google, but no one seems to have a solution.

[BoulderJoe]

As it turns out the problem is with Yoast's Google Analytics plug-in per Yoast.  However, he has not yet released a fix nor given a date for the fix as of yet.  So one either needs to deal with it until fixed or switch plug-ins.

[BoulderJoe]

Hi Sha,

Well, that is a new possible lead, but unfortunately Pictage is basically worthless when it comes to any technological issues.

Hmm, is there some way I could add "noindex" tags to anything link that appears on the Proof page as they are dynamic in appearance?

Thanks,

Joe

[ShaMenz]

Hi again Joe,

After a more detailed look at your site (which has no obvious search box available to users) I was curious as to why all of the things that you are doing on the site seem to have no effect upon the issues you are trying to resolve...and why your site is generating thousands of search queries without a search box!

This says to me "do you have control of all of the content?" ... and it appears that you are using an external service called Pictage to upload and display client portfolios.

So, are you pulling content into your site from Pictage? Is it some kind of white label add-on to your site?

If the pages from Pictage are being generated externally, then the yoast plugin cannot add the "noindex" tag to those pages...if this is the case then I would say you need to contact the Pictage help people and advise them that there is a problem they need to attend to.

Hope that helps,

Sha

[BoulderJoe]

Hi Egol,

Hmm, I have never heard of that possibility.

How can I change the resultant search URL with a Wordpress install?

Thanks.

[BoulderJoe]

Hi Sha,

I made the changes weeks ago, but more pages keep appearing which tells me Google is still trying to index them?

There is already an "s" parameter set in GWT, but I don't really see many options in this screen - are there some settings I'm missing?

There are also page URLs like this one, can they be blocked as well?

www.francesphotography.com/search/no-results:no-results:no-results:no-results%25253Ano-results%252525253Ano-results%252525253Ano-results%252525253Ano-results%252525253Ano-results%252525253Ano-results%252525253Ano-results%25252525253Adanna/feed/rss2/

[EGOL]

In addition to the suggestions already given... if this was my site I would change the URL of the search results page.  Someone might have a robot that is tossing crap queries into your search box.

[ShaMenz]

Hi Joe,

A couple of things:

• If you have made the change to noindex search results recently, it may take some time for the errors to disappear from GWT. If the number of pages continues to grow, then clearly the noindex is not implemented as you expect.
• You could try using the parameter handling feature in GWT to tell googlebot to ignore all pages with the parameter in question. In your search string, the ? says "here come some parameters" and the "s" is the parameter that you want to ignore.

Incidentally, there is definitely something funky happening with the generation of those search strings which should be investigated and resolved as well.

Hope that helps,

Sha

[BoulderJoe]

Yoast's WordPress SEO plug-in automatically does the following:

• RSS feeds are now always noindex, followed. No search engine should ever list an RSS feed as a result in the resultpages.
• Admin, login and registration pages are always noindexed now for the same reason.
• Search result pages are now always noindex, follow.

[ResslerMotors]

This is in your own website's search, right?

I've always heard that you should do on page robots that make it:

no-index, follow

So that all of the links on the page can be followed, but Google will not index it.