Welcome to the Q&A Forum

Thanks Dan,

I appended the asterisk.  I also tested the robots.txt in google webmaster tools.  It alerted me of the required : colon after the disallow statement.  I missed that somehow.

The next crawl should finally ignore those search catalog pages.

Best Regards,

Alan

Hi Dan,

Latest SEOMOZ crawl still reports 108 Duplicate Page Titles.  Almost all of them are from the Search page.

Search

http://www.practicerange.com/search.aspx?s=Name ASC&p=5

Search

http://www.practicerange.com/search.aspx?s=Name ASC&p=50

Search

http://www.practicerange.com/search.aspx?s=Name ASC&p=51

Search

http://www.practicerange.com/search.aspx?s=Name ASC&p=52

etc.....

I added Disallow http://www.practicerange.com/Search.aspx but it still picked it up during the crawl. Here is the robots.txt.

User-Agent: *
Disallow: /Admin/
Disallow: /App_Themes/
Disallow: /Assets/
Disallow: /Checkout/
Disallow: /ClientApi/
Disallow: /ConLib/
Disallow: /FCKeditor/
Disallow: /images/
Disallow: /Install/
Disallow: /Layouts/
Disallow: /Members/
Disallow: /webcharts/
Disallow: /Basket.aspx
Disallow: /BuyProduct.ashx
Disallow: /*.axd$
Disallow http://www.practicerange.com/Search.aspx

Sitemap: <http: www.practicerange.com="" sitemap.xml="">Sitemap: http://www.practicerange.com/sitemap.xml</http:>

Am I missing something on the syntax to get the crawl to ignore our Search Pages?

Best Regards,

Alan

|   |   |   |   |
|   |   |   |   |
|   |   |   |   |
|   |   |   |   |
|   |   |   |   |
|   |

Thanks Dan,

Got it.  The catalog pages are not beneficial and the duplication could be hurting.

So I added the disallow statement for the Search pages.  We will see if it fixes the problem on our next crawl and change syntax as needed.

Would  a similar disallow for the 404.aspx would solve the Home Page 404 crawl errors?  Something like...

Disallow http://www.practicerange.com/404.aspx?*

Best Regards,

Alan Wills

I've been a Pro Member for nearly a year and I am bound and determined to finally clean up all the crawl errors on our site PracticeRange.com.

We have 180 errors for Duplicate Page Titles and Duplicate Content.  I fixed many of the pages that were product pages with duplicate content.  Those product descriptions were edited and now have unique content.

However, there remain plenty of the errors that are puzzling.  Many of the errors reference the same pages, for example, the Home Page, Login Page and the Search page (our catalog pages).
In the case of the Catalog Page errors, these type pages would have the same title every time "Search" and the results differ according to category.

http://www.practicerange.com/Search.aspx?m=6
http://www.practicerange.com/Search.aspx?m=15

If this is  rel=canonical issue, how do I fix it on a search result page?  I want each of the different category type pages to be indexed.  One of them is no more important than the other. So how would I incorporate the rel=canonical?

In the case of the Home Page errors, I'm really confused.  I don't know where to  start to fix these.  They are the result of a 404 error that leads to the home page.  Is the content of the 404 page the culprit since it contains a link to the home page?

Here are examples of the Home Page type of crawl errors.

http://www.practicerange.com/404.aspx?aspxerrorpath=/Golf-Training-Aids/Golf-Nets/~/Assets/ProductImages/products/Golf-Training-Aids/Rubber-Wooden-Tee-Holder.aspx

http://www.practicerange.com/404.aspx?aspxerrorpath=/Golf-Training-Aids/Golf-Nets/~/Assets/ProductImages/products/Golf-Training-Aid/Impact-Bag.aspx

Thanks ,

Alan WillsPracticeRange.com