URL Parameter & crawl stats

[Mr.bfz]

Hey Guys,I recently used the URL parameter tool in WBT to mark different urls that offers the same content.I have the parameter "?source=site1" , "?source=site2", etc...It looks like this: www.example.com/article/12?source=site1The "source parameter" are feeds that we provide to partner sites and this way we can track the referral site with our internal analytics platform.Although, pages like:www.example.com/article/12?source=site1 have canonical to the original page www.example.com/article/12, Google indexed both of the URLs
www.example.com/article/12?source=site1andwww.example.com/article/12Last week I used the URL parameter tool to mark "source" parameter "No, this parameter doesnt effect page content (track usage)" and today I see a 40% decrease in my crawl stats.In one hand, It makes sense that now google is not crawling the repeated urls with different sources but in the other hand I thought that efficient crawlability would increase my crawl stats.In additional, google is still indexing same pages with different source parameters.I would like to know if someone have experienced something similar and by increasing crawl efficiency I should expect my crawl stats to go up or down?I really appreciate all the help!Thanks!

[katemorris]

I wouldn't freak out too much over the crawl rate immediately. Wait a few weeks and see how things go. It sounds like you did the right thing and should see the benefits over the next few weeks.

[Mr.bfz]

Thanks Martin,

I see what are you saying, but I dont think it is possible to equal the amount of pages been crawled every day with the amount of duplicate pages that I have.

Virtually, every page that I have, have a duplicate version "source=site1", and the decrease was only around 35%.

Another thing that happen and I did not mention is that I recently redirected my cdn.site.com version of the site to the original site.com.

Im thinking that all the new redirect inside the site, could also have effected the crawlability. Any idea?

Today, the crawl stats is a bit higher than yesterday but still under the last 90 average.

Thanks

[Martijn_Scheijbeler]

Hi Arie,

Do you have an idea about how many pages were crawled before and what the number of duplicate pages was? Then you could find out if this would clarify the decrease in crawl stats. I've seen it before that making sure that Google isn't able to crawl some pages will decrease the crawl rate so you're probably OK with this.