My website's pages are not being indexed correctly

[digitalMSB]

Hi,

One of our websites, which is actually a price comparison engine, facing indexing problem at Google.

When we check “site:mywebsite.com “, there are lots of pages indexed which are not from mywebsite.com but from merchants websites. The index result page also shows merchant’s page title. In some cases the title is from merchant’s site but when the given link is accessed it points to mywebsite.com/index. Also the cache displays the merchant’s product page as the last indexed version rather than showing ours.

The mywebsite.com has quite few Merchants that send us their product feed. Those products are listed on comparison page with prices. The merchant’s links on comparison page are all no-follow links but some of the (not all) merchant’s product pages are indexed against mywebsite.com as mentioned above instead of product comparison page of mywebsite.com

How can we fix the issue?

Thanks!

[digitalMSB]

Yeah i was thinking the same....

The interesting thing is we've removed the redirect page a week ago and replaced it with javascript redirect code. is that a good practice?

[RyanPurkey]

Ah. Regarding #3: If you have a disallow in the robots.txt the search engines won't pick up the noindex. Ensure the noindex code is in place on the applicable pages, remove the disallow, and the pages should be removed after they're crawled.  getting that relationship straightened out might help with some of the other things as well. Cheers!

[digitalMSB]

Thanks Ryan for the response. We'll surely prevent crawling of search result pages. Please check below points too. Thanks!!!

1. The cache page shows merchant product page in full version as well as in text-only version.
2. The title shown on the result page is also of the merchant's product page title.
3. One thing on the comparison price page is merchants are redirected to their respective websites, the links are nofollow, but redirect page is indexed even after having it on robots.txt and noindex on redirect page.
4. The redirect page is indexed like mywebsite.com/redirect-50187889-0
5. Comparison listing is not similar to internal search result page but result pages are crawl-able.

[digitalMSB]

no iFrames being used.

[RyanPurkey]

Thumbs up to Don's rec. Also when you look at the text only cache what kind of page are you seeing, if any? Sometimes the site: search is a little inconsistent so you can try forcing the delivery of certain pages with the inurl: modifier. One last caveat that comes to mind is that if the comparison listing is similar to an internal search results page, Google may not ever list it, "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search engines." from: https://support.google.com/webmasters/answer/35769  Cheers!

[donford]

How are you merchant prices / info being displayed on your site? From your site or using IFrames?