How do we get search engine bots to the item detail pages?
-
The problem we have is that we have lots of inventory pages. These inventory pages have a bunch of links at the top linking to different styles of the item, up to 56 links in some cases. Then each item listed has a link to the item's detail page and a link to the item's shop owner's page. So if a page has 50 items shown, there are really 100 links just for the inventory. This is not taking into account the header links, footer links, sidebar links to other sections on the site.
We have all these links to help consumers move through the site. The problem is that every item detail page on the site is not getting indexed and actually I think it's more like over 50% of the item detail pages are not indexed because the search engines are too busy following all these other links.
Should we nofollow, index the links to the different styles of the item, the shop owner page? Or what should we do to get the search engine bots to index our item detail pages?
-
CFSSEO,
Can you share the site so we can have a look at it to provide a better, more specific, answer for you? If not, I hope this general answer will help:
You should only have one canonical page per product. Even if there are product variants (size, color, etc...) those should typically be handled with a drop-down selector or checkbox on the product page instead of creating an entire new product page for each variant.
If this is a large affiliate or drop-shipping website where you are pulling in feeds from different merchants/distributors/stores the bitter pill you may have to swallow is that such sites are dying out in the SERPs because Google doesn't think they offer anything above and beyond their own search results. Someone wants to buy a Blue Widget and they go to Google and type "Cheap blue widget" and a bunch of merchants'/distributors' stores show up... why do they need to put another site in-between the searcher and their destination? The only way to work within this paradigm is to prune the site down and focus mostly on category-level searches, price comparison searches, etc... and by offering more useful features, including robust customer reviews for products and merchants, tools to help the shopper compare options, in-depth buying guides to educate shoppers on their choices... but thousands of product pages with default manufacturer content will no longer do the trick, and may actually harm the rest of your site.
This may or may not apply to your situation. It is tough to give advise like this without knowing the site in question. However, I hope it has helped point you in the right direction.
-
I'm worried the search engines are not indexing all the item detail pages due to too many links at the top of the inventory pages. There are so many options of itemA that there are 50 links about different types of itemA at the top of the page, then 50 items are shown with a link to the item detail page and a link to the 'shop' of that seller. Along with a search box on the page. And there could be hundreds of these items but default of 50 shown on a page. So there's well over 100 links on a page, and I'm worried the bots aren't finding all the item detail pages.
-
You have to reduce the number of pages that Google "sees" when you are showing search result pages. If you have too many sort versions on a given query, you are giving Google a bunch of duplicate (or closely duplicate) pages to spider and it prevents them from getting down into your item detail page.
I would find a way to setup a main / generic pagination result so that Google can spider that and then as they go through each page, they find links into the products. All the other versions of your search results, I would no follow links to them or put them in a folder that you can robots.txt.
This way consumers can search and sort to their hearts desire, but you provide a clear path for Google.
Also, be sure to setup a XML sitemap and include links to all of your item detail pages. You can also split out into multiple sitemaps if you have large enough number of item detail pages. A good article on this is here
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Infinite Scroll and SEO - Is it enough to only link to the previous and next page in the pagination?
Hi all, We are implementing an eCommerce site where the results pages of the products will be visibile on one page (always loading new products when you scroll down the page). Now, I have read that the Google spiders cannot "load" new products scrolling down the page, hence the spider only sees the first few products of the results page. Our developer wants to implement a system where a users sees the first products on example.com/products Then scrolling down, he will see new products with the URL changing to example.com/page/2 and so on. Is it enough that we add a pagination link that goes from example.com/products to example.com/page/2 Then another link that goes from example.com/page/2 to example.com/page/3 and so on, so the Google spider can make his way through all the pages? Or is that too much deep linking and the spider wouldn't even crawl all the results pages? Any recommendations how to go about this? Many thanks in advance!
Web Design | | Gabriele_Layoutweb0 -
Advice needed: Google crawling for single page applicartions with java script
Hi Moz community,we have a single page application (enjoywishlist.com) with a lot of content in java script light boxes. There is a lot of valuable content embedded but google can not crawl the content and we can missing out on some opportunities as a result. I was wondering if someone was able to solve a similar issue (besides moving the content from the java script to the HTML body). There appears to be a few services sprouting up to handle single page applications and crawling in google.http://getseojs.com/https://prerender.io/Did anyone use these services? Some feedback would be much appreciated!ThanksAndreas
Web Design | | AndreasD0 -
301 forwarding during site migration problem - several url versions of the same page....
Hello, I'm migrating from an old site to a new site, and 301 forwarding many of the pages... My key problem is this I'm seeing www.website.com/ indexed in SE and www.website.com/default.aspx in showing as URL when I'm on homepage - should I simply 301 forward both of these? Then for several internal pages there are 2/3 versions of each page indexed. Canonicalization issues. Again, I'm wondering whether I should 301 forward each URL even if there are several different indexed URLs for the same page? Your advice will be welcome! Thanks in advance - Luke
Web Design | | McTaggart0 -
How to change the entire contents and design in my site without getting troubles with google?
Hello everyone This is my first post over here. In the next few weeks we going to change the entire content and design in our site. The site has 240 pages with poor contents and design. Except 301 redirects for all the old url’s I wanted to consult with you what is the right way to do it without harm my organic traffic that come from google? How google refers to this kind of changes? Which steps should I need to take to do it properly? Hope to get your help in the issue. Tahnks in advance.
Web Design | | JonsonSwartz0 -
Google Penalizing Websites that Have Contact Forms at Top of Website Page?
Has anyone else heard of Google penalizing websites for having their contact forms located at the top of the website? For example http://www.austintenantadvisors.com/ Look forward to hearing other thoughts on this.
Web Design | | webestate1 -
Map Search Tools to integrate on Accommodation Website
Hello all, Can any recommend a Map Search tool that i can integrate into my accommodation website. Ideally I want to be able to pin my clients on the map search with links back to their listing page on my website and provide an alternative search facility for clients looking for accommodation. Am covering the South Africa region specifically. Am assuming that i could go down the Google Maps route but would really like to know what alternatives there are on offer. Also what SEO considerations do i need to think about when adding this to my website. Thanks in advance for any help.
Web Design | | SamanthaRiggien0 -
How not to get penalized by having a Single Page Interface (SPI) ?
Guys, I run a real estate website where my clients pay me to advertise their properties. The thing is, from the beginning, I had this idea about a user interface that would remain entirely on the same page. On my site the user can filter the properties on the left panel, and the listings (4 properties at each time) are refreshed on the right side, where there is pagination. So when the user clicks on one property ad, the ad is loaded by ajax below the search panel in the same page .. there's a "back up" button that the user clicks to go back to the search panel and click on another property. People are loving our implementation and the user experience, so I simply can't let go of this UI "inovation" just for SEO, because it really is something that makes us stand out from our competitors. My question, then, is: how not to get penalized in SEO by having this Single Page Interface, because in the eyes of Google users might not be browsing my site deep enough ?
Web Design | | pqdbr0 -
Dynamic pages and code within content
Hi all, I'm considering creating a dynamic table on my site that highlights rows / columns and cells depending on buttons that users can click. Each cell in the table links to a separate page that is created dynamically pulling information from a database. Now I'm aware of the google guidelines: "If you decide to use dynamic pages (i.e., the URL contains a "?" character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them few." So we wondered whether we could put the dynamic pages in our sitemap so that google could index them - the pages can be seen with javascript off which is how the pages are manipulated to make them dynamic. Could anyone give us a overview of the dangers here? I also wondered if you still need to separate content from code on a page? My developer still seems very keen to use inline CSS and javascript! Thanks a bundle.
Web Design | | tgraham0