Magento E-Commerce Crawl Issues
-
Hi Guys,
First post here! I am responsible for a Magento e-commerce store and there are a few crawl issues and potential solutions that I am working and would like to get some advice to see if you agree with my approach.
Old Product Pages - The majority of our stock is seasonal, therefore when a product sells out, it is not usually going to come back into stock. However the approach for Magento websites is to leave the page present but take the product off the category pages, so users can still find these pages from the search engines and they are orphaned pages as not linked to from elsewhere and not totally clear products are out of stock (just doesn't show the size pulldown or 'Add to Basket' button). There is no process in place to 301 redirect these pages either.
My solution to this problem is to:
1. Change design of these pages so a clear message is shown to users that the product is out of stock and suggest related products to reduce bounce rates. I was also planning on having a link from an 'Out of Stock' page on the site to these products so they are orphaned but is this required do you think?
2. When I know for sure (e.g. over a month) that the product will not be returned (e.g. refund) by the user, then 301 redirect the product pages back to category page. How do other users 301 redirect their pages in Magento, I would like an easy to use system.
Crawl Errors Identified in Google Webmaster Tools
It seems in the last 2 weeks there has been a sharp increase in the number of soft 404 pages identified on the website. When I inspect these pages they seem to be categories and sub categories that no longer have any products in them.
However, I don't want to delete these pages as new products might come in and go onto these category pages, therefore how should I approach this? A suggestion I have thought of is to put related products on to these pages?
Any better ideas?
Thanks,
Graeme
-
Hi Graeme,
For old product pages - your solution is good regarding showing users alternatives to the out of stock products. No need for an "out of stock page" as there's no value in that for crawlers or users. Regarding point 2 - if you redirect discontinued product pages to category pages that should be fine although Google may regard that as a soft 404. If there are loads of products like this and you 301 them in one go then the chances are it will flag up in Google WMT. If there are a small number and you introduce them gradually then you'll probably be fine.
For the crawl errors question, adding value to the pages in terms of related products is a good solution if that's viable and the pages will be different enough from each other (i.e. no duplicate content). One thing that isn't clear at the moment is if you're redirecting empty category pages all to the homepage - or if it's possible to redirect or canonical them to their parent category.
e.g. For home -> clothing -> men's clothing -> shoes
If all the men's shoes are discontinued, then redirect that page to men's clothing rather than to the homepage. This reduces your chances of getting a soft 404, and is also arguably a better user experience.
Hope that helps,
George
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Shopify Website Page Indexing issue
Hi, I am working on an eCommerce website on Shopify.
Intermediate & Advanced SEO | | Bhisshaun
When I tried Indexing my newly created service pages. The pages are not getting indexed on Google.
I also tried manual indexing of each page and submitted a sitemap but still, the issue doesn't seem to be resolved. Thanks0 -
Complicated Title Tag Issues. Experts, Please Help!
Hey there Moz community! This is the first time I ask a question here so please forgive me if I miss any forum etiquette. I am managing SEO for an educational site which is built in React Js, and so far much of the job has been keyword research and site optimization. The site still has slow PageSpeed though. The Issues - 4 weeks ago we published 20 or so content pieces, I had pre-prepared title tags and meta descriptions. But when we released the content there was a programming error that made all of the pages show another title tag for all 20 pages instead of the pre-prepared individual title tags. I noticed this after 3 days and the issue was fixed within 6 days, but by then Google had crawled and indexed the pages. And now I can't get Google to change to the pre-prepared tags no matter what I do! I've tried changing the content, changing the URL of one of the pages, and I've sent Google spiders to re-crawl the pages multiple times. The super weird thing is that the correct title tag shows in the 'navigation bar/tabs bar' on google chrome: But NOT when I view the source code for the page: Yesterday I was taking a walk in the park and I just couldn't stop thinking about it (it is really starting to get to me by now since nothing works), so I ran back home and looked closely at one of these pages in the Google search console. And I noticed something I hadn't seen before… BOTH of the title tags can be found in the HTML: Pre-prepared title tag: <title></strong>UK Seat Belt & Car Seat Laws: The Definitive Guide<strong></title> The other title tag (in src section): title=Ace%20The%20DMV%20Permit%20Test%20%26%20Get%20Your%20License Could this be the problem or what do you think? I've understood that Google has automated title tags and that they can choose their own if they think it fits better, but the title tags aren't even close to describing the topic as it is now so it doesn’t make any sense. All answers are greatly appreciated! Your advice is life-saving for a learner like me. P.S. I love SEO but it can be very frustrating sometimes! Thank you very much, Leo
Intermediate & Advanced SEO | | Leowa0 -
URL Structure For E-commerce Sites
Hi Guys, I was wondering what would be the optimal and best URL structure for sub-categories on a E-commerce site for SEO purposes. Example if my category was dresses and I had multiple sub-categories within dresses would 1 or 2 below be the better URL structure? 1) Domain + Category + Sub-Category be the most suitable URL structure: Sleeveless Dresses URL: clothingstore.com/dresses/sleeveless-dresses Midi Dresses URL: clothingstore.com/dresses/midi-dresses 2) OR would excluding the category be better Domain + Sub-Category like: Sleeveless Dresses URL: clothingstore.com/sleeveless-dresses Midi Dresses URL: clothingstore.com/midi-dresses Do you think it makes much of a difference, is shorter better and more effective in this case? E.g. Rand discuses in this article: https://moz.com/blog/15-seo-best-practices-for-structuring-urls that having the keyword in the URL serves as anchor text, so wouldn't having additional keywords dilute value in this case? Plus he mentions shorter URLs the better. Cheers, Chris
Intermediate & Advanced SEO | | jayoliverwright1 -
Prevent Google from crawling Ajax
With Google figuring out how to make Ajax and JS more searchable/indexable, I am curious on thoughts or techniques to prevent this. Here's my Situation, we have a page that we do not ever want to be indexed/crawled or other. Currently we have the nofollow/noindex command, but due to technical changes for our site the method in which this information is being implemented if it is ever displayed it will not have the ability to block the content from search. It is also the decision of the business to not list the file in robots.txt due to the sensitivity of the content. Basically, this content doesn't exist unless something super important happens, and even if something super important happens, we do not want Google to know of its existence. Since the Dev team is planning on using Ajax/JS to pull in this content if the business turns it on, the concern is that it will be on the homepage and Google could index it. So the questions that I was asked; if Google can/does index, how long would that piece of content potentially appear in the SERPs? Can we block Google from caring about and indexing this section of content on the homepage? Sorry for the vagueness of this question, it's very sensitive in nature and I am trying to avoid too many specifics. I am able to discuss this in a more private way if necessary. Thanks!
Intermediate & Advanced SEO | | Shawn_Huber0 -
Using rel cannonical to host a blog as a path on our e-commerce website
There has been recent suggestion (from Rand) that hosting your blog as a folder rather than a subdomain is much better from an SEO point of view. Unfortunately, our blog is hosted on a subdomain with a different technology stack to the main e-commerce site. We are finding it quite tricky to migrate to a folder given the different technologies. Is the following a suitable solution? - 301 redirect from mysite.com/blog/cool-blog-post to blog.mysite.com/cool-blog-post - And then put mysite.com/blog/cool-blog-post" /> on blog.mysite.com/cool-blog-post Would be great to have your thoughts on this guys - I can't figure out if it will work or be an SEO fail.
Intermediate & Advanced SEO | | HireSpace0 -
URL Parameter Being Improperly Crawled & Indexed by Google
Hi All, We just discovered that Google is indexing a subset of our URL’s embedded with our analytics tracking parameter. For the search “dresses” we are appearing in position 11 (page 2, rank 1) with the following URL: www.anthropologie.com/anthro/category/dresses/clothes-dresses.jsp?cm_mmc=Email--Anthro_12--070612_Dress_Anthro-_-shop You’ll note that “cm_mmc=Email” is appended. This is causing our analytics (CoreMetrics) to mis-attribute this traffic and revenue to Email vs. SEO. A few questions: 1) Why is this happening? This is an email from June 2012 and we don’t have an email specific landing page embedded with this parameter. Somehow Google found and indexed this page with these tracking parameters. Has anyone else seen something similar happening?
Intermediate & Advanced SEO | | kevin_reyes
2) What is the recommended method of “politely” telling Google to index the version without the tracking parameters? Some thoughts on this:
a. Implement a self-referencing canonical on the page.
- This is done, but we have some technical issues with the canonical due to our ecommerce platform (ATG). Even though page source code looks correct, Googlebot is seeing the canonical with a JSession ID.
b. Resubmit both URL’s in WMT Fetch feature hoping that Google recognizes the canonical.
- We did this, but given the canonical issue it won’t be effective until we can fix it.
c. URL handling change in WMT
- We made this change, but it didn’t seem to fix the problem
d. 301 or No Index the version with the email tracking parameters
- This seems drastic and I’m concerned that we’d lose ranking on this very strategic keyword Thoughts? Thanks in advance, Kevin0 -
Indexing issue or just time?
Hey guys, When I publish a post on our blog, I notice that it barely shows up in SERPs even if I copy and paste the title verbatim into Google. All my settings in Yoast are correct from what I've seen. Is this just Google slowly getting around to crawling our site? Or is something else wrong here? We recently shut down and relaunched our site about 3 weeks ago. Here is the site URL: The Tech Block
Intermediate & Advanced SEO | | ttb0 -
Duplicate Content Issue
Why do URL with .html or index.php at the end are annoying to the search engine? I heard it can create some duplicate content but I have no idea why? Could someone explain me why is that so? Thank you
Intermediate & Advanced SEO | | Ideas-Money-Art0