Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
When rel canonical tag used, which page does Google considers for ranking and indexing? A/B test scenario!
Hi Moz community, We have redesigned our website and launched for A/B testing using canonical tags from old website to new website pages, so there will be no duplicate content issues and new website will be shown to the half of the website visitors successfully to calculate the metrics. However I wonder how actually Google considers it? Which pages Google will crawl and index to consider for ranking? Please share your views on this for better optimisation. Thanks
Web Design | | vtmoz0 -
AMP vs Responsive Design? Mobile SEO
Hello !! We are developing a new website with responsive design. As is recommended, the idea would be to have a unique site for mobile and desktop, with same content and same url for both devices, using responsive design to adapt the layout depending on the device. My doubt in here is about the AMP pages? If my website has responsive design, perfectly optimized for mobile do I need somehow AMP pages? As far as I understand, these amp pages would be useful if I had different pages for mobile, but this is not the case. Am I correct or am I missing something? Thanks for your help :
Web Design | | AutoEurope1 -
Regarding rel=canonical on duplicate pages on a shopping site... some direction, please.
Good morning, Moz community: My name is David, and I'm currently doing internet marketing for an online retailer of marine accessories. While many product pages and descriptions are unique, there are some that have the descriptions duplicated across many products. The advice commonly given is to leave one page as is / crawlable (probably best for one that is already ranking/indexed), and use rel=canonical on all duplicates. Any idea for direction on this? Do you think it is necessary? It will be a massive task. (also, one of the products that we rank highest for, we have tons of duplicate descriptions.... so... that is sort of like evidence against the idea?) Thanks!
Web Design | | DavidCiti0 -
Drupal vs. Wordpress
I'm working with a site that is built completely in Drupal - including the blog. As we begin on a content strategy initiative the blog is vitally important but lacks core SEO functionality and is cumbersome for the content teams to work with as they have no technical background. While there will be more work required (moving the blog to a sub-domain or separate URL) to play nice with the broader drupal environment I believe it may be the better choice to move over to Wordpress and port over any existing content worth keeping. Has anyone encountered a similar scenario or have an opinion on either platform? Love to hear your thoughts both pros and cons. Thanks in advance.
Web Design | | JonClark150 -
Do you know any tool(s) to check if Google can crawl a URL?
Our site is currently blocking search bots that's why I can't use Google Webmaster Tools' URL fetch tool. In Screamingfrog, there are dynamic pages that can't be found if I crawl the homepage. Thanks in advance!
Web Design | | esiow20130 -
Blog vs. News/Editorial Layout?
We're in the coupon blogging space & saw that one of the larger coupon sites move away from the more traditional blog layout: http://thekrazycouponlady.com they now have more of an editorial type layout. Here is another site which is more similar to our layout: http://hip2save.com. So here are my questions: Which layout type do you feel better serves their visitors & why? How does the affect the SEO of the site? How does it affect the advertising revenues? Which layout do you prefer? Is there strategy in this move for the coupon blog, or is this just a preference on how they now display their content? We're making some updates to our design soon & I wanted to get some feedback on the overall direction we take.
Web Design | | seointern0 -
How to Add canonical tags on .ASPX pages?
What is the proper way (or is it possible) to add canonical tags on website pages that end in .aspx? If you add a canonical tag to the Master Page it will put that exact canonical tag on every page, which is bad. Is there a different version of the tag to put on individual pages? And one to put on the home page without the Master Page error?
Web Design | | Ryan-Bradley0 -
Which Shopping Cart is best for SEO? Magento vs. X-Cart
Comparing X-Cart and Magento, which do you think is better for SEO and why? I am leaning towards Magento and wanted to get some opinions?
Web Design | | BlinkWeb0