Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Do you know any tool(s) to check if Google can crawl a URL?
Our site is currently blocking search bots that's why I can't use Google Webmaster Tools' URL fetch tool. In Screamingfrog, there are dynamic pages that can't be found if I crawl the homepage. Thanks in advance!
Web Design | | esiow20130 -
Side Nav. Vs. Top Nav
I have a client that currently has a side navigation and wants to know how changing to a top nav will affect her SEO. We always recommend top nav for user experience but I am not sure if there is a direct effect on SEO. Would the change affect it? Thoughts?
Web Design | | hwade0 -
CSS vs Javascript vs JQuery drop down navigation
For a user / seo perspective, what is the best way to code a drop down menu nav bar? Is it best to use css, javascript or a scripting library like jquery? I am thinking about overall best practice that will not have a negative impact on serps. I am also thinking about what will work best on all types of devices i.e. desk tops, lap tops, smart phones and tablets. What are the Pro's & Cons of Using CSS for Drop Down Menus. What are the Pro's & cons of using Javascript for drop down menus. And the same question for jquery. Thank you all in advance for your ideas.
Web Design | | bronxpad0 -
Pages vs. Posts for SEO
Hi, I would like your thoughts about pages vs. posts for SEO. I understand the difference in terms of WP structure and have read the SEOmoz blog post about setting up your site for SEO success (http://www.seomoz.org/blog/setup-wordpress-for-seo-success). However, if you're trying to rank for a particular keyword, it seems that either one could work, from an on-page SEO perspective, as far as title tag, URL, meta description, etc. So how do you decide whether to set up a page vs. a post? What are the pros and cons, from an SEO perspective, about using one vs. the other? Thanks in advance! Carolina
Web Design | | csmm0 -
What's the best way to structure original vs aggregated content
We're working on a news site that has a mix of news wires such as Reuters and original opinion articles. Currently the site is setup with /world /sports etc categories with the news wire content. Now we want to add the original opinion content. Would it be better to start a new top /Opinion category and then have sub-categories for each Opinion/world, Opinion/sports subject? Or would it be better to simply add an opinion sub-category under the existing news categories, ie /world/opinion? I know Google requests that original content be in a separate directory to be considered for inclusion in Google news. Which would be better for that? Regarding link building, if the opinion sub-categories were under the top news categories, would the link juice be passed more directly than if we had a separate Opinion top category?
Web Design | | ScottDavis0 -
Rel Canonical tag usage on ECommerce website
Hello, I have read up on the rel canonical tag and I'm ready to apply it to my site's categorization structure. However, I'm concerned that, because my website does not have a "view all" button for our product pages, the rel canonical tag would not be appropriate. For example, if you come to my site's main category url, you come to mysite.com/main-category At this level - you get the top 12 items in the category. if you want to see the next page, you click a crawlable link that goes to mysite.com/main-category12-24 etc. etc. The site does not offer a view all function. Would applying the rel canonical tag be appropriate in this instance, or do I have to let Google crawl and index each page independantly? Thanks.
Web Design | | Blenny0 -
Finally have a budget for a great seo ecommerce site but need help choosing wordpress, joomla, modx, magneto or? Thank you in advance for your generosity of time
We finally have a budget and want to dump our intuit/homestead site www.originalartbroker.com Our budget is 5k-10k but could do more if needed. I am slowly catching my competition with this homestead site that I built. But I do realize it is time to step back, figure out what is best, and hire a pro to get the job done. I am green in the seo and web development arena so please go easy on me and please help to point me in the right direction. Just went out on a limb a couple years ago playing with homestead.com site software and built what we have today. Didn't know anything about website development...it sort of just happened. I feel and know that homestead.com solution is hindering what we could be doing due to the bloated nature of the site and inability to perform such task as 301 redirects etc.... I have been able to slowly attain first page seo rankings on keywords based of the artists we carry using this po-dunk homestead platform to build my site after a ton of work education thanks to seomoz and a lot of you. But, have never asked for help and could really use some generosity of time in explaining a solution that would work best for our business. Do we just go with a wordpress site that is similar to our current setup and use their plugins? Do we use a cms software solution like magneto or joomla? We will only have 200-300 pieces at any given time. We are constantly selling and buying new pieces providing us content. We are need of a site that can perform well in terms of seo. I have heard of a lot of people talking about joomla, wordpress, and magneto. Would like to be able to have a product catalogue that ultimately sends whatever inventory we are uploading to our social sites and blogs so I don’t have to pump the product out to all of these sites. We offer free custom framing with our pieces and it would be nice to have a program that could wrap the photos of the pieces with the different frames for our customers. When I add a new piece I would like this software to have a predesigned product page that it plugs the information into. I would like it to create the url extension based of the artists name, medium used, and piece name to create unique and individual urls. I would like it to also create its own H tags throughout that product page according to the artist name description, and medium used. I would like to be able to sink this up to google merchant and other sites to carry our product. Bottom line is we sell art. We sell pieces by specific artists. We are constantly buying and selling. I need something powerful that keeps up with our content
Web Design | | forecastedinvestments1 -
Why is site not being indexed by Google, and not showing on a crawl test??
On a site we developed of which .com is forwarded to .net domain, we quit getting crawled by google on about the 20th of Feb. Now when we try to run a crawl test on either url, we get There was an error fetching this page. Error description For some reason the page returned did not describe itself as an html page. It could be possible that the url is serving an image, rss feed, pdf, or xml file of some sort. The crawl tool does not currently report metrics on this type of data. Our other sites are fine and this was up to this date. We took out noodp, noydir today as the only thing we could think of. Site is on WP cms.
Web Design | | RobertFisher0