Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ecommerce web design read more toggle vs menu link on home page and product pages
Hello, We have an Ecommerce store. We have a lot of content on the home page and product pages and we are going back and forth between which one to use between a toggle "Read More" "Show Less" toggle for each section and a anchor linked menu. We have long product pages We're thinking a read more toggle is more appropriate for category descriptions so that they can go at the top of the category and not take up space. But the read more toggle with lots of content scrolls the page down and doesn't scroll it back up when you hit "show less" We're leaning towards a linked menu for the home pages and product pages for this reason, but an accordion type set of toggles would look nicer. What do you recommend, and how have you set up your read more toggles if they have lots of info so that they are not confusing? Are there other options? ' Not looking for code (I can do that) I'm looking for ideas on the cleanest home page, category pages, and product pages when they have tons and tons of textual content. Wanting to trim it up and make it look compact and neat! Thanks!
Web Design | | BobGW0 -
Hi, I have a doubt. If we want to hide unwanted text in a web page its possible with "" tag. And my question "does a search engine crawl those text? help me.
I want to hide a lot of text behind my site page. I know its possible with that tag. But in what way a search engine looks at those text? Hidden or they are crawled and indexed.
Web Design | | FhyzicsBCPL0 -
Responsive Design vs Mobile For This Site?
They are going to do an entire website redesign for http://gaport.com/ and I think they should adhere to responsive design best practices. However, I'm just saying that because everything I have read says that's the "way of the future" if not the way of the present already. Any reason, they shouldn't do that and keep the desktop/mobile sites? Thanks, Ruben
Web Design | | KempRugeLawGroup0 -
Using a 301 vs. 302?
I'm running into a very confusing situation - and while I think I've worked through the answer, I'm hoping someone can help provide their insight. I have a client who is in the process of rolling out a responsive site. Because we need to host both Responsive and legacy versions of product pages on the domain we are using the following URL pattern. New Responsive Product Page exampleurl.com/product (existing URL structure) Older Product Pages (redirected to) exampleurl.com/legacy/product The rollout will be approximately 2 months to complete. The question becomes - should a 302 redirect be applied from the existing URl to the /legacy/ URLs until the new designs are launched? Given that the timing will be so short this seems reasonable. Or should a 301 be applied until the new responsive designs are rolled out?
Web Design | | JonClark150 -
HTML vs WordPress
Which website gets better SEO results? I was told that HTML is crawled better than WordPress. Which is a more expensive website to design? What are the pros and cons of each?
Web Design | | CapitolShine0 -
3rd party commenting systems vs native?
I'm curious what you all think about using a 3rd party commenting system (like Disqus) vs the native wordpress commenting system? I've read so many reviews online it makes my head spin, so I wanted to see if any of you have any experience, or perhaps some trusted case studies. I was using the native comment system for a while, and then tried out Disqus; which seems to be good, but I'm not sure if people know how to easily get notified of new comments. With the native system there was a check box that said "subscribe," plus I used a plugin to redirect a first time commenter to a welcome page, as well as sent an email to them. I feel like Disqus makes it harder for people to get notified on new comments. However, I like giving people the ability to log in via different 3rd party channels (facebook/twitter/disqus, etc.) I know there are some 3rd party tools that allow you to do this on the native comment system as well. Any way, I'm just curious if anyone had any experience. Also I'm assuming the audience makes a big difference. My target readership is mommy's and parents, and not necessarily a 'tech" one; so I want to make the ability to comment very simple and easy for them. That's key. I'm sort of leaning on moving back over to the Wordpress comment system.
Web Design | | NoahsDad0 -
How will engines deal with duplicate head elements e.g. title or canonicals?
Obviously duplicate content is never a good thing...on separate URL's. Question is, how will the engines deal with duplicate meta tags on the same page. Example Head Tag: <title>Example Title - #1</title> <title>Example Title - #2</title> My assumption is that Google (and others) will take the first instance of the tag, such that "Example Title - #1" and canonical = "http://www.example.com" would be considered for ranking purposes while the others are disregarded. My assumption is based on how SE's deal with duplicate links on a page. Is this a correct assumption? We're building a CMS-like service that will allow our SEO team to change head tag content on the fly. The easiest solution, from a dev perspective, is to simply place new/updated content above the preexisting elements. I'm trying to validate/invalidate the approach. Thanks in advance.
Web Design | | PCampolo0 -
Subdomain vs Sub Folder
Hi. This is my first question here guys and gals so please be gentle. We are creating a town based website that will hold news, events, articles and relevant local content. We are also creating a business directory that will be part of this. My question relates to an issue we have regarding how the directory will be treated as part of the "network". Although we aren't 100% sure on the domain name yet, we want the directory to be found as easily as the domain itself. Even better, if the two could be treated seperately but strengthen the overall theme for the domain we'd be in keyword heaven. Therefore in your opinion (hopefully with pro's and con's) which do you think will rank better in Google, Yahoo and Bing (or all three);
Web Design | | rufo
The directory seperate as a subdomain or as a folder within the main domain? So for example - providing we are including links between the two sections and the site is www.sitetown.com Which is better for the directory itself? http://www.sitetown.com/directory http://directory.sitetown.com or you're mad Steve, use a seperate domain altogether www.sitetowndirectory.com I hope you can help, obi-wan, you're my only hope..... PS - seomoz still rocks. Thanks
Steve0