Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
We added hundreds of pages to our website & restructured the layout to include 3 additional locations within the sub-pages, same brand/domain name. How long could Google take to crawl/index the new pages and rank the keywords used within those pages?
We added hundreds of pages to our website & restructured the layout to include 3 additional locations within the sub-pages, same brand/domain name. The 3 locations old domains were redirected to their sites within our main brand domain. How long could Google take to crawl/index the new pages and rank the keywords used within those pages? And possibly increase our domain authority hopefully? We didn't want our brand spread out over multiple websites/domains on the internet. This also allowed for more content to be written on pages, per each of our locations service's, as well.
Web Design | | BurgSimpson0 -
How can I fix New 4XX Issue on Site Crawl?
Hi all, My recent site crawl shows 27 4xx issues on this website http://www.rrbusinessconsultants.com/ All of them are for 'posts' on this wordpress website. Here is an example of the issue: http://www.rrbusinessconsultants.com/rr-business-consultants-on-the-rise-of-glassdoor-and-how-companies-are-coping/void(null) The blog page seems to be creating links ending in void(null) which are defaulting to 404 pages. I cannot see the links on the site so cannot see how to remove them. Can anyone provide any insight into how to correct his issue? Many thanks in advance.
Web Design | | skehoe0 -
Regarding rel=canonical on duplicate pages on a shopping site... some direction, please.
Good morning, Moz community: My name is David, and I'm currently doing internet marketing for an online retailer of marine accessories. While many product pages and descriptions are unique, there are some that have the descriptions duplicated across many products. The advice commonly given is to leave one page as is / crawlable (probably best for one that is already ranking/indexed), and use rel=canonical on all duplicates. Any idea for direction on this? Do you think it is necessary? It will be a massive task. (also, one of the products that we rank highest for, we have tons of duplicate descriptions.... so... that is sort of like evidence against the idea?) Thanks!
Web Design | | DavidCiti0 -
Site is getting crushed by spam traffic and Google Webmaster Tools giving crawl warnings. Also...
Currently hosting a site I'm planning on moving to a new server ASAP, 301 redirecting and have a domain that has nice authority and very old. On the current site I need to clean up the blog. I have a few questions actually.... 1. I'd like to remove most of the blog articles as I want the new site to be very high quality, but isn't it dangerous to do a 301 redirect to the same page for all these articles? 2. I want to focus on the new site as the current site has too many issues but still managing to hang in their. is highly outdated yet I don't want to spend a ton of time on the site before the 301 redirect. With the Pigeon and Panda 4.0 rumors being released soon, I want to get the new site completed ASAP. Do you think it's better if I fix the 3. Would removing cloudflare make things better or worse with the crashing of my site due to high traffic (mainly spam on the blog.) 4. My best article by far is outdated, but should I waste time updating it before redirecting or should I just get the new site going? I did way too many guest posts thinking content is king, but at least checked the outgoing links Domain Auth, Page Auth, and MozTrust in OSE, but first off I'm going to remove a page that mentions I'm looking for guest bloggers. I tried to keep the posts relevant but at the time you could get away with 5. Anything I can do to slow down these spammers on Wordpress? I noticed most of them are checking for vulnerabilities but I'm keeping it up to date, have caching setup. Thanks!
Web Design | | eugenecomputergeeks0 -
Using a 301 vs. 302?
I'm running into a very confusing situation - and while I think I've worked through the answer, I'm hoping someone can help provide their insight. I have a client who is in the process of rolling out a responsive site. Because we need to host both Responsive and legacy versions of product pages on the domain we are using the following URL pattern. New Responsive Product Page exampleurl.com/product (existing URL structure) Older Product Pages (redirected to) exampleurl.com/legacy/product The rollout will be approximately 2 months to complete. The question becomes - should a 302 redirect be applied from the existing URl to the /legacy/ URLs until the new designs are launched? Given that the timing will be so short this seems reasonable. Or should a 301 be applied until the new responsive designs are rolled out?
Web Design | | JonClark150 -
Forms vs. Buttons
We are an IT services firm. A conversion for us is completion of a lead form. Generally speaking, is it better to have a form to fill out in the sidebar on most organic pages, or a button that takes you to a lead form? I see both used, which do you think converts better?
Web Design | | CsmBill0 -
One big page vs. multi-step pages
Hi mozers! Brand new to SEO and LOVING it! Having several key questions that I don't see answered yet, but I'll start with one we've been very curious about. Consider this guide we have for Forming a Delaware Corp.
Web Design | | Mase
https://www.upcounsel.com/Free-Legal/Guide/17/Form-A-Delaware-Corporation This is our overview page, giving you a breakdown of what this process involves. We love this page, but (Question1:) does it lack better real "content" rather than lots of links to the guide process itself? Then, you can start to walk through the guide beginning with step one, where each step has crowd sourced answers to it. But as you see, the step pages are all very similar, except for the answers and step info. (Question 2) Would it be better to put all our answers into the one overview page and skip having separate pages for each step? We like the process and simplicity of seeing one step at a time, but then these pages don't seem to have enough unique content on them. Related, at what point (if any) is a page too big with too much content and considered bad for SEO? We're recovering from a big hit from Google, and slowly recovering by nailing down various SEO mistakes. We DO have great, unique and valueable content - now we just need it to rank!0 -
How will engines deal with duplicate head elements e.g. title or canonicals?
Obviously duplicate content is never a good thing...on separate URL's. Question is, how will the engines deal with duplicate meta tags on the same page. Example Head Tag: <title>Example Title - #1</title> <title>Example Title - #2</title> My assumption is that Google (and others) will take the first instance of the tag, such that "Example Title - #1" and canonical = "http://www.example.com" would be considered for ranking purposes while the others are disregarded. My assumption is based on how SE's deal with duplicate links on a page. Is this a correct assumption? We're building a CMS-like service that will allow our SEO team to change head tag content on the fly. The easiest solution, from a dev perspective, is to simply place new/updated content above the preexisting elements. I'm trying to validate/invalidate the approach. Thanks in advance.
Web Design | | PCampolo0
