Crawl Budget vs Canonical
-
Got a debate raging here and I figured I'd ask for opinions. We have our websites structured as
site/category/product
This is fine for URL keywords, etc. We also use this for breadcrumbs. The problem is that we have multiple categories into which a category fits. So "product" could also be at
site/cat1/product
site/cat2/product
site/cat3/productObviously this produces duplicate content. There's no reason why it couldn't live under 1 URL but it would take some time and effort to do so (time we don't necessarily have). As such, we're applying the canonical band-aid and calling it good. My problem is that I think this will still kill our crawl budget (this is not an insignificant number of pages we're talking about). In some cases the duplicate pages are bloating a site by 500%.
So what say you all? Do we just simply do canonical and call it good or do we need to take into account the crawl budget and actually remove the duplicate pages. Or am I totally off base and canonical solves the crawl budget issue as well?
-
agreed! we ran into the same problem with content (articles, etc). if you think of it in the same way as blog posts, they each have a unique URL, but with tags (i.e. categories) you are able to get them posted to the appropriate category landing pages.
have a somewhat related issue that i posted here
-
Another great way to go is to not put the category in the product URL. That was usually the best solution when I work on e-commerce sites.
-
Hi Highland,
I would defiantly work on making sure that your product only lives in one category. The canonical tag is a nice little band-aid but it still fix the root of the problem. I would suggest you can have it listed in many different categories but it only lives in one category at the product level. So for instance:
It's displayed here
site/cat1
site/cat2
site/cat3But it only displays product details at a url like this
site/category/product
I'm not a huge fan of having Google crawl 4 or 5 extra pages per product just to find a canonical tag when you could just spend the extra programming time to make it work correctly.
Casey
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
When rel canonical tag used, which page does Google considers for ranking and indexing? A/B test scenario!
Hi Moz community, We have redesigned our website and launched for A/B testing using canonical tags from old website to new website pages, so there will be no duplicate content issues and new website will be shown to the half of the website visitors successfully to calculate the metrics. However I wonder how actually Google considers it? Which pages Google will crawl and index to consider for ranking? Please share your views on this for better optimisation. Thanks
Web Design | | vtmoz0 -
How can I fix New 4XX Issue on Site Crawl?
Hi all, My recent site crawl shows 27 4xx issues on this website http://www.rrbusinessconsultants.com/ All of them are for 'posts' on this wordpress website. Here is an example of the issue: http://www.rrbusinessconsultants.com/rr-business-consultants-on-the-rise-of-glassdoor-and-how-companies-are-coping/void(null) The blog page seems to be creating links ending in void(null) which are defaulting to 404 pages. I cannot see the links on the site so cannot see how to remove them. Can anyone provide any insight into how to correct his issue? Many thanks in advance.
Web Design | | skehoe0 -
Dedicated landing pages vs responsive web design
I've been doing some research into web design and page layout as my company is considering a re-design. However, we have come to an argument around responsive webdesign vs SEO. The argument is around me (SEO specialist) arguing that I want dedicated pages for all my content as it's good for SEO since it focuses keywords and content properly, and it still adheres to good user journeys (providing it's done correctly), and my web designer arguing that mobile traffic is on the rise (which it is I know) so we should have more content under 1 URL and use responsive web design so that users can just scroll through content instead of having to keep be direct to different pages. What do I do... I can't find any blogs, questions, or whiteboards that really touches on this topic, so can anyone advise me on whether I should: Create dedicated landing pages for each bit of content which is good for SEO and taking users on a journey around my site OR All content that is relative to a landing page, put all under that one URL (e.g. "About us" may have info on the company, our team, our history, careers) and allow people to scroll down what could be a very long page on any device, but may effect SEO as I can't focus keywords/content under one URL properly, so it may effect rankings. Any advice SEO and user experience whizzes out there?
Web Design | | blackboxideas0 -
Site is getting crushed by spam traffic and Google Webmaster Tools giving crawl warnings. Also...
Currently hosting a site I'm planning on moving to a new server ASAP, 301 redirecting and have a domain that has nice authority and very old. On the current site I need to clean up the blog. I have a few questions actually.... 1. I'd like to remove most of the blog articles as I want the new site to be very high quality, but isn't it dangerous to do a 301 redirect to the same page for all these articles? 2. I want to focus on the new site as the current site has too many issues but still managing to hang in their. is highly outdated yet I don't want to spend a ton of time on the site before the 301 redirect. With the Pigeon and Panda 4.0 rumors being released soon, I want to get the new site completed ASAP. Do you think it's better if I fix the 3. Would removing cloudflare make things better or worse with the crashing of my site due to high traffic (mainly spam on the blog.) 4. My best article by far is outdated, but should I waste time updating it before redirecting or should I just get the new site going? I did way too many guest posts thinking content is king, but at least checked the outgoing links Domain Auth, Page Auth, and MozTrust in OSE, but first off I'm going to remove a page that mentions I'm looking for guest bloggers. I tried to keep the posts relevant but at the time you could get away with 5. Anything I can do to slow down these spammers on Wordpress? I noticed most of them are checking for vulnerabilities but I'm keeping it up to date, have caching setup. Thanks!
Web Design | | eugenecomputergeeks0 -
Moz crawl showing up ?s=keyword pages as errors
Hi all, Hoping someone can she some light on a fix with ref to wordpress and the search function it uses as Moz is craling some pages which reference the search domain.com/?s=keyword Errors showing up are duplicate pages, descriptions and titles. The search function is not important on this site and I have tried to use a plugin which disables the search page which it does but these errors still show up. Can anyone assist as this is the final piece of the puzzle and then we're down to 0 issues on the site.
Web Design | | wtfi0 -
Side Nav. Vs. Top Nav
I have a client that currently has a side navigation and wants to know how changing to a top nav will affect her SEO. We always recommend top nav for user experience but I am not sure if there is a direct effect on SEO. Would the change affect it? Thoughts?
Web Design | | hwade0 -
Infinite Scrolling vs. Pagination on an eCommerce Site
My company is looking at replacing our ecommerce site's paginated browsing with a Javascript infinite scroll function for when customers view internal search results--and possibly when they browse product categories also. Because our internal linking structure isn't very robust, I'm concerned that removing the pagination will make it harder to get the individual product pages to rank in the SERPs. We have over 5,000 products, and most of them are internally linked to from the browsing results pages in the category structure: e.g. Blue Widgets, Widgets Under $250, etc. I'm not too worried about removing pagination from the internal search results pages, but I'm concerned that doing the same for these category pages will result in de-linking the thousands of product pages that show up later in the browsing results and therefore won't be crawlable as internal links by the Googlebot. Does anyone have any ideas on what to do here? I'm already arguing against the infinite scroll, but we're a fairly design-driven company and any ammunition or alternatives would really help. For example, would serving a different page to the Googlebot in this case be a dangerous form of cloaking? (If the only difference is the presence of the pagination links.) Or is there any way to make rel=next and rel=prev tags work with infinite scrolling?
Web Design | | DownPour0 -
Subdomain vs Sub Folder
Hi. This is my first question here guys and gals so please be gentle. We are creating a town based website that will hold news, events, articles and relevant local content. We are also creating a business directory that will be part of this. My question relates to an issue we have regarding how the directory will be treated as part of the "network". Although we aren't 100% sure on the domain name yet, we want the directory to be found as easily as the domain itself. Even better, if the two could be treated seperately but strengthen the overall theme for the domain we'd be in keyword heaven. Therefore in your opinion (hopefully with pro's and con's) which do you think will rank better in Google, Yahoo and Bing (or all three);
Web Design | | rufo
The directory seperate as a subdomain or as a folder within the main domain? So for example - providing we are including links between the two sections and the site is www.sitetown.com Which is better for the directory itself? http://www.sitetown.com/directory http://directory.sitetown.com or you're mad Steve, use a seperate domain altogether www.sitetowndirectory.com I hope you can help, obi-wan, you're my only hope..... PS - seomoz still rocks. Thanks
Steve0