Good to use disallow or noindex for these?
-
Hello everyone,
I am reaching out to seek your expert advice on a few technical SEO aspects related to my website. I highly value your expertise in this field and would greatly appreciate your insights.
Below are the specific areas I would like to discuss:a. Double and Triple filter pages:
I have identified certain URLs on my website that have a canonical tag pointing to the main /quick-ship page. These URLs are as follows:
https://www.interiorsecrets.com.au/collections/lounge-chairs/quick-ship+black
https://www.interiorsecrets.com.au/collections/lounge-chairs/quick-ship+black+fabricConsidering the need to optimize my crawl budget, I would like to seek your advice on whether it would be advisable to disallow or noindex these pages. My understanding is that by disallowing or noindexing these URLs, search engines can avoid wasting resources on crawling and indexing duplicate or filtered content. I would greatly appreciate your guidance on this matter.
b. Page URLs with parameters:
I have noticed that some of my page URLs include parameters such as ?variant and ?limit. Although these URLs already have canonical tags in place, I would like to understand whether it is still recommended to disallow or noindex them to further conserve crawl budget. My understanding is that by doing so, search engines can prevent the unnecessary expenditure of resources on indexing redundant variations of the same content. I would be grateful for your expert opinion on this matter.
Additionally, I would be delighted if you could provide any suggestions regarding internal linking strategies tailored to my website's structure and content. Any insights or recommendations you can offer would be highly valuable to me.
Thank you in advance for your time and expertise in addressing these concerns. I genuinely appreciate your assistance. If you require any further information or clarification, please let me know. I look forward to hearing from you.
Cheers!
-
@williamhuynh You're correct to pay attention to parameters in your URLs, as they can have an impact on how search engines crawl and index your site. It's crucial, however, to handle them strategically.
Using canonical tags on these pages is already a good move. It signals to search engines which version of the page should be treated as the main one. Canonicalization helps avoid potential duplicate content issues and makes your website easier to understand from a search engine's perspective.
However, I'd be careful to disallow these pages or use a "noindex" tag. Disallowing these URLs in your robots.txt file might seem like a good way to save the crawl budget, but it can have unintended side effects. When you disallow a URL, it means that search engines can't access it at all, which could impact the crawling and indexing of your main (canonical) pages. This is especially true if these parameterized URLs have unique backlinks or user engagement signals that could be beneficial for your canonical URLs.
As for the "noindex" approach, this tells search engines not to include the page in their index. However, if these pages have valuable backlinks or user engagement signals, you might be missing out on some SEO value by not indexing them.
In my opinion, if your website is large and you're genuinely concerned about the crawl budget, a more suitable approach might be to use Google Search Console's URL Parameters tool. This tool lets you inform Google how to handle specific URL parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Solved How to solve orphan pages on a job board
Working on a website that has a job board, and over 4000 active job ads. All of these ads are listed on a single "job board" page, and don’t obviously all load at the same time. They are not linked to from anywhere else, so all tools are listing all of these job ad pages as orphans. How much of a red flag are these orphan pages? Do sites like Indeed have this same issue? Their job ads are completely dynamic, how are these pages then indexed? We use Google’s Search API to handle any expired jobs, so they are not the issue. It’s the active, but orphaned pages we are looking to solve. The site is hosted on WordPress. What is the best way to solve this issue? Just create a job category page and link to each individual job ad from there? Any simpler and perhaps more obvious solutions? What does the website structure need to be like for the problem to be solved? Would appreciate any advice you can share!
Reporting & Analytics | | Michael_M2 -
Site is generating long path URLs
Hi, We've seen recently in Search Console Coverage report that website is generating long path URLs that we actually don't have.
Technical SEO | | eUniverse
Here's an example: https://autocovers.co.uk/car-mats/outdoor-basic/indoor-car-covers/shop/contact-us/shipping-delivery/about-us/about-us/indoor-car-covers/ Does anybody knows what's the issue behind it? Thanks!0 -
Canonical homepage link uses trailing slash while default homepage uses no trailing slash, will this be an issue?
Hello, 1st off, let me explain my client in this case uses BigCommerce, and I don't have access to the backend like most other situations. So I have to rely on BG to handle certain issues. I'm curious if there is much of a difference using domain.com/ as the canonical url while BG currently is redirecting our domain to domain.com. I've been using domain.com/ consistently for the last 6 months, and since we switches stores on Friday, this issue has popped up and has me a bit worried that we'll loose somehow via link juice or overall indexing since this could confuse crawlers. Now some say that the domain url is fine using / or not, as per - https://moz.com/community/q/trailing-slash-and-rel-canonical But I also wanted to see what you all felt about this. What says you?
Technical SEO | | Deacyde0 -
The use of robots.txt
Could someone please confirm that if I do not want to block any pages from my URL, then I do not need a robots.txt file on my site? Thanks
Technical SEO | | ICON_Malta0 -
How to use rel canonical?
Hi, I am having some questions about this and I think you can help me on this. Here I have the example of my problem: pagination: Suppose that I have a new with 2 pages http://www.espectador.com/noticias/208907/fernando-pereira-encuesta-de-cifra-prendio-una-lucecita-amarilla-en-el-pit-cnt you can access the first page by different ways: www.espectador.com/1v4_contenido.php?m=&id=250419&ipag=1 http://www.espectador.com/1v4_contenido.php?m=&id=250419 http://www.espectador.com/noticias/250419/alvaro-vega-fa-creo-que-cosmo-fue-usada-por-bqb-para-evitar-una-subasta-a-la-baja-y-asi-quedar-con-las-manos-libres Same meta descr, same body with different URLs. Can I use rel canonical in the file 1v4_contenido.php that point to the friendly url? <link rel="<a class="attribute-value">canonical</a>" href="[http://www.espectador.com/noticias/250419/alvaro-vega-fa-creo-que-cosmo-fue-usada-por-bqb-para-evitar-una-subasta-a-la-baja-y-asi-quedar-con-las-manos-libres](view-source:http://www.espectador.com/noticias/250419/alvaro-vega-fa-quotcreo-que-cosmo-fue-usada-por-bqb-para-evitar-una-subasta-a-la-bajaquot-y-asi-quotquedar-con-las-manos-libresquot)"/> do I have a loop here? The rel canonical can goes in the page 1? Thanks
Technical SEO | | informatica8100 -
Does the use of sliders for text-on-page, effects SEO in any way?
The concept of using text sliders in an e-commerce site as a solution to placing SEO text above or in between product and high on ages, seems too good to be true.... or is it? How would a text slider for FAQ or other on-page text done with sliding paragraphs (similar but not this specific code- http://demo.tutorialzine.com/2010/08/dynamic-faq-jquery-yql-google-docs/faq.html) might effect text-on-page SEO. Does Google consider it hidden text? Would there be any other concerns or best practices with this design concept? faq.html
Technical SEO | | RKanfi0 -
DISQUS COMMENTS backlinks-good for seo? YES/NO?
DISQUS COMMENTS backlinks-good for seo? YES/NO? I have just started commenting on "powered by disquus" websites in the Disqus comments box and left a link to my website in the name field! Having googled whether Disqus comments backlinks are any good for seo purposes i have discovered that there is a 50/50 view on the subject with some people saying they are a "goldmine" for getting high PR backlinks and others saying they are a waste of time because googlebot cannot read Java. My own experience of commenting on Disqus powered websites is that wordpress blogs powered by disqus comments ARE INDEXED by GOOGLE and the "BACKLINK IS IN THE SOURCE OF THE PAGE" When i comment on normal websites using the Disqus comment system i have found that my Disqus comments ARE NOT indexed by Google and there IS NO BACKLINK in the page source! Has anybody got any views on whether Disqus comments backlinks are any good?
Technical SEO | | Freebetsuk2