Welcome to the Q&A Forum

Hi Kate,

Thank you! I followed you on Twitter, my user name is @curetuvana

Hi Kate,

Thank you for your time to respond! Is there any way that I can contact you directly?

By the way, I've tried crawling after changing configurations as 'rendering Javascript' however, it still didn't discover product URLs.

Thank you!

Hi Moz Community,

We have a headless ecom (Magento) client that I'm trying to crawl the site.  During the crawl, the tool (Screaming Frog) cannot discover the sub-category URLs in the main navigation when I start crawling via homepage.

Similarly, when I start crawling with one of the sub-category page, it doesn't crawl any of the product URLs on the sub-category page itself.

When I inspect product and sub-cat URLs through Search Console, they seem as indexed and if I view how Googlebot rendered the sub-category page, I can see the product URLs on the sub-cat page too.

If you have any idea what's the issue with Screaming Frog and would like to help me out, I'd be so grateful!

Thanks in advance

Hi there,

Thanks very much for your response. I checked the sitemap and there are no URL parameters listed - only the canonical URL listed on the sitemap.

If you have any other suggestions it'll be much appreciated.

Thank you!

Hi Rajesh,

Thank you for your response. I cannot share the website due to client's confidentiality but basically when I search to find a stockist {brand name}, Google lists similar URLs below on the first page. The pages are showing a list of stockists depending on the product availability:

1-website.com/find-stockist?model=10 (5xx status code)
2-website.com/find-stockist?model=11 (200 status code)
3-website.com/find-stockist?model=10 (5xx status code)
4-website.com/find-stockist?model=11 (200 status code)

Thank you!

Hi Gaston,

Thanks very much for your time. The canonicals have implemented around a month ago and the pages are almost identical. I discovered all URL parameters without performing an advanced search.

Also, I come across the 5xx errors when I clicked indexed URL parameters on Google SERP and I cannot discover them when I crawl the site with Screaming Frog.

I'd appreciate if you have any other suggestions based on your experience!

Many thanks

Hi Moz Community,

I'm working on a website that has URL parameters. After crawling the site, I've implemented canonical tags to all these URLs to prevent them from getting indexed by Google. However, today I've found out that Google has indexed plenty of URL parameters..

1-Some of these URLs has canonical tags yet they are still indexed and live.

2- Some can't be discovered through site crawling and they are result in 5xx server error.

Is there anything else that I can do (other than adding canonical tags) + how can I discover URL parameters indexed but not visible through site crawling?

Thanks in advance!

Hi Joe,

Thanks for your reply and your suggestions. I don't think that we are allowed to choose the first option -creating multiple GMB profiles for a business as there is only one business but supplying different products. I believe this can cause a penalty.

I'm going to confirm the second option with the GMB support and update here as well so maybe can help any other person facing the same issue.

Have a nice week!

Hi friends,

I'm trying to work out what would the practice be for a business who is operating in different categories in terms of displaying those categories in Google My Business account. We have a client who is supplying both catering and cleaning products (both categories are core). In this case, listing those two categories in GMB would be alright or should I expect a negative impact on results related to both categories as we have chosen multiple categories?

Any advice would be appreciated greatly!

Hi Joe,

Thanks very much for your reply. This is the extension that we've purchased earlier for this purpose and going to refund it as it turned out that custom canonical URLs can only be implemented on the product pages but not on the category pages with this extension...

Hi folks,

I'm looking for an SEO extension for Magento 2 that enables me to add Custom Canonical URLs on any page (especially on category pages).

Any advice would be appreciated greatly!

Hi friends,

I'm trying to work out what would the practice be for a business who is operating in different categories in terms of displaying those categories in Google My Business account. We have a client who is supplying both catering and cleaning products (both categories are core). In this case, listing those two categories in GMB would be alright or should I expect a negative impact on results related to both categories as we have chosen multiple categories?

Any advice would be appreciated greatly!