URL Parameters
-
Hi Moz Community,
I'm working on a website that has URL parameters. After crawling the site, I've implemented canonical tags to all these URLs to prevent them from getting indexed by Google. However, today I've found out that Google has indexed plenty of URL parameters..
1-Some of these URLs has canonical tags yet they are still indexed and live.
2- Some can't be discovered through site crawling and they are result in 5xx server error.
Is there anything else that I can do (other than adding canonical tags) + how can I discover URL parameters indexed but not visible through site crawling?
Thanks in advance!
-
I'm also facing the same problem with my website pages. My Blackpods pro website pages don't show the exact permalink urls.
-
Hi there,
Thanks very much for your response. I checked the sitemap and there are no URL parameters listed - only the canonical URL listed on the sitemap.
If you have any other suggestions it'll be much appreciated.
Thank you!
-
Hi Rajesh,
Thank you for your response. I cannot share the website due to client's confidentiality but basically when I search to find a stockist {brand name}, Google lists similar URLs below on the first page. The pages are showing a list of stockists depending on the product availability:
1-website.com/find-stockist?model=10 (5xx status code)
2-website.com/find-stockist?model=11 (200 status code)
3-website.com/find-stockist?model=10 (5xx status code)
4-website.com/find-stockist?model=11 (200 status code)Thank you!
-
Hi Gaston,
Thanks very much for your time. The canonicals have implemented around a month ago and the pages are almost identical. I discovered all URL parameters without performing an advanced search.
Also, I come across the 5xx errors when I clicked indexed URL parameters on Google SERP and I cannot discover them when I crawl the site with Screaming Frog.
I'd appreciate if you have any other suggestions based on your experience!
Many thanks
-
Just so you know, if a URL results in a 5XX server error then it usually won't render your canonical tag to begin with! You might want to check your sitemap XML, to check that it's not 'undoing' your canonical tags by feeding these URLs to Google. Indexation tags must be perfectly aligned with your sitemap XML, or you are sending Google mixed messages (e.g: a URL is in sitemap XML so Google should index it, but when it is crawled it contains a canonical tag citing itself as non-canonical, which is the opposite signal)
Everything which Gaston said is right on the money
-
I think you need to show some examples.
-
Hi there,
Its important to note that canonicals are a signal. Google can obey them if its algorithm considers that those pages are actually canonicals between each other.
In my experience, this does not happen immediately, it usually takes Google some time to figure out if the canonicalization is correct. Keep in mind that pages being canonicalized HAVE TO be nearly identical and refer to the same topic.
And on the indexation part, pages can be indexed and be shown only when you search for that specific URL or using any advanced search parameter (such as site:).
More information about canonicals
- Consolidate duplicate URLs - Google Search supportRegarding the second issue, if you refer to "site crawling" as what you do with an external tool, such as Screaming Frog or Moz, you are getting 5xx errors because that tool is making to many requests, try lowering its crawl frequency. I know for a fact that Screaming Frog allows you to do that.
But, unfortunately, I don't know any other way of discovering URL parameters in bulk but using an external tool.Hope it helps,
Best luck.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Pagination new pages vs parameters
I'm working on a site that currently handles pagination like this cars-page?p=1 cars-page?p=2 In webmaster tools I can then tell ?p= designates pagination However I have a plugin I want to add to fix other seo issues, among those it adds rel="prev" rel="next" and it modifies the pagination to this cars-page-1.html cars-page2.html Notice I lost the parameter here and now each page is a different page url, pagination is no longer a parameter. I will not longer be able to specify the pagination parameter in webmaster tools. Would this confuse google as the pagination is no longer a parameter and there will now be multiple urls instead of one page with parameters? My gut says this would be bad, as I haven't seen this approach often on ecommerce site, but I wanted to see what the community thought?
Intermediate & Advanced SEO | | K-WINTER0 -
Htaccess Issue: URL not resolving properly
I am merging a niche site, tshirts.com to another site mainsite.com. I am using an htaccess file on a linux server, and the homepage of the niche site is being directed to the corresponding category page on the main site (i.e nichesite.com to mainsite.com/niche.html). Everything else is also a page to page redirect. I have something like this in the htaccess file: Redirect 301 http://tshirts.com/ http://www.mainsite.com/tshirts.html
Intermediate & Advanced SEO | | inhouseseo
Redirect 301 http://tshirts.com/blue.html http://www.lampclick.com/blue-t-shirts.html
Redirect 301 http://tshirts.com/white.html http://www.mainsite.com/white-t-shirts.html
Redirect 301 http://tshirts.com/black-tshirts.html http://www.mainsite.com/bk-t-shirts.html When I check 301 for lets say http://tshirts.com/blue.html, I get: http://tshirts.com/blue.html -** 301 Moved Permanently** http://www.mainsite.com/tshirts.htmlblue.html -** 302 Found** http:www.mainsite.com/ How do I fix this? Why is everything being appended to minsite/tshirts.html? I appreciate your help.0 -
Ecommerce SEO URL Structure Questions
| I am in the process of developing a new Magento ecommerce store. Take for instance this website is in the apparel industry and i have the following main categories. Clothing Shoes Accessories Beauty Sub categories for clothing would be: Dresses Pants jeans Tops Products would be: Kelly Maxi dresses What is the best SEO Structure for this? Main categories obviously: www.example.com/clothing Sub Categories:
Intermediate & Advanced SEO | | WayneRooney
www.example.com/clothing/dresses Or www.example.com/dresses (Zappos seem to pursue the second type) Products:
www.example.com/clothing/dresses/kelly-maxi-dresses/ Or www.example.com/kelly-maxi-dresses ? Which one would be the best way to structure your site? Also what about filters that available in category pages? Say if i were to filter by color. what would be the best URL? I am sure canonical tag is needed here. New to Ecommerce SEO so i need some guidance! |0 -
Does having a ? on the end of your URL affect your SEO?
I have some redirects that were done with at "?" at the end of the URL to include google coding (i.e. you click on an adwords link and the google coding follows the redirected link). When there is not coding to follow the link just appears as "filename.html?". Will that affect us negatively SEO-wise? Thank you.
Intermediate & Advanced SEO | | RoxBrock1 -
Can Anybody Link to my URL to Hurt SEO? Weird URL pointing at my Domaine!
Our ranking has drop since a few weeks. I did not do any major change in my site. Surfing WebMaster Tool, I found lots of new URL linking at our site: url.org linkarena.com seoprofiler.com folkd.com digitalhome.ca bustingprice.com surepurchase.com lowpricetoday.com oyax.com couponfollow.com aspringcleaning.com pamabuy.com etzone.ca How do I find if those was done intentionelly to hurt SEO? Could it be possible? Thank you, BigBlaze
Intermediate & Advanced SEO | | BigBlaze2050 -
URL structure + process for a large travel site
Hello, I am looking at the URL structure for a travel site that will want to optimise lots of locations to a wide variety of terms, so for example hotels in london
Intermediate & Advanced SEO | | onefinestay
hotels in kensington (which is in london)
five star hotels in kensington
etc I am keen to see if my thought process is correct as you see so many different URL techniques out there. Or am i overthinking it too much? Lets assume we make the page /london/ as our homepage. we would then logically link to /london/hotels to optimise specifically for 'london hotels' We then have two options in my mind for optimising for 'kensington hotels': Link to a page that keeps /london/hotels/ in its URL to maintain consistency ie A. /london/hotels/kensington or should we be linking to: B. /london/kensington/hotels/ (as it allows us to maintain a logical geo-landing page hierarchy) I feel A is good as the URL matches the search phrase 'hotels in kensington' matches the order of the search phrase, but it loses value if any links find these pages with 'kensington' in the anchor text, as they would not really strengthen the 'kensington' hub page. /london/kensington Ie: i land on the 'kensington hotels' page and want to see more about kensington, then i could go from /london/kensington/hotels
to
/london/kensington quite easily and logically in the breadcrumb. I feel B. is the best option for now.. Happy to I am only musing as i see some good sites that use option A, which effectively pushes the location (/kensington/ to the end of the URL for each additional niche sub page, ie /london/hotels/five-star-hotels/kensington/) Some of the bigger travel sites dont even use folder, they just go:
example.com/five-star-hotels-in-kensington/ Comments welcome!!! Thanks0 -
Linking to Short vs Long URL
Suppose I have a long url on an established site and created a shorter version of it so it is easier for people to enter directly and click. I 301 the short version to the long. I don't think there is much concern for people linking to the long version pages, but will there be a tendency for people to link to the short url instead of the long for the domain links? Will I not benefit as much from links to the short vs the long? Thanks.
Intermediate & Advanced SEO | | AWCthreads0 -
Best way to find all url parameters?
In reference to http://googlewebmastercentral.blogspot.com/2011/07/improved-handling-of-urls-with.html, what is the best way to find all of the parameters that need to be addressed? Thanks!
Intermediate & Advanced SEO | | nicole.healthline0