Can you 404 any forms of URL?
-
Hi seomozzers,
<colgroup><col width="548"></colgroup>
| http://ex.com/user/login?destination=comment%2Freply%2F256%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F258%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F242%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F257%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F260%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F225%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F251%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F176%23comment-form |These are duplicate content and the canonical version is: http://www.ex.com/user (login and pass page of the website)
Since there were multiple other duplicates which mostly have been resolved by 301s, I figured that all "LOGIN" URLs (above) should be 404d since they don't carry any authority and 301 those wouldn't be the best solution since "too many 301s" can slow down the website speed.
But a member of the dev team said:
"Looks like all the urls requested to '404 redirect' are actually the same page http://ex.com/user/login. The only part of the url that changes is the variables after the "?" . I don't think you can (or highly not recommended) make 404 pages display for variables in a url. "
So my question is: I am not sure what he means by that? and Is it really better to not 404 these?
Thanks
-
I believe this file is the base template for all the others. So if you put it on the main one, it should be on the others. That is why I suggested to add it and then view the code of the others to verify it is there as well.
-
Hi Dr.Peter,
Thanks for this great suggestion but could you tell me if the canonical would be a solution and if yes, could you tell me if my answer to Brent is right?
Thanks
-
Thanks Brent,
Interesting, I thought you would put the canonical version of the URL on all duplicates. Have you done it before?
-
I believe all you have to do is put this tag
On the main login page (should be the same url) and it will cover all variations.
If you aren't sure, go ahead and add it then view the source code of the pages above to see if they all have the same rel=canonical tag.
-
Hey Brent,
For Rel=canonical
I should insert this tag:
rel="canonical" href="http://www.ex.com/user/login" />
in each of these pages
| http://ex.com/user/login?destination=comment%2Freply%2F256%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F258%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F242%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F257%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F260%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F225%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F251%23comment-form |
| http://ex.com/user/login?destination=comment%2Freply%2F176%23comment-form |Is that correct?
Thanks
-
I would either go with parameter blocking or META NOINDEX this page (that's probably a bit more effective). It would be better to block it from Google than to 404 100s of variants, as you could see a spike in 404s and that can cause some problems.
Sorry, edited this - you don't want to 404 the login pages, because that's going to return a 404 for visitors as well, and the pages won't function properly. You want to just keep this away from the bots.
-
Are they actual not found pages? If not then I wouldn't 404 them, and I have never tried to 404 parameters.
Also, I would make sure you setup a rel=canonical tag. This will also help with the duplicates.
-
Drupal and Logintoboggan is maybe not configured correctly. It seems that cleanurls is not setup as well.
Maybe that will solve your issues?
-
Thanks Brent:)
But can you tell me why this wouldn't be good to 404? what is the reason behind that statement?
Thanks
-
Instead of trying to 404 those, I would suggest utilizing GWT (Google Webmaster Tools) and use their "URL Parameters". This will allow you to tell Google not to index all of them.
Log into GWT, under "Site configuration" select "URL paremters".
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Clean URL vs. Parameter URL and Using Canonical URL...That's a Mouthfull!
Hi Everyone, I a currently migrating a Magento site over to Shopify Plus and have a question about best practices for using the canonical URL. There is a competitor that I believe is not doing it the correct way, so I want to make sure my way is the better choice. With 'Vendor Pages' in Shopify, they show up looking like: https://www.campusprotein.com/collections/vendors?q=Cellucor. Not as clean. Problem is that Shopify also creates https://www.campusprotein.com/collections/cellucor. Same products, same page, just a different more clean URL. I am seeing both indexed in Google. What I want to do is basically create a canonical URL from the URL with the parameter that points to the clean URL. The two pages are very similar. The only difference is that the clean URL page has some additional content at the top of the page. I would say the two pages are 90% the same. Do you see any issue with that?
Technical SEO | | vetofunk0 -
GWT, URL Parameters, and Magento
I'm getting into the URL parameters in Google Webmaster Tools and I was just wondering if anyone that uses Magento has used this functionality to make sure filter pages aren't being indexed. Basically, I know what the different parameters (manufacturer, price, etc.) are doing to the content - narrowing. I was just wondering what you choose after you tell Google what the parameter's function is. For narrowing, it gives the following options: Which URLs with this parameter should Googlebot crawl? <label for="cup-crawl-LET_GOOGLEBOT_DECIDE">Let Googlebot decide</label> (Default) <label for="cup-crawl-EVERY_URL">Every URL</label> (the page content changes for each value) <label style="color: #5e5e5e;" for="cup-crawl-ONLY_URLS_WITH_VALUE">Only URLs with value</label> ▼(may hide content from Googlebot) <label for="cup-crawl-NO_URLS">No URLs</label> I'm not sure which one I want. Something tells me probably "No URLs", as this content isn't something a user will see unless they filter the results (and, therefore, should not come through on a search to this page). However, the page content does change for each value.I want to make sure I don't exclude the wrong thing and end up with a bunch of pages disappearing from Google.Any help with this is greatly appreciated!
Technical SEO | | Marketing.SCG0 -
/out/ URLs in GWMTs
I am recently seeing some URLs come up as 404s in GWMTs for a client. They look like this: http://client-url/out/www.linkedin.com/company/client-linkedin-name /out/client-url/sub-directory/postname/ We thought they might have something to do with the social plugins but they are all over the place and they are sometime for internal pages on the site. Anyone run into these and know why they are happening?
Technical SEO | | DragonSearch0 -
Javascript --can SE crawl?
I have a couple of nested div's. I'd like to do an onclick="location.href='http://www.example.com';" - within the outermost div so that all content within will link to one url. Can the Search Engines crawl this? Thanks!
Technical SEO | | Morris770 -
Automatic redirect to external urls
Hi all, I'm developing a dynamic qr code service.. The service works in the following way: You create an account with an associated QR CODE pointing to a url like:
Technical SEO | | raulo79
- http://domain.me/username The user can change the target of this url.. he can:
- point to an external url ( his website for example)
- point to a vCard download page
- a mobile ready webpage ( no redirection in this case)... Visiting http://domain.me/username my company logo is displayed and we redirect the visitor with a: header("Refresh: 5;URL=http://userdomain.tld"); Google is indexing many user's URLs, this is good for those users pointing to the mobile ready webpage, in this case there is no redirection, but Google is indexing many urls that redirect to an external url and I don't know how to avoid this.. I can't do an header('Location: http://www.example.com/'); because I need to display our logo after redirection.. how can I do google friendly? Sorry for my english, I hope you can undestand the problem. Best regards.
Mauro.0 -
404 errors on non-existent URLs
Hey guys and gals, First Moz Q&A for me and really looking forward to being part of the community. I hope as my first question this isn't a stupid one but I was just struggling to find any resource that dealt with the issue and am just looking for some general advice. Basically a client has raised a problem with 404 error pages - or the lack thereof- on non-existent URLs on their site; let's say for example: 'greatbeachtowels.com/beach-towels/asdfas' Obviously content never existed on this page so its not like you're saying 'hey, sorry this isn't here anymore'; its more like- 'there was never anything here in the first place'. Currently in this fictitious example typing in 'greatbeachtowels.com/beach-towels/asdfas**'** returns the same content as the 'greatbeachtowels.com/beach-towels' page which I appreciate isn't ideal. What I was wondering is how far do you take this issue- I've seen examples here on the seomoz site where you can edit the URI in a similar manner and it returns the same content as the parent page but with the alternate address. Should 404's be added across all folders on a site in a similar way? How often would this scenario be and issue particularly for internal pages two or three clicks down? I suppose unless someone linked to a page with a misspelled URL... Also would it be worth placing 301 redirects on a small number of common mis-spellings or typos e.g. 'greatbeachtowels.com/beach-towles' to the correct URLs as opposed to just 404s? Many thanks in advance.
Technical SEO | | AJ2340 -
Funky 404 error on reports
The report is showing a 404 error where a URL is being appended to the end of the address. It does not show up on the website of on the Sitemap so am wondering if I am missing something or is it a system error?
Technical SEO | | ccbseo0 -
Canonical URLs and screen scraping
So a little question here. I was looking into a module to help implement canonical URLs on a certain CMS and I came a cross a snarky comment about relative vs. absolute URLs being used. This person was insistent that relative URLs are fine and absolute URLs are only for people who don't know what they are doing. My question is, if using relative URLs, doesn't it make it easier to have your content scraped? After all, if you do get your content scraped at least it would point back to your site if using absolute URLs, right? Am I missing something or is my thinking OK on this? Any feedback is much appreciated!
Technical SEO | | friendlymachine0