Dates in URL's
-
I have an issue of duplicate content errors and duplicate page titles which is penalising my site. This has arisen because a number of URLs are suffixed by date(s) and have been spidered . In principle I do not want any url with a suffixed date to be spidered.
Eg:-
www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/06_07_13/13_07_13
http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/20_07_13/27_07_13
Only this URL should be spidered:-
http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm
I have over 10,000 of these duplicates and firstly wish to remove them on block from Google ( not one by one ) and secondly wish to amend my robots.txt file so the URL's are not spidered. I do not know the format for either.
Can anyone help please.
-
Thanks Kyle.
Particularly grateful for the Disallow format, they are the only URL's using an underscore so will work for me. WIll be checking why these are being created.
Do I need to remove them using the Removal Tool in Google, is there a format for doing this on block ?
Thanks again,
Alan
-
Hi Alan,
I would probably start by adding a disallow rule to robots.txt.
**Disallow: /*_** _may work and block all your dated URLs from being indexed but may also have adverse affects if you have any URLs containing underscores. To test whether this solution would work I would firstly implement a disallow directly on a chosen dated URL, _**Disallow: /20_07_13 **_for example, and then test whether Google has noindexed the page. GWT should tell you whether you have inadvertently blocked any other pages by doing so.
You should also be thinking about how these URLs are being created and taking actions to prevent it. Consider implementing canonical tags if you haven't already to clean up any potential duplication issues.
Cheers,
K
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What's the best Magento Community blog extension?
We are looking at FishPig's Word Press Integrations extension. has anybody used it? Possibly a dumb question, but is SEO adversely affected by the fact it's a WordPress extension on a Magento site?
On-Page Optimization | | Anne_Marie_English0 -
Canonical URL, cornerstone page and categories
If I want to have a cornerstone "page", can I substitute an actual page with a category archive of posts "page" (that contains many posts containing the target key phrase)? This way, if I make blog posts about a certain topic/ key phrase (example "beach weddings") and add a canonical URL of the category archive page to the individual posts, am I right then to assume google will see the archive page as the cornerstone page (and thereby won't see the individual posts with the same key phrase as competing)?
On-Page Optimization | | stephanwb0 -
URL for location pages
Hello all We would like to create clean, easy URLs for our large list of Location pages. If there are a few URLs for each of the pages, am I right when I'm saying we would like this to be the canonical? Right now we would like the URL to be: For example
On-Page Optimization | | Ferguson
Domain.com/locations/Columbus I have found some instances where there might be 2,3 or more locations in the same city,zip. My conclusion for these would be: adding their Branch id's on to the URL
Domain.com/locations/Columbus/0304 Is this an okay approach? We are unsure if the URL should have city,State,zip for SEO purposes?
The pages will have all of this info in it's content
BUT what would be best for SEO and ranking for a given location? Thank you for any info!0 -
Seo'ing Sub domains for images
We are currently adding some performance improvement to our websites, to improve user experience. One of the things we are looking at is splitting of images over several sub-domains, to increase the number of images that can be downloaded at the same. We have seen that using key phrases within image names has an improvement in rankings. So, the question is should we create sub-domains as key-phrase.example.co.uk or as i1.example.co.uk? K
On-Page Optimization | | soltec0 -
How many urls per page is to many
I know it used to be 100 urls per page, but recently Matt cutts has said that they can count a lot more now. I was wonder what you guys thought was how many was to many per page?
On-Page Optimization | | Gordian0 -
What's the Best Way to Hide Redirects from Search Engines?
Hey everybody, I like to use php redirects for affiliate links so they look better. I keep them all in the same directory. I read recently that these may hurt SEO. A couple quick questions: Is this the best way to redirect affiliate links? Should I simply block the directory in robots.txt? Any other suggestions from you SEO guyses and galz? Thanks! Jared
On-Page Optimization | | JaredB0 -
Canonical URL's - Fixed but still negatively impacted
I recently noticed that our canonical url's were not set up correctly. The incorrect setup predates me but it could have been in place for close to a year, maybe a bit more. Each of the url's had a "sortby" parameter on all of them. I had our platform provider make the fix and now everything is as it should be. I do see issues caused by this in Google Webmaster, for instance in the HTML suggestions it's telling me that pages have duplicate title tags when in fact this is the same page but with a variety of url parameters at the end of the url. To me this just highlights that there is a problem and we are being negatively impacted by the previous implementation. My question is has anyone been in this situation? Is there any way to flush this out or push Google to relook at this? Or is this a sit and be patient situation. I'm also slightly curious if Google will at some point look and see that the canonical urls were changed and then throw up a red flag even though they are finally the way they should be. Any feedback is appreciated. Thanks,
On-Page Optimization | | dgmiles
Dave0 -
2 URLs, same content, 1 with keywords. Does this hurt me?
I'm in the process of adding some new features to our site and have a question about our URLs. Most of our URLs consist of either sitename.com/contentname or sitename.com/content/contentid I'm in the process of building a directory to those pages. The directory has a number of filters which will ultimately point to the destination page. sitename.com/filter1/filter2/contentid or sitename.com/filter1/filter2/contentname The destinations will have references. From an SEO perspective, I would think I want the filter1/filter2 version of the link indexed since this will add keywords that someone might search for. However, since the filters are dynamic, if someone just searches for contentname I would want to have sitename.com/contentname returned in the search results. Do I get any SEO benefit out of building those filter links as described if they are not the canonical links?
On-Page Optimization | | JoeCotellese810