Best solution to get mass URl's out the SE's index
-
Hi,
I've got an issue where our web developers have made a mistake on our website by messing up some URL's . Because our site works dynamically IE the URL's generated on a page are relevant to the current URL it ment the problem URL linked out to more problem URL's - effectively replicating an entire website directory under problem URL's - this has caused tens of thousands of URL's in SE's indexes which shouldn't be there.
So say for example the problem URL's are like
www.mysite.com/incorrect-directory/folder1/page1/
It seems I can correct this by doing the following:
1/. Use Robots.txt to disallow access to /incorrect-directory/*
2/. 301 the urls like this:
www.mysite.com/incorrect-directory/folder1/page1/
301 to:
www.mysite.com/correct-directory/folder1/page1/3/. 301 URL's to the root correct directory like this:
www.mysite.com/incorrect-directory/folder1/page1/
www.mysite.com/incorrect-directory/folder1/page2/
www.mysite.com/incorrect-directory/folder2/301 to:
www.mysite.com/correct-directory/Which method do you think is the best solution? - I doubt there is any link juice benifit from 301'ing URL's as there shouldn't be any external links pointing to the wrong URL's.
-
Cheers Ryan.
-
Option 2 is preferred.
You definitely do not want to use the robots.txt method. In general, avoid using robots.txt unless there are no other options.
Whenever your site's visitors have a link to an invalid URL, 301 them to the correct URL if you have the content they are seeking. It creates the best user experience and the best SEO results.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Getting Google to index our sitemap
Hi, We have a sitemap on AWS that is retrievable via a url that looks like ours http://sitemap.shipindex.org/sitemap.xml. We have notified Google it exists and it found our 700k urls (we are a database of ship citations with unique urls). However, it will not index them. It has been weeks and nothing. The weird part is that it did do some of them before, it said so, about 26k. Then it said 0. Now that I have redone the sitemap, I can't get google to look at it and I have no idea why. This is really important to us, as we want not just general keywords to find our front page, but we also want specific ship names to show links to us in results. Does anyone have any clues as to how to get Google's attention and index our sitemap? Or even just crawl more of our site? It has done 35k pages crawling, but stopped.
Intermediate & Advanced SEO | | shipindex0 -
How long will old pages stay in Google's cache index. We have a new site that is two months old but we are seeing old pages even though we used 301 redirects.
Two months ago we launched a new website (same domain) and implemented 301 re-directs for all of the pages. Two months later we are still seeing old pages in Google's cache index. So how long should I tell the client this should take for them all to be removed in search?
Intermediate & Advanced SEO | | Liamis0 -
Can 'follow' rather than 'nofollow' links be damaging partner's SEO
Hey guys and happy Monday! We run a content rich website, 12+ years old, focused on travel in a specific region, and advertisers pay for banners/content etc alongside editorial. We have never used 'nofollow' website links as they're no explicitly paid for by clients, but a partner has asked us to make all links to them 'nofollow' as they have stated the way we currently link is damaging their SEO. Could this be true in any way? I'm only assuming it would adversely affect them if our website was peanalized by Google for 'selling links', which we're not. Perhaps they're just keen to follow best practice for fear of being seen to be buying links. FYI we now plan to change to more full use of 'nofollow', but I'm trying to work out what the client is refering to without seeming ill-informed on the subject! Thank you for any advice 🙂
Intermediate & Advanced SEO | | SEO_Jim0 -
Question about getting domain name re-indexed
I recently swapped my domain from www.davescomputers.com to www.computer-help.com . Originally www.computer-help.com was 301 re-directing to www.davescomputers.com ...however my long term goal is to eventually rebrand my business so I decided to utilize the other domain by swapping the main domain. Is consistant blogging the best way to get Google to re-index the entire website? My focus has been quality posts and sharing them with vairus social profiles I created.
Intermediate & Advanced SEO | | DavidMolnar0 -
Why Google isn't indexing my images?
Hello, on my fairly new website Worthminer.com I am noticing that Google is not indexing images from my sitemap. Already 560 images submitted and Google indexed only 3 of them. Altough there is more images indexed they are not indexing any new images, and I have no idea why. Posts, categories and other urls are indexing just fine, but images not. I am using Wordpress and for sitemaps Wordpress SEO by yoast. Am I missing something here? Why Google won't index my images? Thanks, I appreciate any help, David xv1GtwK.jpg
Intermediate & Advanced SEO | | Worthminer1 -
Getting a Facebox Item De-Indexed
Hello all, Select pages on a website I manage utilizes Facebox lightbox elements for additional information. These Faceboxes have been indexed as their own pages by Google. Unfortunately, when they were created, there is no call to action or navigation elements or anything. So I would prefer that Google not index them since it is a pretty horrible user experience if you navigate to one of these Faceboxes directly. I have tossed up no-index tags on each about three weeks ago, but they are still being indexed at present. Is there any tips or tricks that anyone has to handle this scenario or any ideas that I am not thinking of outside of just no-indexing them? Thanks in advance!
Intermediate & Advanced SEO | | ClayPotCreative0 -
I'm afraid I may have messed up my site's organization
So I recently started working on an existing site for a company, and I'm afraid I may have done something to lose some backlinks. So to start off, say the website is www.domain.net and when I arrived domain.net and www.domain.net showed up as two separate sites so I changed my web.config file to direct all domain.net to www.domain.net The homepage was called default.asp, and I wanted the homepage to always show up as www.domain.net instead of www.domain.net/default.asp. Of course they both showed the same thing but I couldn't figure it out. So I removed www.domain.net/default.asp from indexing and changed the my internal links to the homepage to point at www.domain.net instead of simply pointing at the file default.asp. So now www.domain.net/default.asp still brings up the page, but I want it to revert to www.domain.net. I'm also a little worried because I noticed that one of my incoming links points at www.domain.net/default.asp and it doesn't get passed along to www.domain.net and I think i may have damaged my sites SEO I guess this is a very complicated and roundabout way of saying this, but how can I get www.domain.net/default.asp to take you to www.domain.net
Intermediate & Advanced SEO | | bcrabill0 -
What is the best canonical url to use for a product page?
I just helped a client redesign and launch a new website for their organic skin care company (www.hylunia.com). The site is built in Magento which by default creates MANY urls for each product. Which of these two do you think would be the best to use as the canonical version? http://www.hylunia.com/pure-hyaluronic-acid-solution
Intermediate & Advanced SEO | | danielmoss
or http://www.hylunia.com/products/face-care/facial-moisturizers/pure-hyaluronic-acid-solution ? I'm leaning on the latter, because it makes sense to me to have the breadcrumbs match the url string, and also it seems having more keywords in the url would help. However, it's obviously a very long url, and there might be some benefits to using the shorter version that I'm not aware of. Thanks in advance for sharing your thoughts. Best, Daniel0