Google has indexed a lot of test pages/junk from the development days.
-
With hind site I understand that this could have been avoided if robots.txt was configured properly.
My website is www.clearvisas.com, and is indexed with both the www subdomain and with out.
When I run site:clearvisas.com in Google I get 1,330 - All junk from the development days.
But when I run site:www.clearvisas.com in Google I get 66 - these results all post development and more in line with what I wanted to be indexed.
Will 1,330 junk pages hurt my seo?
Is it possible to de-index them and should I?
If the answer is yes to any of the questions how should I proceed?
Kind regards,
Fuad
-
Thanks Ryan.
-
It's impossible to say conclusively without examining your site and the content; however, since you refer to them as "junk" pages, it is likely they should best be removed to protect your other pages.
-
Thanks Ryan.
Are the un-wanted/irrelevant pages likely to affect my organic seo?
-
Thanks for your view David, its much appreciated. Thanks, Fuad
-
I would suggest following option 3 from David's recommendations.
Simply add the "noindex" tag to the pages you want removed from Google. The pages will then be removed the next time they are crawled.
You are correct the issue could have been avoided by blocking the site during development, which is a recommended practice; however, it is recommended to minimize entries in the robots.txt file of a live site. You can add the pages in robots.txt and Google can still index them.
The above applies if you feel the need to keep the pages around. If you no longer need those pages, removing them and providing a 410 error (GONE) would be the best approach.
-
Go to Google Webmaster Tools => Optimization => Remove URLS
In order for Google to remove the URL, you will need to do 1 of the following:
1. Block it with robots.txt, but it sounds like it's too late for that.
2. If you removed the old development content, make sure that the old content's URL produces a 404 or 410 status code.
3. Block the content with a Meta noncontent tag.
In my opinion, option 2 is the easiest since you should have a 404 page anyway.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can 'Jump link'/'Anchor tag' urls rank in Google for keywords?
E.g. www.website.com/page/#keyword-anchor-text Where the part after the # is a section of the page you can jump to, and the title of that section is a secondary keyword you want the page to rank for?
Algorithm Updates | | rwat0 -
Google & Tabbed Content
Hi I wondered if anyone had a case study or more info on how Google treats content under tabs? We have an ecommerce site & I know it is common to put product content under tabs, but will Google ignore this? Becky
Algorithm Updates | | BeckyKey1 -
Ecommerce SEO: Is it bad to link to product/category pages directly from content pages?
Hi ! In Moz' Whiteboard friday video Headline Writing and Title Tag SEO in a Clickbait World, Rand is talking about (among other things) best practices related to linking between search, clickbait and conversion pages. For a client of ours, a cosmetics and make-up retailer, we are planning to build content pages around related keywords, for example video, pictures and text about make-up and fashion in order to best target and capture search traffic related to make-up that is prevalent earlier in the costumer journey. Among other things, we plan to use these content pages to link directly to some of the products. For example a content piece about how to achieve full lashes will to link to particular mascaras and/or the mascara category) Things is, in the Whiteboard video Rand Says:
Algorithm Updates | | Inevo
_"..So your click-bait piece, a lot of times with click-bait pieces they're going to perform worse if you go over and try and link directly to your conversion page, because it looks like you're trying to sell people something. That's not what plays on Facebook, on Twitter, on social media in general. What plays is, "Hey, this is just entertainment, and I can just visit this piece and it's fun and funny and interesting." _ Does this mean linking directly to products pages (or category pages) from content pages is bad? Will Google think that, since we are also trying to sell something with the same piece of content, we do not deserve to rank that well on the content, and won't be considered that relevant for a search query where people are looking for make-up tips and make-up guides? Also.. is there any difference between linking from content to categories vs. products? ..I mean, a category page is not a conversion page the same way a products page is. Looking forward to your answers 🙂0 -
Google Mobile Algorithm update
Hi there, On April the 21st Google seems to going to update their Mobile algorithm. I have a few questions about this one. Our current mobile website is very mobile friendly. We block all mobile pages with a noindex, so the desktop pages have been indexed on mobile devices. We use a redirect from desktop page to mobile page when someone hits a result on a mobile device. My gut tells me this is not April 21st-proof so I'm thinking about an update to make this whole thing adaptive. By making the thing adaptive, our mobile pages will be indexed instead of the desktop pages. Two questions: Will Google treat the mobile page as a 100% different page than the desktop page? Or will it match those two because everything will tell Google those belong together. In other words: will the mobile page start with a zero authority and will pages lose good organic positions because of authority or not? Which ranking factor will be stronger after April 21st for mobile pages: page authority or mobile friendliness? In other words: is it worth ignoring the 21 April update because the authority of the desktop pages is more important than making every page super mobile friendly? Hope to get some good advice! Marcel
Algorithm Updates | | MarcelMoz0 -
Wordpress Canonical Tag Pointing to Same Page
So I noticed on a few of my clients wordpress tags (via moz) that there are canonical tags on URLs, pointing to that same URL. What is the point of that, and is it harming the website? Is this being done automatically via a plugin? Should I remove the canonical tags or leave as is?
Algorithm Updates | | WebServiceConsulting.com0 -
Recovered from penguin/panda but which one?
So the good news is that for the first time since April 24th, one of our websites is back in the search results as of around December 12 but I am still unsure as whether it was panda or penguin (or both) that was impacting the site?? Note this was not a manual penalty. I diagnosed it as a penguin issue (drop on April 24th, aggressive on-page optimisation, around 10% of links from spammy directories like addyourfreelinks.com with anchor text built by a questionable agency), but on further advice it was thought that panda was also an issue because it is a hotel microsite so there was duplication with our own brand site and across third party travel sites and there were a number of pages with bare content. I figured it was a good time to clean everything up to address both. Here is a summary of actions taken: submitted disavow file on October 24th with all questionable links including actions taken and comments. Since then I have cleaned up some content so it is less aggressively targeting certain keywords. Amended several third party listings with duplicate content No follow,indexed pages that were directly duplicated with our brand site and over the last month have built a few good quality links. Cleaned up 404's in webmaster tools over the last week I have searched to see if there were any algorithm updates around December 12 but cannot find any mentions. Thoughts?
Algorithm Updates | | jay.raman0 -
Removing secure subdomain from google index
we've noticed over the last few months that Google is not honoring our main website's robots.txt file. We have added rules to disallow secure pages such as: Disallow: /login.cgis Disallow: /logout.cgis Disallow: /password.cgis Disallow: /customer/* We have noticed that google is crawling these secure pages and then duplicating our complete ecommerce website across our secure subdomain in the google index (duplicate content) https://secure.domain.com/etc. Our webmaster recently implemented a specific robots.txt file for the secure subdomain disallow all however, these duplicated secure pages remain in the index. User-agent: *
Algorithm Updates | | marketing_zoovy.com
Disallow: / My question is should i request Google to remove these secure urls through Google Webmaster Tools? If so, is there any potential risk to my main ecommerce website? We have 8,700 pages currently indexed into google and would not want to risk any ill effects to our website. How would I submit this request in the URL Removal tools specifically? would inputting https://secure.domain.com/ cover all of the urls? We do not want any secure pages being indexed to the index and all secure pages are served on the secure.domain example. Please private message me for specific details if you'd like to see an example. Thank you,0 -
Google place page Images
Is there any real difference in uploading an images directly to your google places page or linking an image from another site? I have heard that you get better results if you upload a photo to photo bucket then to insider pages then post that link to your google places page. To me it just seems a bit odd to do things this way. I get that it's suppose to give you more back links however I don't think it would necessarily be relevant or useful for the user. Any thoughts??
Algorithm Updates | | christinarule0