Test contet/pages indexed by search engines
-
During the web development stages of our Joomla CMS website, we have managed to get our site indexed for totally irrelevant test pages mainly to do with Joomla and some other equally irrelevant test content. How damaging is this to our domain from an SEO prospective and is there something we can do about it?
When we do a site:domain.com search we see hundreds of testpages with test/irrelevant meta tags etc.
-
Search engines regularly recrawl every website and will update their information based on changes you make to your site. It is a natural part of the internet. The "site under construction" information is not harmful, but in the future should be blocked from indexing.
-
Thankfully its only test urls that have been indexed by Google only.
However all 3 major engines have indexed our domain against "Site under construction" page with untitled/incomplete tags.
Is this harmful or will this be overwritten when we launch properly and get our site indexed?
-
When you begin developing a site, you should use the robots.txt file to block all search engine access to the site. This is one of the few times where a robots.txt file is very useful.
With respect to fixing the issue, it depends on whether the URLs will be used on the live site, how long it will be until your site launched, and whether unique URLs such as /testing were used or you are working with the same URLs which will exist on the live site.
If your site is still in testing and it will remain in testing for 30+ days, you could add the noindex tag sitewide. Once all the pages were removed from the index, you can then add the robots.txt file. Be careful not to adjust the robots.txt file prior to the pages being removed as the search engines wont be able to see the noindex tag.
You did not mention which search engine indexed your pages. If you are working with Google and the URLs will not exist on the live site, you could use the Google Removal Tool. This is really overkill and should not be necessary, but if the site owner is paranoid about the test pages causing damage to SEO you can take this approach. Any URL removed in this manner cannot be re-added to the index for 90 days.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ecommerce SEO: Is it bad to link to product/category pages directly from content pages?
Hi ! In Moz' Whiteboard friday video Headline Writing and Title Tag SEO in a Clickbait World, Rand is talking about (among other things) best practices related to linking between search, clickbait and conversion pages. For a client of ours, a cosmetics and make-up retailer, we are planning to build content pages around related keywords, for example video, pictures and text about make-up and fashion in order to best target and capture search traffic related to make-up that is prevalent earlier in the costumer journey. Among other things, we plan to use these content pages to link directly to some of the products. For example a content piece about how to achieve full lashes will to link to particular mascaras and/or the mascara category) Things is, in the Whiteboard video Rand Says:
Algorithm Updates | | Inevo
_"..So your click-bait piece, a lot of times with click-bait pieces they're going to perform worse if you go over and try and link directly to your conversion page, because it looks like you're trying to sell people something. That's not what plays on Facebook, on Twitter, on social media in general. What plays is, "Hey, this is just entertainment, and I can just visit this piece and it's fun and funny and interesting." _ Does this mean linking directly to products pages (or category pages) from content pages is bad? Will Google think that, since we are also trying to sell something with the same piece of content, we do not deserve to rank that well on the content, and won't be considered that relevant for a search query where people are looking for make-up tips and make-up guides? Also.. is there any difference between linking from content to categories vs. products? ..I mean, a category page is not a conversion page the same way a products page is. Looking forward to your answers 🙂0 -
Doorway Algorithm Update Affecting Location Based Pages?
Hi all, I read this article concerning the doorway algorithm update - http://searchengineland.com/google-to-launch-new-doorway-page-penalty-algorithm-216974 This quote is what got my attention: "How do you know if your web pages are classified as a “doorway page?” Google said asked yourself these questions: Is the purpose to optimize for search engines and funnel visitors into the actual usable or relevant portion of your site, or are they an integral part of your site’s user experience? Are the pages intended to rank on generic terms yet the content presented on the page is very specific? Do the pages duplicate useful aggregations of items (locations, products, etc.) that already exist on the site for the purpose of capturing more search traffic? Are these pages made solely for drawing affiliate traffic and sending users along without creating unique value in content or functionality? Do these pages exist as an “island?” Are they difficult or impossible to navigate to from other parts of your site? Are links to such pages from other pages within the site or network of sites created just for search engines?" We utilize location based pages for ourselves and a few clients too. **Example Case: ** -We attempt to rank for "keyword city/state" - "keyword city/state" - "keyword city/state" The keywords will often be the same such as "AC Repair" or "Physical Therapy" etc. with city / state combination such as "Tulsa, OK" "Seattle, WA" etc. The goal is to rank locally for those terms (NAP is applicable in some circumstances). Does the above case classify as a Doorway page? According to that definition, it does. However, this is a business that services that area. Some don't have physical address there but they do service that area (whether it be AC Repair or Website Design). Please advise me as to what a doorway page is exactly & if my practice is in-line. Thanks, Cole
Algorithm Updates | | ColeLusby0 -
What is the impact of HTTP/2 on SEO ?
I think it's good for the user experience and speeds up websites, especially if your site has a lot of requests. But i'm not sure if there are other side effects, and if there's an impact on SEO or technical configuration. Most of my websites are built with Wordpress, some with Joomla.
Algorithm Updates | | Croco_Web_Solutions1 -
Ahrefs - What Causes a Drastic Loss in Referring Pages?
While I was doing research on UK Flower companies I noticed that one particular domain had great rankings (top 3), but has slid quite a bit down to page two. After investigating further I noticed that they had a drastic loss of referring pages, but an increase in total referring domains. See this screenshot from ahrefs. I took a look at their historical rankings (got them from the original SEO provider's portfolio) and compared it to the Wayback Machine. There did not seem to be any drastic changes in the site structure. My question is what would cause such a dramatic loss in total referring pages while showing a dramatic increase in referring domains? It appears that the SEO company was trying rebound from the loss of links though. Any thoughts on why this might happen? 56VD5jD
Algorithm Updates | | AaronHenry0 -
Ecommerce good/bad? Showing product description on sub/category page?
Hi Mozers, I have a ecommerce furniture website, and I have been wondering for some time if showing the product descriptions on the sub/category page helps the website. If there is more content displayed on the subcategory, it should be more relevant, right? OR does it not matter, as it is duplicate content from the product page. I think showing the product descriptions on non-product pages is hurting my design/flow, but i worry that if I am to hide product content on sub/category pages my traffic will be hurt. Despite my searches I have not found an answer yet. Please take a look at my site and share your thoughts: http://www.ecustomfinishes.com/ Chris 27eVz
Algorithm Updates | | longdenc_gmail.com0 -
Google dance/over optimized/paranoid?
Hi guys, hope your all OK and thanks in advance for taking a nosey at this. OK where to start - my rankings for the last 12 months have progressively improved every week, usually of the 300 KWs i track the last few months has seen approx 70 up/70down per week, but the improvements usually outweigh the declines. This week I saw a sudden drop though - 35 improvements and 112 declines. The strange thing was though, the improvements came on the more competitive KWs, and the less competitive words I haven't done much or any back linking for dropped. Seems silly me asking this question when I run that through my head ofcouse KWs you don;t work on will drop like flies? It should be plainly obvious those words would drop off but all have been improving on there own slowly over the last 6/7 months. Now if this was a penalty (nothing showing in webmaster tools) I would have expected it to come through on my KWs I have over done the backlinking for, but these are the 1's that improved. So is it just the Google Dance? I normally see some words such as the big 1 we target DJ Equipment go from position 13 - 24 can change hourly sometimes! Could it just be quite a few have dropped all at once and will pop back up this week? Also if anyone could give us any pointers in general on where you think we should be taking our SEO it would be much appreciated. I know we have been a little lazy with our backlinking and could do with some much better/ industry related websites linking to us, and there are title tags/metas on product page that need sorting.. aside these couple of issue's? DJs Only
Algorithm Updates | | allan-chris0 -
Google UK search volumes
If a user searches using Google.com but is based in the UK, is it classed as a Google UK search or a Google US search in terms of monthly search volumes? Most of my clients are targeting UK consumers and often rank well on Google UK but outside the top fifty for Google US. I have mentioned that that is fine unless a client happens to use google.com. Am I talking rubbish?
Algorithm Updates | | Switch_Digital0
