Test contet/pages indexed by search engines
-
During the web development stages of our Joomla CMS website, we have managed to get our site indexed for totally irrelevant test pages mainly to do with Joomla and some other equally irrelevant test content. How damaging is this to our domain from an SEO prospective and is there something we can do about it?
When we do a site:domain.com search we see hundreds of testpages with test/irrelevant meta tags etc.
-
Search engines regularly recrawl every website and will update their information based on changes you make to your site. It is a natural part of the internet. The "site under construction" information is not harmful, but in the future should be blocked from indexing.
-
Thankfully its only test urls that have been indexed by Google only.
However all 3 major engines have indexed our domain against "Site under construction" page with untitled/incomplete tags.
Is this harmful or will this be overwritten when we launch properly and get our site indexed?
-
When you begin developing a site, you should use the robots.txt file to block all search engine access to the site. This is one of the few times where a robots.txt file is very useful.
With respect to fixing the issue, it depends on whether the URLs will be used on the live site, how long it will be until your site launched, and whether unique URLs such as /testing were used or you are working with the same URLs which will exist on the live site.
If your site is still in testing and it will remain in testing for 30+ days, you could add the noindex tag sitewide. Once all the pages were removed from the index, you can then add the robots.txt file. Be careful not to adjust the robots.txt file prior to the pages being removed as the search engines wont be able to see the noindex tag.
You did not mention which search engine indexed your pages. If you are working with Google and the URLs will not exist on the live site, you could use the Google Removal Tool. This is really overkill and should not be necessary, but if the site owner is paranoid about the test pages causing damage to SEO you can take this approach. Any URL removed in this manner cannot be re-added to the index for 90 days.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Have you ever seen or experienced a page indexed which is actually from a website which is blocked by robots.txt?
Hi all, We use robots file and meta robots tags for blocking website or website pages to block bots from crawling. Mostly robots.txt will be used for website and expect all the pages to not getting indexed. But there is a condition here that any page from website can be indexed by Google even the site is blocked from robots.txt; because crawler may find the page link somewhere on internet as stated here at last paragraph. I wonder if this really the case where some webpages have got indexed. And even we use meta tags at page level; do we need to block from robots.txt file? Can we use both techniques at a time? Thanks
Algorithm Updates | | vtmoz0 -
Why is Page Authority dropping?
Hi I'm trying to review pages which have previously ranked, but in March have dropped out completely. Some of these pages I can see have dropped to having a Page Authority of 1, we haven't changed anything on these pages, so is there a reason why the authority has dropped? These pages only had around 8 - 10 Page Authority to begin with. I'm trying to identify why we have lost keywords, and if it has anything to do with the Google Updates in March Here are examples of the pages with drops: http://www.key.co.uk/en/key/heavy-duty-shelving-1830x1830mm-blue-orange
Algorithm Updates | | BeckyKey
http://www.key.co.uk/en/key/metal-feet-for-heavy-duty-steel-shelving
http://www.key.co.uk/en/key/health-and-safety-law-poster-a2 Thank you!0 -
What is the new / current google algorithm update targeting and how to adapt / fix SEO approach accordingly ?
Over the past c. 3 weeks our average Google SERP rankings for a site have been as follows : week 1 : average 3rd in SERP week 2 : average 1st in SERP week 3 : average 7th in SERP Reading on sites including Moz, SEO Roundtable and looking at Moz's mozcast tool it seems there is allot of change going on with Google's algorithm at the moment. Is there a general consensus as to what the new algorithm update is targeting so that we can adapt our site / marketing accordingly ? Or is it too early to adapt ? When Google rolls out an update like this is it common to see fluctuations as they try out things or if we are currently down in the rankings should we take action now. I know there a tools that look at your google analytics data an overlay a list of major google updates such a penguin and panda, but from what ive read this is a generic algorithm change rather than a defined named update.
Algorithm Updates | | jpeg800 -
Images not getting indexed in google image search :( " site: hdwallpaperzones.com " )
hi as i have mentioned in title.. my website images are not getting indexed in google image search engine.. out of 360 images only 5 got indexed from 3 days.. please help me out.. thanks
Algorithm Updates | | toxicpls0 -
Google is really NOT SAYING IN "HOW SEARCH WORKS” ?
Hi All SEOmoz members and team, As I was reading this, is it true that Google does this . Simply, I don't think so, I haven't experienced any of such what is being talked [http://www.fairsearch.org/search-manipulation/what-google-isnt-saying-in-how-search-works/ C](http://www.fairsearch.org/search-manipulation/what-google-isnt-saying-in-how-search-works/ "http://www.fairsearch.org/search-manipulation/what-google-isnt-saying-in-how-search-works/")ome on, let us discuss the real thing about Google. Teginder Ravi
Algorithm Updates | | Futura0 -
Does Search Volume Directly Effect Organic Search Result Rankings?
For example, if 20,000 people searched for "seomoz toasters," do you think a page on seomoz.org that mentioned toasters would begin to rank well for the query "toasters"?
Algorithm Updates | | tatermarketing0 -
Phantom Indexed: 301 Redirected Old URL Shows in Google Search Result!
Today, I have read about Phantom Indexed in Google search result. Because, I was searching about 301 redirect due to indexing of 301 redirected old URLs in Google search result rather than new landing pages. I've added my comment on jennita's blog post about 301 redirect. I would like to paste similar question over here! I have 301 redirected following 3 domains to new website... http://www.lampslightingandmore.com/ To http://www.vistastores.com/table-lamps http://www.vistapatioumbrellas.com/ To http://www.vistastores.com/patio-umbrellas http://www.spiderofficechairs.com/ To http://www.vistastores.com/office-chairs I have done it before 3 months but, Google still shows me home page URL in search result rather than new landing page. You can check following search results to know more about it. For LampsLightingandMore ~ On second or third page::: For VistaPatioUmbrellas ~ On second or third page::: For SpiderOfficeChairs ~ On Second or third page::: I come to know about Phantom Indexed after raised my comment over there. So, why should not start discussion on it. Because, It's all about branding and who'll love to hang old address in front of new home.
Algorithm Updates | | CommercePundit0 -
Yahoo/Bing cache date went back in time
Within 12 hours of submitting a new site to Yahoo/Bing webmasters it was ranking #3 for the primary homepage search term and in the top 5 for about a dozen other. On 7/23 the rankings were steady or climbing with the most recent cache date of 7/21. Now the site only comes up when searching for the domain name with a cache date of 7/11. I launched the site about 14 days ago so I am not expecting results yet but I had never seen this happen so I am just curious if anyone else had.
Algorithm Updates | | jafabel0
