Test contet/pages indexed by search engines
-
During the web development stages of our Joomla CMS website, we have managed to get our site indexed for totally irrelevant test pages mainly to do with Joomla and some other equally irrelevant test content. How damaging is this to our domain from an SEO prospective and is there something we can do about it?
When we do a site:domain.com search we see hundreds of testpages with test/irrelevant meta tags etc.
-
Search engines regularly recrawl every website and will update their information based on changes you make to your site. It is a natural part of the internet. The "site under construction" information is not harmful, but in the future should be blocked from indexing.
-
Thankfully its only test urls that have been indexed by Google only.
However all 3 major engines have indexed our domain against "Site under construction" page with untitled/incomplete tags.
Is this harmful or will this be overwritten when we launch properly and get our site indexed?
-
When you begin developing a site, you should use the robots.txt file to block all search engine access to the site. This is one of the few times where a robots.txt file is very useful.
With respect to fixing the issue, it depends on whether the URLs will be used on the live site, how long it will be until your site launched, and whether unique URLs such as /testing were used or you are working with the same URLs which will exist on the live site.
If your site is still in testing and it will remain in testing for 30+ days, you could add the noindex tag sitewide. Once all the pages were removed from the index, you can then add the robots.txt file. Be careful not to adjust the robots.txt file prior to the pages being removed as the search engines wont be able to see the noindex tag.
You did not mention which search engine indexed your pages. If you are working with Google and the URLs will not exist on the live site, you could use the Google Removal Tool. This is really overkill and should not be necessary, but if the site owner is paranoid about the test pages causing damage to SEO you can take this approach. Any URL removed in this manner cannot be re-added to the index for 90 days.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Rank on Moz compared to Ahrefs
So there seems to be a huge philosophical difference behind how Moz and Ahrefs calculates page rank (PA). On Moz, PA is very dependent on a site's DA. For instance, any new page or page with no backlinks for a 90DA site on Moz will have around 40PA. However, if a site has around 40 DA, any new page or page with no backlinks will have around 15PA PA. Now if one were to decide to get tons of backlinks to this 40 DA/15PA page, that will raise the PA of the page slightly, but it will likely never go beyond 40PA....which hints that one would rather acquire a backlink from a page on a high DA site even if that page has 0 links back to it as opposed to a backlink from a page on a low DA site with many, many backlinks to it. This is very different from how Ahrefs calculates PA. For Ahrefs, the PA of any new page or page with no backlinks to it will have a PA of around 8-10ish....no matter what the DA of the site is. When a page from a 40DA site begins acquiring a few links to it, it will quickly acquire a higher PA than a page from a 90DA site with no links to it. The big difference here is that for Ahrefs, PA for a given page is far more dependent on how many inbound links that page has. On the other hand, for Moz, PA for a given page is far more dependent on the DA of the site that page is on. If we were to trust Moz's PA calculations, SEOrs should emphasize getting links from high DA sites....whereas if we were to trust Ahref's PA calculations, SEOrs should focus less on that and more on building links to whatever page they want to rank up (even if that page is on a low DA site). So what do you guys think? Do you agree more with Moz or Ahref's valuation of PA. Is PA of a page more dependent on the DA or more dependent on it's total inbound links?
Algorithm Updates | | ButtaC1 -
Placement of /p/ in URL structure for ecommerce site product URLs
Hi, We're a discussion about how to structure a clients ecommerce site product page URLs where 12345 represent the product SKU/number: https://domain.com/Item--i-12345 https://domain.com/product-name/p/12345 https://domain.com/p/12345 It's a toss up between the second and the third URL, but the SEO company is saying the third is best because of the placement with the /p/ and creating a silo for "products" that help search engines recognize it is a product. Does anyone have thoughts on this? Thanks!
Algorithm Updates | | AliMac260 -
MOZ.com Page Rank of 2?
I don't recall the page rank of SEOMoz.com prior to the company's change to MOZ.com. But did notice that MOZ.com currently has a Page Rank of 2 (which I find weird since it's such a strong, content rich, highly-regarded site). I'd be interested in hearing about findings from the MOZ.com team on why the low PR and how has it affected your site since the change? (...and perhaps a look at the future through a crystal ball 🙂 I recall reading the MOZ domain changing article titled "Domain Migrations: Surviving the "Perfect Storm" of Site Changes" which had great info and addresses some reasons for PR loss in the 'Traffic and Ranking Loss' section: http://moz.com/blog/domain-migration-lessons
Algorithm Updates | | Prospector-Plastics0 -
Diluting your authority - adding pages diluting rankings of other pages?
I'm looking after a site that has around 400 pages. All of these pages rank pretty well for the KW they are targetting. My question is: if we add another 400 pages without doing any link building work, holding DA the same, 1) would the rankings of those 400 previously good pages diminish? and 2) Would the new pages, as more and more new ones are created, rank less and less well?
Algorithm Updates | | xoffie0 -
Does articles for SEO purposes have a minimal and maximum word count in ordered to be crawled/indexed by Google and other search engines?
Does articles for SEO purposes have a minimal and maximum word count in ordered to be crawled/indexed by Google and other search engines?
Algorithm Updates | | WebRiverGroup0 -
How to add search box to the snippet?
Hi All, Ho to add search box o the snippet, like in Openoffice.org snippet? http://goo.gl/TCv4I
Algorithm Updates | | bubliki0 -
Google Page Rank?
We have had a quality website for 12 years now, and it seems no matter how many more links we get and how much new content we add daily, we have stayed at PR3 for the past 10 years or so. Our SEOMoz domain authority is 52. We have over 950,000 pages linking to us from 829 unique root domains. Is this in line with PR3 or should we be approaching PR4 soon? We do daily blog posts with all unique, fresh quality content that has not been published elsewhere. We try to do everything with 'white hat' methods, and we are constantly trying to provide genuine content and high quality products, and customer service. How can we improve our PR and how important is PR today?
Algorithm Updates | | applesofgold0 -
Why am i seeing a "conduit" line for search engine sources in Google Analytics ?
Among Google, Yahoo, Bing etc... One of the line is "Conduit". I never heard about this engine but, accordingly to Google Analytics metrics, it is the engine that bring the best traffic to my site in terms of pages per visit.
Algorithm Updates | | betadvisor0