Test contet/pages indexed by search engines
-
During the web development stages of our Joomla CMS website, we have managed to get our site indexed for totally irrelevant test pages mainly to do with Joomla and some other equally irrelevant test content. How damaging is this to our domain from an SEO prospective and is there something we can do about it?
When we do a site:domain.com search we see hundreds of testpages with test/irrelevant meta tags etc.
-
Search engines regularly recrawl every website and will update their information based on changes you make to your site. It is a natural part of the internet. The "site under construction" information is not harmful, but in the future should be blocked from indexing.
-
Thankfully its only test urls that have been indexed by Google only.
However all 3 major engines have indexed our domain against "Site under construction" page with untitled/incomplete tags.
Is this harmful or will this be overwritten when we launch properly and get our site indexed?
-
When you begin developing a site, you should use the robots.txt file to block all search engine access to the site. This is one of the few times where a robots.txt file is very useful.
With respect to fixing the issue, it depends on whether the URLs will be used on the live site, how long it will be until your site launched, and whether unique URLs such as /testing were used or you are working with the same URLs which will exist on the live site.
If your site is still in testing and it will remain in testing for 30+ days, you could add the noindex tag sitewide. Once all the pages were removed from the index, you can then add the robots.txt file. Be careful not to adjust the robots.txt file prior to the pages being removed as the search engines wont be able to see the noindex tag.
You did not mention which search engine indexed your pages. If you are working with Google and the URLs will not exist on the live site, you could use the Google Removal Tool. This is really overkill and should not be necessary, but if the site owner is paranoid about the test pages causing damage to SEO you can take this approach. Any URL removed in this manner cannot be re-added to the index for 90 days.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
New Featured Links in Organic Search Results?
Hi guys, I just performed a search and came across something that looks like "featured links" under a regular organic search result (see screenshot). This is the first time I'm seeing this. It looks like a combination of callout and sitelink ad extensions for Google ads. Basically, linked callouts. I went to the landing page to check out the source code and it seems like they are calling it "featured link" in their code. I tried to find more info online but wasn't able to find anything. (I might not be using the correct search terms.) Does anyone know how to take advantage of this? Thanks a lot for your feedback. dJ9dmTr
Algorithm Updates | | HinterP0 -
What happens when a de-indexed subdomain is redirected to another de-indexed subdomain? What happens to the link juice?
Hi all, We are planning to de-index and redirect a sub domain A to sub domain B. Consequently we now need to d-index sub domain B also. What happens now to the link juice or page rank they gained from hundreds and thousands of backlinks? Will there be any ranking impact on main domain? Backlinks of these sub domains are not much relevant to main domain content. Thanks
Algorithm Updates | | vtmoz1 -
404s in Google Search Console and javascript
The end of April, we made the switch from http to https and I was prepared for a surge in crawl errors while Google sorted out our site. However, I wasn't prepared for the surge in impossibly incorrect URLs and partial URLs that I've seen since then. I have learned that as Googlebot grows up, he'she's now attempting to read more javascript and will occasionally try to parse out and "read" a URL in a string of javascript code where no URL is actually present. So, I've "marked as fixed" hundreds of bits like /TRo39,
Algorithm Updates | | LizMicik
category/cig
etc., etc.... But they are also returning hundreds of otherwise correct URLs with a .html extension when our CMS system generates URLs with a .uts extension like this: https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.html
when it should be:
https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.uts Worst of all, when I look at them in GSC and check the "linked from" tab it shows they are linked from themselves, so I can't backtrack and find a common source of the error. Is anyone else experiencing this? Got any suggestions on how to stop it from happening in the future? Last month it was 50 URLs, this month 150, so I can't keep creating redirects and hoping it goes away. Thanks for any and all suggestions!
Liz Micik0 -
How much is Page Rank really worth?
We are in a position to purchase a domain, made of relevant keywords to our company with a current page ranking of 4 for their home page. However in looking at their analytics and other information they do not do well on significant keywords and have very low site traffic. In fact they do very, very poorly. With their high page ranking would it be relatively easy to conduct a successful SEO campaign on the domain if we were to take it over as our own and attempt to climb in the SERP's? I know Page Rank doesn't mean everything when it comes to your ranking, but 4 is relatively high in our field, so I don't really understand why they do so poorly when it comes to their actual rankings on key words.
Algorithm Updates | | absoauto0 -
Google Search CTR % By Position
Hello I am looking for an updated report regarding the CTR % by position for Google search results. I have the compete.com report which Gives the 1st organic position a 53% CTR but I have not be able to duplicate that number with any other report or research. I am just trying to validate this report before I suggest any recommendations to my company regarding our search efforts. Thank you Ben
Algorithm Updates | | bhalverson30 -
Does the Search Algorithm vary considerably locally?
Hey, i am from india and I just noticed that most of our searches are extremely different to those from the gooogle.com searches. Not some searches. I mean entire layouts. For instance, there were no google places in the search results in India. There was hardly any integration with the G+ for a long time after it launched, even though a large population on G+ was Indian. I got thinking on these lines. Any pointers?
Algorithm Updates | | rahul.bitmesra0 -
How do I rank multiple pages for my busness/domain name?
When someone searches for our business's name (which is also the domain name) we have one listing (with sitelinks) at the top - however I would also like to rank 2nd, 3rd and 4th for this term. Any suggestions on how this might be done? Thanks.
Algorithm Updates | | CaBStudios0 -
Google +1 link on Domain or Page?
Since its release, I've seen Google +1 being used across an entire domain but only reference the root href in the code snippet. At the same time, you see other sites use +1 more naturally with the button being specific to the page you're on. What's your take on this? To clarfiy, do you add: or .. on each page.
Algorithm Updates | | noeltock0