Having possible problems with rankings due to development website
-
Hi all,
I've got an interesting issue and a bit of a technical challenge for you.
It's a bit complicated to explain, but please bear with me.
We have a client website (http://clientwebsite.com) which we are having a hard time ranking in the past few months. Main keywords simply don't show up in Top100 searches, even though we are constantly building backlinks through Guest Posts, Citations, Media mentions, Profile links etc.
Normally, we use ahrefs to look at the client's website backlinks, but just today we used Majestic to look at the backlink profile and one backlink stood out.
This is a backlink from a development server (http://developmentwebsite.com) which redirects to http://clientwebsite.com
The developers who were working on the redesign of the client website, put it up on their server and forgot to delete it.
Also, the content inside the development website is almost identical with the client website.We then checked to see if http://developmentwebsite.com is indexed.
It's not.Although, inside the robots file http://developmentwebsite.com/robots.txt there's:
User-agent: *
Allow: /
The funny (and weird thing) is that http://developmentwebsite.com/ and all development website inner pages are not indexed in Google.But if we go to http://developmentwebsite.com/inner-page, it doesn't redirect to the corresponding http://clientwebsite.com/inner-page, it's the same development website page URL and the pages even have links to the client website, but like I said, none of the pages of the development website are indexed, even though crawlers are allowed in the robots.txt's development website.
In your opinion, could this be the reason why we are having a hard time to rank the client website?
Second question is:
How do we approach in solving this issue?
Do we simply delete the whole http://developmentwebsite.com with all the inner pages?
Or should we do 301 redirrects on a per-page basis? -
the problem, might be about your template .
-
Thank you Jake, will definitely give this some thought.... Appreciate the help.
-
Recommending link development would be difficult without knowing more about your specific industry, competitive advantages, etc.
Generally speaking, you want to focus first on ensuring you have valuable assets capable of justifying new links to your site. This would be a variety of content types depending upon your business, but can range from Free Advice and How To Guides, Data Insights, Tools/Widgets, Detailed comparison/reviews, Interviews and insights from industry through leaders, etc. You can use methods like broken link building to identify new content opportunities that you can capitalize upon, or look at top linked pages on competitor sites to see the types of content that has earned their links.
Once you have established your linkable assets, you can begin the process of outreach and amplification to bring attention to those assets and earn natural links. Of course, you can always reach out to other thought leaders, cited sources, etc... to encourage some additional sharing, linking, etc.
You can also do link profile comparisons to see if there are reputable outlets citing your competitors but not mentioning you.
Those are just a few examples off the top of my head. If you have the resources to do link dev internally and just need strategy guidance, you should check out the Moz recommended list, as many of those agencies (I know ours does) will offer ad-hoc consultation to help your team get the systems in place to be successful with your link dev efforts.
Cheers,
Jake
-
Thanks, Jake. We've had the dev site removed.
As a side note, what link-building would you recommend if you think we should not be doing outreach / guest posting?
Cheers!
-
You have to search and replace the new sites URLs with the same consistent URL not from the dev server then block the dev server.
- search and replace the URLs in the new site.
- block the developer site with a password
First things first take Make sure you have all the images and code on the http://clientwebsite.com/
Then go to the new site and do a search and replace on all the URLs including the database.
Search for http://developmentwebsite.com & replace with http://clientwebsite.com
- https://www.funduc.com/search_replace.htm
- https://interconnectit.com/products/search-and-replace-for-wordpress-databases/
After doing this.
Go on to the development site and block URLs by password-protecting your server directories
Place a password on the site preventing it from being indexed by anything or even accessed without using the password.
- https://support.google.com/webmasters/answer/93708
- https://blogvault.net/how-to-block-google-from-indexing-your-wordpress-site/
You might want to check the header tags in the development site URLs. This would only be for curiosity you really want these things gone.
Did you check for
_(…)_ **X-Robots-Tag: noindex** _(…)_
Or no index meta tag in the section of your page:
Lastly, remove the offending URL's from Google if you must & follow this guide when launching sites.
I hope this helps,
Tom
-
You can also use Moz's Open Site Explorer to look at the spam scores of your incoming links.
-
Zakkyg, as previously mentioned, the problem may be the "building backlinks through Guest Posts and Profile links, and the type of links that you're building to the site. I would take one step further and run the site's links through Link Research Tools' Link Detox to make sure that the links to the site are good links and you're not building any toxic links.
I would also immediately take down the development site completely. Make it deliver 404 error pages, with all URLs. There's no need to set up redirects at all. Since it is an old dev site, it really shouldn't have any traffic or links to it--so I would just remove it.
-
Great question..
First... some of your rankings problem may be around this - "Guest Posts, Citations, Media mentions, Profile links etc."... at least 2 of those items carries some decent risk, the consequences of which would be "not ranking". Have you checked search console to see if any manual actions exist? Did you have rankings and they disappeared, or have you been unable to secure rankings in general? Does the site currently get organic traffic, but just maybe on terms that you aren't targeting?
As far as the development site, this could easily be a separate issue from your rankings, and I would doubt this would cause significant rankings drops if the dev site isn't being indexed by Google. Would you be lucky enough to have any type of access to this through a CMS login? If so, you could easily verify the site in search console and see where things stand in terms of indexing. Additionally, if you could access through the CMS, you might be able to change a few settings, such as canonical tags, "website" or discourage indexing (if wordpress), etc. .that would cause the site to begin redirecting to your clientwebsite.com or at least give the indication that it is a known copy.. Did you check the individual pages of the site to see if a meta tag is in place on the pages blocking robots?
Ultimately the easiest thing regarding the dev site would be to simply ask the dev company to remove the site...
Cheers,
Jake
-
I would do the redirect to be safe then in the future make sure you run noindex on any development sites.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Website Indexing Issues - Search Bots will only crawl Homepage of Website, Help!
Hello Moz World, I am stuck on a problem, and wanted to get some insight. When I attempt to use Screaming Spider or SEO Powersuite, the software is only crawling the homepage of my website. I have 17 pages associated with the main domain i.e. example.com/home, example.com/sevices, etc. I've done a bit of investigating, and I have found that my client's website does not have Robot.txt file or a site map. However, under Google Search Console, all of my client's website pages have been indexed. My questions, Why is my software not crawling all of the pages associated with the website? If I integrate a Robot.txt file & sitemap will that resolve the issue? Thanks ahead of time for all of the great responses. B/R Will H.
Intermediate & Advanced SEO | | MarketingChimp100 -
My SERP Rankings are unbelievably inconsistent, how is it possible
Almost all of the keywords i track in moz's keywords ranking report show insane inconsistency. nearly all my keywords (according to moz's rank tracker history) were in the first or second rank for a week or 2 during the last 6 month.
Intermediate & Advanced SEO | | datawork
nearly all my keywords were nowhere in the top 5 pages in the weeks prior or after the week that the keyword was ranked at the top of the first page. I'm adding a image of an excel chart showing the ranking - each row is a different keyword, each column is a different date. 51 means 51+ (or - nowhere to be found in the first top pages) Could you please help me make sense out of this? How can it change so frantically? Thanks, eZDVEEW.png0 -
.Com version of my site is ranking better than .co.uk for my UK Website for branded search. 301 redirect mess
Dear Mozzers, I have an issue with my UK Website (short url is - http://goo.gl/dJ7IgD ) whereby when I type my company name in to google.co.uk search the .com version returns in Search as opposed to the .co.uk and from looking at open site explorer the page rank of the .com is higher than the .co.uk ?. Infact I cant even see the .co.uk homepage version but other pages from my site. The .com version is also 301'd to the .co.uk. From looking at Open Site Explorer, I have noticed that we have more links pointing to .com as opposed to .co.uk. Alot of these are from our own separate microsites which we closed down last year and I have noticed the IT company who closed them down for some reason 301'd them to the .com version of our site as opposed to the .co.uk but If I look in http://httpstatus.io/ (http status checker tool) to check one of these mircosites it shows - 301 - 302 - 200 status codes which to me looks wrong ?. I am wondering what it should read ... e.g should it just be a 301 to a 200 status code ?. My Website short url is - http://goo.gl/dJ7IgD and an example of some of 10 microsites we closed down last year which seems to be redirected to .com is http://goo.gl/BkcIjy and http://goo.gl/kogJ02 As these were redirected almost a year ago - it is okay if I now get them redirected to the .co.uk version of my site or what should I do ? They currently redirect to the home page but given that each of the microsites are based on an individual category of my main site , would it be better to 301 them to the relevant category on my site. My only concern is that , may cause to much internal linking and therefore I wont have enough links on my homepage ? How would you suggest I go about building up my .co.uk authority so it ranks betters than the .com- I am guessing this is obviously affecting my rankings and I am losing link juice with all this. Any advice greatly appreciated . thanks Pete
Intermediate & Advanced SEO | | PeteC120 -
Linking to External Websites?
Is it good to link external websites from every page. Since, the on-page grader shows there should be one link pointing to an external source. I have a website that can point to an external website from every page using the brand name of the specific site like deal sites do have. Is it worth having external link on every page, of-course with a no-follow tag?
Intermediate & Advanced SEO | | welcomecure0 -
SEO of blogging websites
What are the best practices of doing SEO of article/blogging websites.
Intermediate & Advanced SEO | | Obbserv0 -
Keyword density for a website
I wanna make landing pages for my web site .going to write 500 words article but how about keyword density .i wanna insert 4 keywords into that article.looking for expert advices .
Intermediate & Advanced SEO | | innofidelity0 -
Is my other domain making me not rank?
Hi there, We have a .co.uk website which was ranking well for a number of highly competitive keywords, however in February 2012 those rankings for those keywords suddenly dropped off Google all together and have never came back. A few possibilties to why this has happened: We launched a .ie website which has exactly the same content, could this be the reason for the drop? I have put in all the necessary steps in making sure Google ranks these geographically correct by using hreflang and making sure everything is setup properly in webmaster tools. Why I think it could be this: If I copy and paste the first few paragraphs of text from the pages in the .co.uk website that were ranked highly in Google.co.uk it's the .ie version that appears not the .co.uk version. Here is the webpages in question: http://www.avogel.co.uk/health/menopause/ http://www.avogel.ie/health/menopause/ Forgot to mention, the reason we have these two websites is due to different currency and legalities. Hope someone can help me out with this.
Intermediate & Advanced SEO | | Paul780 -
Sudden drop in ranks and traffic after migrating community website into main domain
Hi, We recently moved our community website (around 50K web pages) to our main domain. It now resides as a sub-domain on our main website. e.g. Before - we had www.mainwebsite.com and www.communitywebsite.com After - we have www.communitywebsite.mainwebsite.com This change took place on July 19th. After a week, we saw 16% drop in organic traffic to mainwebsite.com. Our ranks on most of the head keywords including brand keywords have dropped. We had created 301 redirects from pages on www.communitywebsite.com before this change was made. Has anybody seen this kind of impact when domains are merged? Should we expect that within 3-4 weeks Google will be able to re-index and re-rank all the pages? Is there anything else we could do to rectify the situation? Any feedback/suggestions are welcome!
Intermediate & Advanced SEO | | Amjath0