you can use whatever CMS or code you want on this part.

How to keep your hosting and create a staging environment NON WordPress as well

you can use it on a server as bespoke as you please. (You must have root access)

https://confluence.atlassian.com/display/JIRA/Establishing+Staging+Server+Environments+for+JIRA

http://stackoverflow.com/questions/3844369/git-production-staging-server-workflow

I hope this will be more helpful.

all the best,

Tom

Are you running WordPress? If you are here are a list of hosting companies that provide staging servers.

I'm glad to hear you guys are looking into possibly working this out hopefully it's just a miscommunication.

I think staging servers are a godsend and allow people to do things that they will know if they could have gotten away with are not once they try it.

This list is partial to staging every web post is very good there's one I state do not use unless you use their VIP plan I wish I did have to say that but I can't recommend them without stating that.

http://wpengine.com/plans/ (probably the best staging environment available)

You can get four months for free of WPengine & 30% off for life from Pagely via https://Moz.com/perks

https://kinsta.com/features/ (outstanding staging and hosting)

https://pantheon.io/features/version-control-workflow (awesome)

http://www.presslabs.com/ (great hosting and staging)

https://getflywheel.com/tour/staging-sites/ (must just say please turn on staging)

https://pagely.com/plans-pricing/wordpress-vps-hosting/ (one of my favorites but staging only available on VPS & enterprise plans. I must say it is worth it)

https://pressable.com/ (unless you're going to get a private “VIP” cloud the downtime is very bad and will hopefully get better soon but until then I cannot say use these guys in less you are going for the VIP plan an amazing staging set up.)

https://www.siteground.com ( inexpensive offers staging)

http://mediatemple.net/webhosting/wordpress/ (offers staging)

** How to keep your hosting and create a staging environment NON WordPress as well**

http://www.wpbeginner.com/wp-tutorials/how-to-create-staging-environment-for-a-wordpress-site/

https://confluence.atlassian.com/display/JIRA/Establishing+Staging+Server+Environments+for+JIRA

http://stackoverflow.com/questions/3844369/git-production-staging-server-workflow

I think if the person is really trying to do things without other people knowing what they are that's not good. At the same time I would be wary of anyone keeping secrets.

I hope this information is a step towards you guys working it out.

Sincerely,

Thomas

Hi Andy,

I do the same thing with SearchLove I unfortunately just missed localU (root canal no fun)

Smart idea I think you will love Seattle it's a beautiful city. Don't forget to check out the SubPub in the airport if you're a Nirvana fan.

Sincerely,

Thomas

Try using a staging server it's kind of crazy to be making changes to the live site regardless. There are some great staging servers out there that will allow access to them then only once the site is definitely not broken or heard in any other fashion then you can move forward.

Saves you having to do their work and them having this issue. I do agree everyone talking about the secrecy stuff there's no secret sauce it's create the best website and best user experience.

Most people would be proud to say what they're doing not say that this is MI fives operation

I want to say I agree anything they say is "secret" is probably not white hat. I don't think anyone here is worried about someone stealing their formula for on page search.

However it's possible the person he's used to working in a certain way.

If you're going to do the work and allow them to delegate to you and you're actually going to do what they say I don't see an issue with it all.

I understand how that SEO feels you are preventing them from doing their job if you do not allow them root access.

If you do not trust them enough to actually be in there on the server then you should not be using them.

There are alternatives like staging servers but search engine optimization is done over a long time and definitely requires admin access to the server I imagine this person does not want you doing what you think is best and affecting their work just the same way as you do not want them affecting you by doing something else to the server.

A good SEO should work with a developer if they do not have that knowledge.

this is actually something a SEO should know before taking the job although it is definitely assumed.

I would not presume to tell you how to do your job I feel that they probably feel the same way.

Hope that helps,

Tom

I will see you there although I'd think there will be enough to do at MozCon

well some of them are and your navigation which obviously should contain internal URLs does contain internal URLs. I am sending a screenshot of one of your widgets and a look at the actual URL which is an internal URL as your other widgets contained the same domain "hindimeaning.com" they are internal URLs as well.

here is a screenshot below of your entire home page sorry if it looks strange my browser is dragged pretty wide.

Regardless it looks like you using a Genesis 1 framework is this correct?

Go into appearance then widgets you will see the URLs and be able to modify them to your liking.

So far I have not seen external URLs meaning from another domain.

I hope this is helpful,

Thomas

f7Vqv0z.png t3hEgNM.png

in addition to analyticsseo having an API SEMrush which is now doing on page and off page seo through their site audit option is allowing this at an extremely low price via their API. So those are two companies that will offer on page SEO via API I hope that helps,

Tom

popular posts and related posts are dynamic so they would not show up as URL's when you say widgets

you mean the widgets containing the URLs that make up most of the website as I have only seen two pages but they look very similar.

this Would not allow me to click on the other URL until after the video that's what the second link is for

http://cl.ly/ZkJn

http://cl.ly/Zk4r

your widgets contain URLs

could you please rephrase the question and take note that the code I looked at is not ideal.

XBK7Qer.png

Glad to be of help. Check out this Google link to confirm you picked up the 180 day crawl

https://support.google.com/webmasters/answer/83106?hl=en

Second URLs helpful as well.

http://blog.raventools.com/moving-site-from-http-to-ssl/

all the best,

tom

"Now that https is the canonical version, should I block the http-Version with robots.txt?"

Absolutely not GWT will handel all of it think about backlinks both https:// & http:// urls you will not want to lose the flow of link juice that you would cut off

Remake robost.txt with

http://www.internetmarketingninjas.com/seo-tools/robots-txt-generator/

But use https:// for the xml sitemap.

hi Tim,

if you have anything that gets a lot of traffic you could move that to the new site like moz.com/rand/ this would help get the crawlers going. I will give you a full crawl and also I am sorry I just  Getting back to you.

I am so sorry I did not mean to take this on and I will check your private message I've been so busy.

This is something to hold onto download it and keep it fill it out keep it somewhere safe

http://netdna.copyblogger.com/documents/WordPress-Emergency-Checklist.pdf

SEO gadget change their name recently to builtvisible and made awesome post about how to keep link juice

http://builtvisible.com/surviving-seo-site-migration/

http://builtvisible.com/domain-migration/

http://builtvisible.com/change-of-address/

https://www.distilled.net/blog/seo/7-technical-seo-wins-for-web-developers/

https://www.distilled.net/blog/seo/a-wordpress-theme-change-seo-checklist/

Sean Anderson at hobo SEO to let the name fool you has a great article

http://www.hobo-web.co.uk/how-to-change-domain-names-keep-your-rankings-in-google/

http://www.searchenginepeople.com/blog/can-you-avoid-losing-google-rankings-when-moving-your-site-to-a-new-domain.html

http://moz.com/blog/seo-guide-how-to-properly-move-domains

http://mysiteauditor.com/blog/changing-your-domain-name-without-losing-your-rankings-a-12-step-guide/

http://www.webseoanalytics.com/blog/seo-guide-how-to-change-your-domain-name-without-losing-your-rankings/

I will reply and I strongly recommend running screaming frog SEO spider or deep crawl in fact I will go into your private message right now and start a deep crawl on your site. This will give you everything. http://deepcrawl.co.uk  I will also of course run screaming frog and send you the archives.

Sorry it took so long to respond,

Tom

User-agent: *

User-agent: dotbot
Disallow: /
User-agent: rogerbot
Disallow: /

if you want to prevent robots from crawling your site truly prevent them you will need to use either a password restriction or a tool similar to this

http://www.distilnetworks.com/

If you see what is being said by Google and Moz a robots.txt file can not guarantee   blocking something that is linked to. if you want to do that you will have to block the referral using a WAF like distilnetworks

http://moz.com/help/guides/search-overview/crawl-diagnostics

https://moz.com/researchtools/ose/dotbot

&

https://support.google.com/webmasters/answer/6062608?rd=2

Also blocking link analysis user agents that are nothing but a drain on your resources is a good idea. Simple enough to do in htaccess with something like this:

Search Engine Blocked by Robots.txt

This page cannot be crawled by search engines due to the robots.txt protocol. If you're seeking to remove this page from search results, we recommend that you use meta robots (with noindex, follow values) instead of robots.txt. This will ensure that the page does not appear in the results but allows link juice to flow through the page's links and count towards the relevance/popularity of other pages on your site.

How to block DotBot from crawling your site

If you don't want dotbot crawling your site, we always respect the standard Robots Exclusion Protocol (aka robots.txt). If you would like to block dotbot, all you need to do is add our user-agent string to your robots.txt file.

If you want to ban dotbot from most areas of your site, it looks a little something like this:

      User-agent: dotbot
      Disallow: /admin/
      Disallow: /scripts/
      Disallow: /images/
    

below this I have placed what somebody has created that they state works I do not know if it works I told you that distill networks will work but I cannot guarantee the very bottom I think you will not have any trouble if you set up the robots.txt as configured at the top.

If you want to ban dotbot from crawling any part of your site, add this text instead:

      User-agent: dotbot
      Disallow: /

    

BEGIN

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^rogerbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^exabot [OR]
RewriteCond %{HTTP_USER_AGENT} ^MJ12bot [OR]
RewriteCond %{HTTP_USER_AGENT} ^dotbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^gigabot [OR]
RewriteCond %{HTTP_USER_AGENT} ^AhrefsBot
RewriteRule .* – [F]

SetEnvIfNoCase User-Agent .rogerbot. bad_bot
SetEnvIfNoCase User-Agent .exabot. bad_bot
SetEnvIfNoCase User-Agent .mj12bot. bad_bot
SetEnvIfNoCase User-Agent .dotbot. bad_bot
SetEnvIfNoCase User-Agent .gigabot. bad_bot
SetEnvIfNoCase User-Agent .ahrefsbot. bad_bot
SetEnvIfNoCase User-Agent .sitebot. bad_bot

Order Allow,Deny
Allow from all
Deny

END

Thomas

Hi Tim,

I think you did a great job of writing everything out. Would you mind sharing your URL's with me via private message if you are uncomfortable sharing it here?

I would be very hesitant to tell you to merge sites without knowing what type of content they have.

However having said that Granger.com is a good example of a site that sells just about everything. The only common theme is industrial products. But those range so vastly that you can understand what I mean.

as far as the best way to do it take the domain with the most authority and keep that domain 301 redirect the other domain page by page to it.

Here is some information

• http://moz.com/community/q/how-to-keep-old-url-juice-during-site-switch
• http://moz.com/community/q/301-redirect-subdirectory-to-new-domain
• http://moz.com/community/q/which-page-should-i-301-redirect-to-the-other

let me know if this is enough information I'd be happy to field any questions.

All the best,

Thomas

HiI know there are two crawlers that Moz uses Roger bot and open site Explorer uses dotbot

Make sure there is no forward slash "/" after e.g. (Disallow:/)

Mozused to have an awesome writeup on it but it just forwards to Moz.com/help now it could be that they have another great write up but the URL changed. For now here's the information

User-agent: rogerbot
Disallow: 
User-agent: dotbot
Disallow:

User Agent Analyser
Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)

http://www.useragentstring.com/Dotbot1.1_id_16014.php

https://udger.com/resources/ua-list/bot-detail?bot=rogerbot

http://www.botopedia.org/user-agent-list/crawlers/item/369-rogerbot-seomoz

hope this helps,

Thomas

I actually forgot about another tool that does have this ability. Analytics SEO enterprise version does allow you to use their API externally. I have to warn you it's not cheep however it's a very good service.

http://www.analyticsseo.com/us/seo-data-api/

deep crawl in this may be the best on page SEO tool in the world in my opinion. It combines everything screaming frog has and places it hosted on servers so you can run millions of URI's they do have a API which you can integrate with.

on their enterprise offering. They start out at $80 a month and go up to thousands of dollars a month.

raven tools has a decent on page search tool however I think that there is no API access but I could be wrong.

I hope this helps and remember to utilize what Moz does offer because in my opinion they are the most complete tool out there and really thing everyone should have.

all the best,

Tom

Moz has access to their API as a pro member you have the ability to get acquainted with your information.

Check out scribecontent.com  it is a tool made by copy blogger that might do what you're asking.

Let me know if I can be of more help,

Tom