SSH Keys Management - Header Image

A better way to manage SSH Keys

0 Flares 0 Flares ×

For IT administrators, DevOps leads and developers SSH Keys are a way of life. These little gibberish looking files, placed in esoteric .ssh folders inside your laptop or cloud server is the plumbing that makes all the applications and scripts work together. However, SSH key management is a severe pain for many organizations.

A little exercise for you: “Are you affected?”

How about a little exercise? Send a quick email to your IT lead/DevOps lead and ask them – how often do we rotate SSH Keys for all our employees? How much time does it take? You will be surprised with the answers.

Asking these questions is very important especially if your business needs to comply with PCI, HIPAA, SOX, SOC 1, SOC 2, FFIEC or any similar regulations.

The most common answer that people receive to the above two questions are – we don’t really have a way to do this easily. The last time we did this it was a year ago and it was painful.

Why is key rotation important and painful at the same time

Let us try to understand what SSH Keys are, why is key rotation important and most importantly why is it painful. Through this we can get a better understanding of how to bake in a more easy, yet secure way to do this. The goal should be to help your IT and DevOps teams provide compliance faster for your business.

SSH is a way for your employees to connect to cloud servers (and more) in a secure way. The security comes from the fact that any information going from your laptop to the server is encrypted. This prevents someone from snooping on your data. The second aspect from a security perspective is that SSH works when you authenticate yourself as the person who is allowed to connect to the servers. This is done In many ways. The primary two ways are by typing in a username-password combination or by using SSH keys.

SSH keys are basically small files that can be generated by the OpenSSH program on your computer (most computers running OS X or linux will have no problems doing this easily). These small files which seemingly contain gibberish actually show identity information. An SSH Key consists of two parts. A public key and a private key. Think about these two as your two hands that can palm over each other to fit nicely. Someone else’s palm on your palm does not fit that well.

Private and Public Key

The private portion of the key, again a small file is kept hidden on your laptop. In a specific directory to be accessed when you try to connect to a server. The public part of the SSH key is placed on the server you are trying to get access to. Now, when you try to connect to the server your laptop presents information to the server via the SSH key on your laptop that proves to the server its really you trying to get access. This completes a simple yet effective authentication process. Without your public SSH key nobody can get access to your account using SSH.

SSH keys are used in most organizations. The problem with SSH keys is keeping track of whose SSH key has been placed on which machines and making sure that the keys get changed every couple of days/weeks so that it is hard for anyone to guess what the key Is and try to attack the server by forging your identity. One of the main issues is that IT admins have to spend a lot of time visiting each server, changing out the private keys then go back to each user and have them make changes on their laptop. As any seasoned IT administrator knows, this strategy is a no go. A better way to do things is to use automated configuration management tools, however these tools are not built to solve this problem and hence have limitations like not being able to tie in changes of ssh keys when things change on a developers laptop. Furthermore these tools are meant to handle configuration on a machine – what operating system is running, what security patches are in place but not who is authenticating and what are they doing.

The most effective techniques for SSH Key Management

One of the most effective techniques we have seen in organizations which suffer from issues like these is to build a detailed process which includes policies for key generation, policies for key distribution and key rotations. Then based on the security level of each choosing an existing solution or building their own system based on Chef/Puppet/Openstack Keystone or some other mechanism. The great news is that with Open Source Software the amount of flexibility and customization you can get is endless. However the downside is that you will need to spend development time in order to build out the system you really want. One size fits all does not work in this case. An example would be : (1) We need to generate SSH Keys every month/week/quarter (2) We will use keywhiz FS to keep a central store for keys (3) the servers can be hooked on to keywhiz or can be linked to LDAP or other data sources to validate logins. This is an effective mechanism but it is time consuming to implement and more often than not requires a lot expertise in the area.

I hope that you find what the right balance for your organization but Keywhiz-FS, in conjunction with Chef/Puppet and LDAP/AD is a good way to go if you want to invest the time to build your own solution.

Before I leave here are two amazing example of key management by an IT person. Originally found on

SSH Keys - 1 SSH Keys Management - 2


Photo credit: and


Also published on Medium.

Share this post

0 Flares Twitter 0 Facebook 0 Google+ 0 Reddit 0 LinkedIn 0 Buffer 0 0 Flares ×