Okay, I’ll admit; I am shit at updating this site… blog… thing… Whatever, today we’re going to build something cool; centralized SSH authentication using Hashicorp’s Vault. We’re going to stand up a Vault server and use it to generate One-Time-Passwords for SSHing (totaly a word) onto other servers.
That’s actually a good question. Honestly, PAM just seems like a shitfest to set up and maintain.
All right, I might be biased towards Hashicorp products. Having not actually managed a PAM enviroment before I could be wrong about its ease of management. It does give you to benifit of unique users logging onto servers helping auditability. But, if you’re already using Vault to manage other secrets, why not use it for SSH access management as well? It gives you one-time-passwords so you won’t have to worry about reuse. Even if you’re not using it for authenticating your users, it could be useful for providing credentials during the provisioning of new servers.
Standing up the Environment
This part’s pretty easy. We’re going to use Vagrant to create two virtual machines. One to act
as our Vault server and other for testing authentication. First, copy the following into a file named
Vagrant.configure '2' do |config| config.vm.box = 'ubuntu/xenial64' config.vm.define 'vault' do |vault| vault.vm.network 'private_network', ip: '172.23.1.11' end config.vm.define 'test' do |test| test.vm.network 'private_network', ip: '172.23.1.12' end end
vagrant up. This can take a while as it creates our new VM’s. To connect to either server, use one of the following:
$ vagrant ssh vault $ vagrant ssh test
Set up Vault
Next we’re going to have to set up our Vault server. Installing Vault is really simple; you pretty much just download and run. This entire step is run on our “vault” server.
$ sudo apt-get install -y unzip sshpass $ curl https://releases.hashicorp.com/vault/0.6.4/vault_0.6.4_linux_amd64.zip > vault.zip $ unzip vault.zip $ sudo cp vault /usr/local/bin
Now just run
vault to confirm it’s on your PATH.
$ vault usage: vault [-version] [-help] <command> [args] ...
With Vault installed the next step is to start a Vault server. We’re going to run the server in “dev” mode. It’s not secure, but it means we won’t have to screw around with the Sealing/Unsealing process. To read more on that check the official documentation here.
$ vault server -dev -dev-listen-address="0.0.0.0:8200" ==> Vault server configuration: Backend: inmem Cgo: disabled Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "", tls: "disabled") Log Level: info Mlock: supported: true, enabled: false Version: Vault v0.6.4 Version Sha: f4adc7fa960ed8e828f94bc6785bcdbae8d1b263 ==> WARNING: Dev mode is enabled! In this mode, Vault is completely in-memory and unsealed. Vault is configured to only have a single unseal key. The root token has already been authenticated with the CLI, so you can immediately begin using the Vault CLI. The only step you need to take is to set the following environment variables: export VAULT_ADDR='http://0.0.0.0:8200' The unseal key and root token are reproduced below in case you want to seal/unseal the Vault or play with authentication.
You’re going to want to run the part where it says to export the environment variable. By default Vault is expecting to be able to connect via HTTPS but the dev server doesn’t have SSL so it won’t connect.
$ export VAULT_ADDR='http://127.0.0.1:8200' $ vault status Sealed: false Key Shares: 1 Key Threshold: 1 Unseal Progress: 0 Version: 0.6.4 Cluster Name: vault-cluster-c75b1c92 Cluster ID: 0b68057c-c620-5c5d-aff1-9a8a4e1321ba
Setting up the SSH Backend
Awesome. So now we’ve got our Vault server running, we can set it up to use the SSH backend. First we’re going to have to mount this on “vault”.
$ vault mount ssh
Create a new role with the
key_type set to
otp. This tells the backend to use One-Time-Passwords everytime a client wants
to SSH onto a server. In this case our username is going to be
localadmin and the machines authenticating on the
$ vault write ssh/roles/otp_key_role key_type=otp default_user=localadmin cidr_list=172.23.1.0/24
Install the Agent on the Test Server
Jump onto the test server and install the agent.
$ sudo apt-get install -y unzip $ curl https://releases.hashicorp.com/vault-ssh-helper/0.1.2/vault-ssh-helper_0.1.2_linux_amd64.zip > vault_agent.zip $ unzip vault_agent.zip $ sudo mv vault-ssh-helper /usr/local/bin
Create the config file
vault_addr = "http://172.23.1.11:8200" tls_skip_verify = true ssh_mount_point = "ssh" allowed_roles = "*"
/etc/pam.d/sshd file as per below. Note that you’re commenting out
@include common-auth if it already exists.
##@include common-auth auth requisite pam_exec.so quiet expose_authtok log=/tmp/vaultssh.log /usr/local/bin/vault-ssh-helper -config=/etc/vault-ssh-helper.d/config.hcl -dev auth optional pam_unix.so not_set_pass use_first_pass nodelay
/etc/ssh/sshd_config with the following
ChallengeResponseAuthentication yes UsePAM yes PasswordAuthentication no
Check the config
$ sudo vault-ssh-helper -verify-only -config=/etc/vault-ssh-helper.d/config.hcl -dev 2017/02/04 11:04:48 ==> WARNING: Dev mode is enabled! 2017/02/04 11:04:48 [INFO] using SSH mount point: ssh 2017/02/04 11:04:48 [INFO] vault-ssh-helper verification successful!
Create the user
sudo adduser localadmin
We’re now ready to test. Back on the “vault” server run
$ vault ssh -role otp_key_role firstname.lastname@example.org Welcome to Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-38-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage Get cloud support with Ubuntu Advantage Cloud Guest: http://www.ubuntu.com/business/services/cloud 118 packages can be updated. 48 updates are security updates. *** System restart required *** Last login: Sat Feb 4 11:16:44 2017 from 172.23.1.11 localadmin@ubuntu-xenial:~$
If you get
Failed to establish SSH connection: "exit status 6" it means that you don’t yet trust the test server. Just SSH
onto it normally and it will save this to the list on known hosts.
$ ssh email@example.com ECDSA key fingerprint is SHA256:9rwKBR6UpLOFmX+Et49hFVpNRyhP4i2jJfw/CC70lTw. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '172.23.1.12' (ECDSA) to the list of known hosts.
Awesome. So now we can use Vault to generate credentials for logging onto servers. Next up you’d want to stand this up as a production Vault cluster and lock down who can access the “ssh” mount point. That is definitely not going to be covered here.