I have an alternative: keep all your clients' SSH keys on one server, and the configure your servers to ask that server for authorized SSH keys. This can be done using the AuthorizedKeysCommand directive in sshd_config.
From the manpage:
AuthorizedKeysCommand
Specifies a program to be used to look up the user's public keys.
The program must be owned by root and not writable by group or
others. It will be invoked with a single argument of the
username being authenticated, and should produce on standard
output zero or more lines of authorized_keys output (see
AUTHORIZED_KEYS in sshd(8)). If a key supplied by
AuthorizedKeysCommand does not successfully authenticate and
authorize the user then public key authentication continues using
the usual AuthorizedKeysFile files. By default, no
AuthorizedKeysCommand is run.
AuthorizedKeysCommandUser
Specifies the user under whose account the AuthorizedKeysCommand
is run. It is recommended to use a dedicated user that has no
other role on the host than running authorized keys commands.
The command can be simply a script with:
#! /bin/bash
ssh bastion "cat $HOSTNAME/$1/authorized_keys"
Here the server's AuthorizedKeysCommandUser has a key-based (passwordless) login, and simply outputs the authorized_keys file for the hostname/user combination from the keyserver. Another way would be:
#! /bin/bash
wget --post-data="host=$HOSTNAME&user=$1" https://bastion
where the keyserver runs an HTTP server which returns a set of authorized keys based on the hostname and user. A one-file PHP or Python script would be sufficient to process such a form. An advantage would be that using a database server like PostgreSQL or MySQL would be easier. (Of course, you don't need to use the hostname.)
This method is most commonly used with LDAP, I think. There are a couple of scripts for LDAP: http://www.sysadmin.org.au/index.php/2012/12/authorizedkeyscommand/ and http://blather.michaelwlucas.com/archives/1562 that you can look at.
To set this up, in the servers, create a user (call it authkeys) to run the the command (or you can use the nobody user):
adduser --disabled-password --disabled-login --shell /usr/sbin/nologin authkeys
On the intermediate server (call it bastion), a slightly different command: we will set shell to /bin/bash for convenience (but you can limit that):
adduser --disabled-password --disabled-login --shell /bin/bash authkeys
This very unprivileged user will be used for the sole purpose of connecting to the intermediate server and getting the keys.
Create a script that this user will run:
sudo tee -a /usr/local/bin/get_keys.sh <<"EOF"
#! /bin/bash
ssh bastion "cat $HOSTNAME/$1/*.pub 2>/dev/null"
EOF
sudo chmod 0755 /usr/local/bin/get_keys.sh
Edit the SSH server configuration /etc/ssh/sshd_config on the servers and add:
AuthorizedKeysCommand /usr/local/bin/get_keys.sh
AuthorizedKeysCommandUser authkeys
And restart the SSH servers:
sudo service ssh restart
Then, for simplicity, generate an SSH key pair and use the same keypair on all servers for authkeys user:
# On one server
ssh-keygen -N '' -f id_rsa
On all the other servers, copy these id_rsa and id_rsa.pub files created by the above command, then add them to the authkeys user's SSH configuration:
cp id_rsa* ~authkeys/.ssh/
sudo chown authkeys:authkeys .ssh -R
sudo chmod 0700 .ssh
chmod 0600 .ssh/*
On the intermediate server, add this key to the authorized_keys list:
cat id_rsa.pub >> ~authkeys/.ssh/authorized_keys
Finally, on the intermediate serves, make a directory structure following the script we used above:
mkdir -p ~authkeys/some_host/some_login
Then add the authorized keys of your administrators to the directory:
echo "some_key" > ~authkeys/some_host/some_login/admin1.pub
echo "some_other_key" > ~authkeys/some_host/some_login/admin2.pub
Chown everything to root (or your user - but anything other than authkeys):
chown root:root ~authkeys/* -R
chmod o=rx ~authkeys/*/*
chmod o=r ~authkeys/*/*/*.pub -R
Use hostnames or IPs, but remember to change the script to use the appropriate value.
