Occasionally you find yourself in a situation where utilizing a shared account cannot be avoided. One such scenario is managing the deployment of NodeJS applications with Shipit and PM2. Here’s how the scenario typically works:
Alice, Bob, and Carol are three developers working on NodeJS applications that need to be deployed to their staging server. They’ve decided on the use of PM2 as their process manager, and ShipIt as their deployment tool. Their
shipitfile.js file contains a block for the staging server, and it looks something like:
As we can see the
deployer user will be used to deploy our application, and by extension, will be the user that
pm2 runs the application under. Alice, Bob, and Carol all have their SSH keys put in
/home/deployer/.ssh/authorized_keys so they can deploy. Makes sense.
Unfortunately what this also means is Alice, Bob, or Carol can
ssh to the staging server as the
deployer user. Even though
deployer is an unprivileged user, we really don’t want that. Moreover, by default, we can’t trace who deployed, or if someone is misusing the
deployer user. Let’s take a look at how we can address this.
Creating a Deployment Group
The first thing we want to do is to create a security group for those that are authorized to perform deployments. I’ll be using an Active Directory security group in this example, but a standard Unix group would work as well. We can use
getent to see the members of the group.
getent will come in handy to help determine whether someone attempting to deploy is authorized.
# getent group "application email@example.com"
SSH authorized_keys command
Until I started researching this problem of auditing and restricting shared account usage I was unaware of the
command option in the SSH
authorized_keys file. One learns something new every day. What the
command option provides for is executing a command immediately upon SSHing via a given key. Consider that we put the following entry in the
ssh-rsa AAAA...sCBR alice
and this is Alice’s public key. We would expect that Alice would be able to
ssh firstname.lastname@example.org and get a shell. But what if we wanted to intercept this SSH and run a script instead? Let’s try it out:
command="/usr/bin/logger -p auth.INFO Not permitted" ssh-rsa AAAA...sCBR alice
When Alice tries to ssh as the
deployer user, we get an entry in
Jul 5 22:30:58 apps deployer: Not permitted
and Alice sees
Connection to apps closed..
Well that’s no good! We do want Alice to be able to use the
deployer account to deploy code.
A Wrapper Script
First, we want Alice to be able to deploy code with the
deployer user, but we also want to:
- know that it was Alice
- ensure Alice is an authorized deployer
- not allow Alice to get a shell
Let’s look at how we can create a script to execute each SSH invocation that will meet all of these criteria.
Step 1, let’s log and execute whatever Alice was attempting to do.
logger -p auth.INFO $SSH_REMOTE_USER executed "$SSH_ORIGINAL_COMMAND"
exec /bin/bash -c "$SSH_ORIGINAL_COMMAND"
SSH_ORIGINAL_COMMAND will be set automatically by
sshd, but we need to provide
SSH_REMOTE_USER, so in the
command="export SSH_REMOTE_USERemail@example.com;/usr/local/bin/deploy.sh" ssh-rsa AAAA...sCBR alice
Note that we explicitly set
firstname.lastname@example.org. The takeaway here is that it associates any attempt by Alice to use the
deployer account to her userid. We then execute
deploy.sh which logs the invocation. If Alice tries to
ssh and get a shell with
ssh deployer@apps the connection will still be closed, as
SSH_ORIGINAL_COMMAND is null. But, let’s say she runs
ssh deployer@apps ls /:
email@example.com@apps ~> ssh deployer@apps ls /
/var/log/auth.log we see:
Jul 6 13:43:25 apps sshd: Accepted publickey for deployer from ::1 port 48832 ssh2: RSA SHA256:thZna7v6go5EzcZABkieCmaZzp+6WSlYx37a3uPOMSs
Jul 6 13:43:25 apps sshd: pam_unix(sshd:session): session opened for user deployer by (uid=0)
Jul 6 13:43:25 apps systemd-logind: New session 54 of user deployer.
Jul 6 13:43:25 apps systemd: pam_unix(systemd-user:session): session opened for user deployer by (uid=0)
Jul 6 13:43:26 apps deployer: firstname.lastname@example.org executed ls /
What is important here is that we can trace what Alice is executing.
Managing a Deployment Security Group
Left as is this technique is much preferred to a free-for-all with the
deployer user, but more can be done using security groups to have finer control of who can use the account at any given time. Let’s add an additional check in the
/usr/local/bin/deploy.sh script with the
uig function introduced in the last post.
uig "$SSH_REMOTE_USER" "$DEPLOY_GROUP"
if [ "$?" == 0 ]; then
logger -p auth.INFO $SSH_REMOTE_USER is in $DEPOY_GROUP and executed "$SSH_ORIGINAL_COMMAND"
exec /bin/bash -c "$SSH_ORIGINAL_COMMAND"
logger -p auth.INFO $SSH_REMOTE_USER is not in $DEPLOY_GROUP and is not authorized to execute "$SSH_ORIGINAL_COMMAND"
authorized_keys file gets updated, and let’s add Bob and Carol’s keys for our additional positive (Bob) and negative (Carol) test:
command="export SSH_REMOTE_USERemail@example.com DEPLOY_GROUP='application firstname.lastname@example.org';/usr/local/bin/deploy.sh" ssh-rsa AAAA...sCBR alice
command="export SSH_REMOTE_USERemail@example.com DEPLOY_GROUP='application firstname.lastname@example.org';/usr/local/bin/deploy.sh" ssh-rsa AAAA...r4Gz bob
command="export SSH_REMOTE_USERemail@example.com DEPLOY_GROUP='application firstname.lastname@example.org';/usr/local/bin/deploy.sh" ssh-rsa AAAA...f5Xy carol
Since Bob is a member of the
application email@example.com group, he can proceed:
Jul 6 20:09:26 apps sshd: Accepted publickey for deployer from ::1 port 49148 ssh2: RSA SHA256:gs3j1xHvwJcSMBXxaqag6Pb7A595HVXIz2fMoCX2J/I
Jul 6 20:09:26 apps sshd: pam_unix(sshd:session): session opened for user deployer by (uid=0)
Jul 6 20:09:26 apps systemd-logind: New session 79 of user deployer.
Jul 6 20:09:26 apps systemd: pam_unix(systemd-user:session): session opened for user deployer by (uid=0)
Jul 6 20:09:27 apps deployer: firstname.lastname@example.org is in application email@example.com and executed ls /
Now, Carol’s turn to try
ssh deployer@apps ls /:
Jul 6 20:15:37 apps deployer: firstname.lastname@example.org is not in application email@example.com and is not authorized to execute ls /
For some teams the idea of having to manage a deployment security group and bespoke
authorized_keys file may be overkill. If you’re in an environment with enhanced audit controls and accountability the ability to implement safeguards and audits to code deployments may be a welcome addition.