A Bash Script For When You Need To Duplicate Host Keys Across Servers

Why would you need to do this? Let’s say you have three production nodes: server1.test.com, server2.test.com, server3.test.com. In general, your production application (at prod.test.com) point to server1.test.com. But OH NO you totally messed something up on that node and you need to easily failover to server2.test.com. Easiest way? If they all have matching server host keys, you can just point prod.test.com to server2’s ip and you won’t get any errors. Otherwise, you’ll get loads. I also added a noop (no operation) mode to this script so it could be run as ./move_keys.sh -n and only print out the lines that it would run.

This script is written to be run on the server you want to copy the keys from as your user, which means you have to have ssh access into each server. For each server in the list that is not the host, it backs up the /etc/ssh directory, rsyncs the keys over, restarts ssh, then removes the relevant lines for that server in the known_hosts file.

#!/bin/bash
# use noop mode by passing -n
NOOP=false
# parse the options
while getopts 'n' opt ; do
  case $opt in
    n) NOOP=true ;;
  esac
done

SSHC='/usr/bin/ssh'
HOST=`hostname -f`

# Params: $1, host to log into and copy ssh dir into backup dir
copyit() {
  if [ $NOOP = true ]
  then
    echo "$SSHC -A $1 'sudo cp -r /etc/ssh /etc/ssh_bak_$(date +%Y%m%d%H%M)'"
  else
    $SSHC -A "$1" "sudo cp -r /etc/ssh /etc/ssh_bak_$(date +%Y%m%d%H%M)"
  fi
}

# Params: $1: src path, $2: dest path
moveit () {
  if [ $NOOP = true ]
  then
    echo "sudo -E rsync -avz -e 'ssh -o StrictHostKeyChecking=no' --rsync-path='sudo rsync' $1 $2"
  else
    sudo -E rsync -avz -e 'ssh -o StrictHostKeyChecking=no' --rsync-path='sudo rsync' $1 $2
  fi
}

# Params: $1, host to log into
restartit() {
  if [ $NOOP = true ]
  then
    echo "$SSHC -A $1 'sudo service ssh restart'"
  else
    $SSHC -A "$1" "sudo service ssh restart"
  fi
}

# Parmas: $1: host to deploy keys to
removeknown() {
  if [ $NOOP = true ]
  then
    echo "sudo ssh-keygen -f '/root/.ssh/known_hosts' -R $1"
    echo ""
  else
    sudo ssh-keygen -f "/root/.ssh/known_hosts" -R $1
  fi
}

# Params: $1: host to deploy keys to
movekeys () {
  copyit "$1"
  moveit "/etc/ssh/ssh_host_key" "$USER@$1:/etc/ssh/"
  moveit "/etc/ssh/ssh_host_key.pub" "$USER@$1:/etc/ssh/"
  restartit  "$USER@$1"
  removeknown "$1"
}


set -e

declare -a Servers
Servers[0]='server1.test.com'
Servers[1]='server2.test.com'
Servers[2]='server3.test.com'

Servers_minus_host=( ${Servers[@]/$HOST*/} )

for server in "${Servers_minus_host[@]}"
do
  movekeys $server
done