How to centrally administer Ubuntu servers

Question

I'm curious as to whether there is a convenient way to push updates to multiple Ubuntu servers from a central location instead of remoting into each one individually and manually updating them. Bonus points if this central location would be able to reboot the systems if needed due to updates.

I made a script for that using ssh to run two scripts I put on every server I can show you later — derHugo, Jul 06 '17 at 05:15

score 0 · Answer 1 · answered Jul 06 '17 at 06:44

The keyword is configuration management. One tool I use for this purpose is Ansible. There you express your desired configuration with a YAML file and Ansible takes care of it all. Of course you can also reboot hosts with this, see this answer which also gives you a first impression of how it will look.

There are also other options like Salt, Puppet, or Chef.

score 0 · Answer 2 · answered Jul 06 '17 at 06:46

If you like you can use my script I wrote for my servers ... there allways might be better ways to do it but it does what it should ;)

1. On each client Server

a) update script (this one also on the control server)

Make a script in /usr/local/bin/update

#!/bin/bash

# First get sudo rights
sudo echo ""

echo -e "\e[92m UPDATE PACKAGE SOURCES \e[39m"
sudo apt update
echo -e "\e[92m UPGRADE TO NEW PACKAGES \e[39m"
sudo apt upgrade
echo -e "\e[92m UPGRADE DISTRIBUTION-PACKAGES \e[39m"
sudo apt dist-upgrade
echo -e "\e[92m AUTOREMOVE OLD PACKAGES \e[39m"
sudo apt autoremove
clear
if [ -f /var/run/reboot-required ]; then
  echo -e "\e[92m#####################################\e[39m"
  echo -e "\e[92m#                                   #\e[39m"
  echo -e "\e[92m#               DONE                #\e[39m"
  echo -e "\e[92m#                                   #\e[39m"
  echo -e "\e[92m# \e[91mNOTE: A reboot of the System is\e[92m   #\e[39m"
  echo -e "\e[92m#       \e[91mrequired in order to\e[92m        #\e[39m"
  echo -e "\e[92m#       \e[91mcomplete the Update!\e[92m        #\e[39m"
  echo -e "\e[92m#                                   #\e[39m"
  echo -e "\e[92m#####################################\e[39m"
else
  echo -e "\e[92m#####################################\e[39m"
  echo -e "\e[92m#                                   #\e[39m"
  echo -e "\e[92m#               DONE                #\e[39m"
  echo -e "\e[92m#                                   #\e[39m"
  echo -e "\e[92m#####################################\e[39m"

if you trust in the updates you also can use the -y option so you don't have to interact (the most time; of course sometimes you are prompted for input depending on the updates).

Make it executable

chmod +x /usr/local/bin/update

b) rebootServer

make a script in /usr/local/bin/rebootServer

#!/bin/bash

echo -n "0"
sudo reboot &> /dev/null
exit

and make it executable

chmod +x /usr/local/bin/rebootServer

c) visudo

run

sudo visudo

add the lines

<userName> ALL=NOPASSWD: /usr/local/bin/update
<userName> ALL=NOPASSWD: /usr/local/bin/rebootServer

save and exit.
This allows the sudo user to run rebootServer and update (and any code/script within them) without being promoted to enter the sudo password.

d) add two more control scripts

One in /usr/local/bin/sshResponse

#!/bin/bash

echo -n "0"

and one in /usr/local/bin/rebootResponse

#!/bin/bash

#!/bin/bash

if [ -f /var/run/reboot-required ];
then
   echo -n "1"
else
   echo -n "0"
fi

And make them executable too

chmod +x /usr/local/bin/sshResponse
chmod +x /usr/local/bin/rebootResponse

The first one is only used to confirm the working ssh connection (there might be better ways).
The second one is used to display later wether a reboot is required on the updated servers.

2. On the main/controle Server

a) ssh-keygen

Because you don't won't to enter the password on every ssh login we use keys. Run

ssh-keygen

and follow the instructions.

Now you pass the key to every client server. So for each server run (on the control server)

ssh-copy-id <usernameOnClient>@<clientServerIP>

after that you should be able to connect to ssh from the control server without being prompted for a password.

b) the generator script

Now the crazy part. I use the following setup to automatically generated:

an individual reboot Script for each client Server in /usr/local/bin/rebootSERVERNAME
an individual update Script for each client Server in /usr/local/bin/updateSERVERNAME
a /usr/local/bin/rebootAllServers
a /usr/local/bin/updateAllServers (the last one also updates the control server itself)

add MY GENERATOR SCRIPT to /usr/local/bin/generateAllUpdateScripts and make it executable by

chmod +x /usr/local/bin/generateAllUpdateScripts

c) the config file

Finally you need a file in /root/.virtualMachines from where the generator reads the client Server's. It should look like e.g.

ClientServer1 192.168.1.2
ClientServer2 192.168.1.3

3. Usage

a) add client servers to `/root/.virtualMachines`

The generator script uses this file and reads the client server configs line for line to generate according scripts. (No empty lines! No spaces nor _ or - allowed in Server names!!)

b) generate the scripts

simply run

generateUpdateAllScripts

After finishing you should be able to call any of those commands:

to update all Servers

performs the update script on each machine and gives a summary in the end wether clientservers where updated and if reboot is required.

updateAllServers

to reboot all Servers

reboots all Servers and in the end waits until all servers get back online or displays of errors occurred

rebootAllServers

update a specific server

updateSERVERNAME

reboot a specific server

rebootSERVERNAME