Install Ubuntu to USB Flash Drives Automatically

Question

I need to create many USB flash drives that run Ubuntu. These flash drives should:

1. contain custom pre-installed software
2. be persistent: even after restart, programs and new files should be preserved

How can I automate this process to avoid manual steps (except for plugging in the drives)?

Answer 1

Create a script doing all the necessary steps and execute it after installation.

For example:

#!/bin/bash

set -e

apt update
apt install -y \
    vlc \
    curl \
    git \
    filezilla \
    ... others to be installed

apt purge -y \
    thunderbird \
    ... others to be uninstalled

gsettings set org.gnome.shell.extensions.desktop-icons show-trash false

apt upgrade -y
apt autoremove -y

You can even put the script somewhere online and run it via

wget -qO- https://example.com/script.sh | sudo bash

Answer 2

Based on the advice by @user535733, I ended up doing the following:

1. Install Ubuntu/Lubuntu on a flash drive

These instructions loosely follow this answer.

(Preparation)

• Create a live USB or DVD.
• Turn off and unplug the computer.
• Unplug all hard drives from your computer.
• Plug the computer back in.
• Disable secure boot and fast boot in UEFI settings (the exact steps for this are vendor-specific)

(Run Installer)

• Insert the Live USB or Live DVD.
• Start the computer, the USB/DVD should boot.
• Select Install Ubuntu.
• Insert a new target drive to install Ubuntu.
• Follow the instructions.
• Do not install "third-party software", as it may not work on other machines.
• At "Installation type" select "Something else".
• Select "Continue".
• Select target drive (in my case /dev/sdb)
• Select "New Partition Table" and "Continue".

(EFI partition)

• Click "Free space" and "+".
• "Size": 100 MB
• Select "Primary"
• "Location": "Beginning of this space"
• "Use as": "EFI System Partition"
• Select "OK"

(/ partition)

• Click "free space" and then "+".
• "Size": 7800 MB (or something else)
• Select "Primary"
• "Location": "Beginning of this space"
• "Use as": "Ext4"
• Mount point: "/"
• Select "OK"

(Important)

• Ensure "Device for boot loader installation" points to the root of the flash drive (/dev/sdb in my case).

• Click "Install Now" and "Continue".

• Follow the instructions.
• Wait until install is complete.

(Wrap up)

• Turn off computer and plug in the HDD.

2. Save the resulting flash drive

Next, save the resulting image to your machine. I wrote a script that handles this step.

#!/bin/bash
#
# Saves flash drive partitions to disk
#
# Example Usage:
# ./copy.sh /dev/sde ./images

# enable bash strict mode
# http://redsymbol.net/articles/unofficial-bash-strict-mode/
set -euo pipefail
IFS=$'\n\t'

##########
# INPUTS #
##########

# Checking inputs
if [ $# -ne 2 ]; then
  echo "USAGE: ./copy.sh DEVICE OUTPUTDIR"
  echo "Available devices:"
  sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL,UUID,MODEL
  exit
fi

# parse inputs
DEVICE=$1
OUTPUTDIR=$2
EFIFILE=$OUTPUTDIR/ubuntu-efi.img
ROOTFILE=$OUTPUTDIR/ubuntu-root.img
PARTITIONFILE=$OUTPUTDIR/partitions.txt

# Display helpful information
echo "Saving device $DEVICE to $OUTPUTDIR:"
sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL,UUID,MODEL $DEVICE

# Confirm
read -r -p "Continue? [y/N] " response
if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
    echo "Start copying (this could take a long time)..."
else
    echo "Aborting" >&2
    exit
fi

mkdir -p "$OUTPUTDIR"

########
# SAVE #
########

# record sizes
sudo fdisk -l | grep ${DEVICE} | sed "s#${DEVICE}#/dev/sdX#g" > $PARTITIONFILE

# dd: copy and convert
# if: source disk
# bs: sector size value from fdisk output
# count: last "end" sector from fdisk output incremented by one
# conv=sync,noerror: sync I/O and don't stop in case of errors on the source disk
sudo dd if=${DEVICE}1 conv=sync,noerror of=$EFIFILE
sudo dd if=${DEVICE}2 conv=sync,noerror of=$ROOTFILE

3. Install to New Drive

Finally, install the saved image to new flash drives. Again, I wrote a script that handles this step.

#!/bin/bash
#
# Installs Ubuntu/Lubuntu to flash drive
#
# Example Usage:
# ./copy.sh /dev/sde ./images

###########
# HELPERS #
###########

# enable bash strict mode
# http://redsymbol.net/articles/unofficial-bash-strict-mode/
set -euo pipefail
IFS=$'\n\t'

# directory containing this script
BASEDIR="$( dirname "$0")"

# run commands at the end of the script (even on errors)
EXIT=""
function addExit {
    # call the argument before ending the script
    EXIT="$@ ; $EXIT"
    trap "echo 'Final tasks before exit...' ; $EXIT" EXIT HUP TERM INT QUIT
}

##########
# INPUTS #
##########

# Checking inputs
if [ $# -ne 2 ]; then
  echo "USAGE: ./copy.sh DEVICE SOURCEDIR"
  echo "Available devices:"
  sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL,UUID,MODEL
  exit
fi

# parse inputs
DEVICE="$1"
SOURCEDIR="$2"
MNT=/mnt

# Display helpful information
echo "Installing to device $DEVICE:"
sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL,UUID,MODEL $DEVICE

# Confirm
read -r -p "Continue? [y/N] " response
if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
    echo "Preparing partitions..."
else
    echo "Aborting" >&2
    exit
fi

# record sizes
PARTITIONFILE="$SOURCEDIR/partitions.txt"
EFISTART=$( cat "$PARTITIONFILE" | grep /dev/sdX1 | awk -F" "  '{ print $2 }')
EFIEND=$(   cat "$PARTITIONFILE" | grep /dev/sdX1 | awk -F" "  '{ print $3 }')
ROOTSTART=$(cat "$PARTITIONFILE" | grep /dev/sdX2 | awk -F" "  '{ print $2 }')
ROOTEND=$(  cat "$PARTITIONFILE" | grep /dev/sdX2 | awk -F" "  '{ print $3 }')

#######################
# COPY TO FLASH DRIVE #
#######################

# umount all paritions of this device
echo "Unmounting all partitions of $DEVICE..."
sudo umount $DEVICE?* || true

# create partition table
sudo wipefs --all $DEVICE
sudo parted $DEVICE mklabel gpt

# efi partition
sudo parted $DEVICE mkpart primary fat32 ${EFISTART}s ${EFIEND}s
sudo parted $DEVICE set 1 boot on
sudo parted $DEVICE set 1 esp on

echo "Start copying efi..."
EFIFILE="$SOURCEDIR/ubuntu-efi.img"
sudo dd if="$EFIFILE" of=${DEVICE}1

# root partition
sudo parted $DEVICE mkpart primary ext4 ${ROOTSTART}s ${ROOTEND}s
# sudo parted $DEVICE name 2 UbuntuUSB

echo "Start copying root (this could take a long time)..."
ROOTFILE="$SOURCEDIR/ubuntu-root.img"
sudo dd if="$ROOTFILE" of=${DEVICE}2

##################
# PREPARE CHROOT #
##################
# prepare running command in root directory of new installation

sudo mount ${DEVICE}2 $MNT
addExit "sudo umount $MNT"

sudo mount ${DEVICE}1 $MNT/boot/efi
addExit "sudo umount $MNT/boot/efi"

for i in /dev /dev/pts /proc /sys; do
    sudo mount -B $i ${MNT}$i
    addExit "sudo umount ${MNT}$i"
done
sudo cp /etc/resolv.conf $MNT/etc/
modprobe efivars

########
# GRUB #
########

# update grub
sudo chroot $MNT grub-install -d /usr/lib/grub/x86_64-efi --efi-directory=/boot/efi/ --removable ${DEVICE}

#############
# CUSTOMIZE #
#############

# prepare scripts
sudo cp -r "${BASEDIR}/customize" "$MNT/opt/customize"

# install software
sudo chroot $MNT "/opt/customize/software.sh"

This script also installs additional custom software, using the following script in ./customize/software.sh

#!/bin/bash

# enable bash strict mode
# http://redsymbol.net/articles/unofficial-bash-strict-mode/
set -euo pipefail
IFS=$'\n\t'

###########
# PREPARE #
###########

# avoid locale issues and in order to import GPG keys
export HOME=/root
export LC_ALL=C

#######
# APT #
#######

apt-get update -y
apt-get clean
apt-get -y autoremove --purge

####################
# INSTALL SOFTWARE #
####################

# -y: auto-confirm
# -qq: quiet installation
apt-get install -y -qq \
    htop \
    gparted \
    build-essential