How To Create a Bootable Clone of boot disk

Question

In my environment I have 2 MAC Minis, a Windows10 machine, and Ubuntu 20.04 LTS. On the MAC minis, I run SuperDuper to create a bootable clone on an external hard disk. Is it possible to do the same thing on Ubuntu? I saw an earlier answer to this, but it involved many steps monkeying around with dd, /etc/fstab and grub, and is several years old. Is there a new and better way? I considered Clonezilla but it doesn't specifically say on their web page that it will create a bootable clone. I want my solution for the Ubuntu machine to work like SuperDuper does for my MACs.

Answer 1

My first answer does not clone the system in general. I have succeeded in cloning everything and making it bootable. (This is for EFI|UEFI, including Mac, not MBR booting.)

The device configuration:

mac2011-linux% lsblk -f|grep -v loop
NAME FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
sda                                                                                           
├─sda1 vfat     FAT32 EFI                 67E3-17ED                             172.8M    12% /boot/efi
├─sda2 hfsplus        SSD 1T              0492d65c-8b14-356a-9d3d-337f44e8d58a                
├─sda3 hfsplus        Recovery HD         87867aee-1d5c-3c9e-ae36-8d2df5c162f1                
└─sda4 ext4     1.0                       dc68435c-f80c-4beb-a09b-69015ad516e6  161.5G    60% /
sdb                                                                                           
├─sdb1 vfat     FAT32 EFI                 B4A1-A82A                             172.8M    12% /media/alba/EFI
├─sdb2 ext4     1.0   Ubuntu backup       e370d38b-7cdf-4a2a-b9e5-cc4ff1723f51    165G    59% /media/alba/Ubun
├─sdb4 hfsplus        Mac OS X El Capitan 3771b656-357f-3ac4-a993-704353fc89b7                
└─sdb5 hfsplus        Recovery HD         1925d052-aca6-307a-8025-ad6321a399f4 

Check that all the above four partitions are mounted and mount them if necessary (with Ubuntu Disks).

sdb2 (Ubuntu backup) is the intended clone of the Ubuntu system and user files, shortly, the root partition. As sdb is an external device, sdb2 is mounted by default as /media/alba/Ubuntu backup. sdb1 is the intended clone of the EFI system partition, mounted as /media/alba/EFI.

Each partition must have a different UUID. FAT partition UUID can be changed only by formatting. The clone partitions should be large enough to contain all the files of the original partitions.

Clone the root partition:

sudo rsync --archive --executability --hard-links --update --verbose --xattrs --one-file-system --delete --progress --exclude={/etc/fstab,/boot/grub/grub.cfg,/export,/home/alba/.cache,"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*",/lost+found} / /media/alba/Ubuntu\ backup

The contents of /dev etc. do not need to be copied because they are useless or should not be copied because Ubuntu creates them. However Ubuntu won't boot if /dev etc. do not exist, so only /dev/* not /dev etc. are excluded.

I also exclude /export that is a NFS mount in my configuration. Not excluding /export may cause duplicates.

Copy /etc/fstab from original to clone:

mac2011-linux% sudo cp /etc/fstab /media/alba/Ubuntu\ backup/etc/fstab

In /media/alba/Ubuntu\ backup/etc/fstab, replace the original root and EFI system partition UUIDs by the UUIDs found in device configuration above, so that:

mac2011-linux% sudo cat /media/alba/Ubuntu\ backup/etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda4 during installation
UUID=e370d38b-7cdf-4a2a-b9e5-cc4ff1723f51 /               ext4    errors=remount
-ro 0       1
# /boot/efi was on /dev/sda1 during installation
UUID=B4A1-A82A  /boot/efi       vfat    umask=0077      0       1
/swapfile                                 none            swap    sw            
  0       0

Copy the EFI system partition:

mac2011-linux% sudo -i                                        
root@mac2011-linux:~# cp -r /boot/efi/* /media/alba/EFI

Edit /media/alba/EFI/EFI/ubuntu/grub.cfg to define root partition, so that:

root@mac2011-linux:~# cat /media/alba/EFI/EFI/ubuntu/grub.cfg
search.fs_uuid e370d38b-7cdf-4a2a-b9e5-cc4ff1723f51 root hd1,gpt2 
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

Partition numbers in GRUB are the same as in Ubuntu.

Copy /boot/grub/grub.cfg from original to clone (as you did for /etc/fstab) and make it writable:

root@mac2011-linux:~# chmod +w /media/alba/Ubuntu\ backup/boot/grub/grub.cfg

Replace everywhere gpt4 by gpt2 and original root partition UUID by clone root partition UUID.

Reset permission:

root@mac2011-linux:~# chmod -w /media/alba/Ubuntu\ backup/boot/grub/grub.cfg

Reboot, select the clone EFI in the firmware boot menu. If you made no mistake, then GRUB will launch Ubuntu, else it will prompt.

Remarks

I have checked in he GRUB shell that the GRUB device number hd0 used in /boot/grub/grub.cfg does not refer to any of the /dev/sdb?. It is not necessary to edit it.

I advise to disconnect the original at least once while booting the clone in order to make sure that the clone can boot by itself. If you don't want to open the computer case, you can just check that the original partitions can be unmounted after booting on the clone.

update-grub is not needed but once you have booted on clone you can update GRUB and reboot on clone to check that the clone is still bootable.

Using --delete-excluded in rsync would wipe out the edits of /media/alba/Ubuntu\ backup/etc/fstab and /media/alba/Ubuntu\ backup/boot/grub/grub.cfg.

In Bash script to backup/clone Ubuntu to another partition is a script that does very similar things.

It is better first to clone manually in order to understand and check every step, and only later script, according to the system peculiarities.

The copying and editing /etc/fstab, /boot/grub/grub.cfg and the EFI system partition should be repeated if those files change on the original. This happens during system updates.

Bibliography or references

Does rsync requires changing UUID after copying a partition?

Dual boot and the files /boot/grub/grub.cfg — which one is used?

UEFI

GRUB starts in command line after reboot

How to Rescue a Non-booting GRUB 2 on Linux

Bash script to backup/clone Ubuntu to another partition

File system cloning Arch Linux

Answer 2

Create the necessary partitions, set the labels are run the script with both devices attached.

#!/bin/zsh
# automates the method explained in
# https://askubuntu.com/questions/1267225/how-to-create-a-bootable-clone-of-boot-disk/1327623#1327623
# USE CAREFULLY, THIS IS INTRINSICALLY DANGEROUS SOFTWARE
# Pierre ALBARÈDE 2021/4/24
# tested with Ubuntu 20.10
Assumptions:
- /boot is not on a separate partition.
- root partition has label $UbuntuLabel and the first partition on the same device is the EFI system partition,
- root partition backup has label $UbuntuBackupLabel and the first partition on the same device is available for the EFI system partition backup,
- device names are like sd[a-z].
To do:
- pick and exclude duplicate mount (e. g. for NFS) from /etc/fstab to avoid overfilling the backup device,
-check that backup volume partition sizes are enough.
--- begin config
UbuntuLabel='Ubuntu'
UbuntuBackupLabel='Ubuntu backup'
logfile="rsync.log"
excluded from rsync
/swapfile can be big and transfering it is a waste of time,
not having a swapfile backup makes some errors at boot and shutdown.
excludedByUser='/export,"/home/*/.cache",/swapfile,$logfile'
excludedByUser='/export,"/home/*/.cache",$logfile'
--- end config
Partition attributes are named as in
% lsblk -PO /dev/sda1
[filtered]
NAME="sda1" PATH="/dev/sda1" FSTYPE="vfat" FSUSED="24.1M" FSUSE%="12%" FSVER="FAT32" MOUNTPOINT="/boot/efi" LABEL="EFI" UUID="67E3-17ED" PTUUID="b5ca7ab4-d564-49a1-a43b-d9125d4dcbab" PTTYPE="gpt" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PARTTYPENAME="EFI System" PARTLABEL="EFI System Partition" PARTUUID="05bdacf6-78ff-451d-8c4d-bb812fd68e83" PKNAME="sda"
Many attributes look similar, I use LABEL and UUID, not PARTLABEL and PTUUID, PARTUUID.
alias lsblkn='lsblk -n'
#########
TESTS
#########
Are all UUIDs distinct?
[[ -n $(lsblkn -o UUID|grep '\S'|sort|uniq --repeated) ]]&&{echo "Error: not all UUIDs are distinct.  Did you use dd?";exit 1}
Does only one partition exists with LABEL $UbuntuLabel?
[[ $(lsblkn -o LABEL|grep -x $UbuntuLabel|wc -l) != 1 ]]&&{echo "Error: $UbuntuLabel is not unique.";exit 1}
Does only one partition exists with LABEL $UbuntuBackupLabel?
[[ $(lsblkn -o LABEL|grep -x $UbuntuBackupLabel|wc -l) != 1 ]]&&{echo "Error: $UbuntuBackupLabel is not unique.";exit 1}
#######################################
DETERMINE PARTITION PATHS AND UUIDS
#######################################
paths for mount
UUIDs for (display and) replacement in config files
blkid (or findfs) finds only one occurrence, that is why I have checked unicity
UbuntuPath=$(blkid --label $UbuntuLabel)
UbuntuBackupPath=$(blkid --label $UbuntuBackupLabel)
devicePath=/dev/$(lsblkn -o PKNAME $UbuntuPath)
deviceBackupPath=/dev/$(lsblkn -o PKNAME $UbuntuBackupPath)
UbuntuUUID=$(lsblkn -o UUID $UbuntuPath)
echo "Ubuntu partition UUID:            $UbuntuUUID"
EFI is always first partition with number 1
EFIPath="$devicePath"1
EFIUUID=$(lsblkn -o UUID $EFIPath)
echo "EFI system partition UUID:        $EFIUUID"
UbuntuBackupUUID=$(lsblkn -o UUID $UbuntuBackupPath)
echo "Ubuntu backup partition UUID:     $UbuntuBackupUUID"
EFIBackupPath="$deviceBackupPath"1
EFIBackupUUID=$(lsblkn -o UUID $EFIBackupPath)
echo "EFI system partition backup UUID: $EFIBackupUUID"
determine GRUB partition number from Ubuntu partition name
on original
UbuntuGpt=$(echo $UbuntuPath|sed "s//dev/sd[a-z]/gpt/")
on backup
UbuntuBackupGpt=$(echo $UbuntuBackupPath|sed "s//dev/sd[a-z]/gpt/")
I don't know how to determine GRUB device number but it does not matter so it is not changed (it is 0).
#########
mount
#########
If we are here, Ubuntu must be mounted.
[[ -z $(findmnt $EFIPath) ]]&&{echo "Mounting EFI system partition.";udisksctl mount --block-device $EFIPath}
[[ -z $(findmnt $UbuntuBackupPath) ]]&&{echo "Mounting Ubuntu backup partition.";udisksctl mount --block-device $UbuntuBackupPath}
[[ -z $(findmnt $EFIBackupPath) ]]&&{echo "Mounting EFI system partition backup.";udisksctl mount --block-device $EFIBackupPath}
All 4 partition are now mounted.
determine destination mountpoints
UbuntuBackupMountPoint should normally be /media/$USER/$UbuntuBackupLabel
UbuntuBackupMountPoint=$(lsblkn -o MOUNTPOINT "$UbuntuBackupPath")
EFIBackupMountPoint=$(lsblkn -o MOUNTPOINT "$deviceBackupPath"1)
echo "You are going to backup to:"
echo $UbuntuBackupMountPoint
echo $EFIBackupMountPoint
read -q "REPLY?Confirm by typing y?"
[[ $REPLY != 'y' ]]&&{echo "\nExiting without file change.";exit 0}
echo "\nProceeding with rsync."
##########
BACKUP
##########
rsync would not transfer /etc/fstab and /boot/grub/grub.cfg and /media/$USER/EFI/EFI/ubuntu/grub.cfg after their backups have been edited and given a more recent date so I erase the backups. One could also cheat the modification time.
sudo rm $UbuntuBackupMountPoint{/etc/fstab,/boot/grub/grub.cfg}
sudo rm $EFIBackupMountPoint/EFI/ubuntu/grub.cfg
Backup root partition, including /etc/fstab /boot/grub/grub.cfg.
--archive = -rlptgoD
--recursive, -r          recurse into directories
--links, -l              copy symlinks as symlinks
--perms, -p              preserve permissions
--times, -t              preserve modification times
--owner, -o              preserve owner (super-user only)
--group, -g              preserve group
-D          <             equivalent to --devices --specials
--devices                preserve device files (super-user only)
To debug, replace "rsync" by "echo" or use '--dry-run.
Single quotes prevent eval to remove double quote around "/dev/*" and split words in '$UbuntuBackupLabel'.
eval sudo rsync --log-file=$logfile --archive --no-D --executability --hard-links --update --verbose --xattrs --one-file-system --delete --progress --exclude={$excludedByUser} --exclude={'"/dev/"','"/proc/"','"/sys/"','"/tmp/"','"/run/"','"/mnt/"','"/media/"','"/lost+found/"',$logfile} --delete-excluded / '$UbuntuBackupMountPoint'
backup the EFI system partition
sudo rsync --recursive --times --delete /boot/efi/ $EFIBackupMountPoint
###############
EDIT BACKUP
###############
sudo sed -i "s/$EFIUUID/$EFIBackupUUID/g;s/$UbuntuUUID/$UbuntuBackupUUID/g" $UbuntuBackupMountPoint/etc/fstab
grubcfg=$UbuntuBackupMountPoint/boot/grub/grub.cfg
sudo chmod +w $grubcfg
sudo sed -i "s/$UbuntuUUID/$UbuntuBackupUUID/g;s/$UbuntuGpt/$UbuntuBackupGpt/g" $grubcfg
sudo chmod -w $grubcfg
set root UUID in EFI
sed -i "s/$UbuntuUUID/$UbuntuBackupUUID/;s/$UbuntuGpt/$UbuntuBackupGpt/" $EFIBackupMountPoint/EFI/ubuntu/grub.cfg
exit 0

Answer 3

As for cloenzilla, check that the destination drive/partition is formatted. And then in Clonezilla choose expert mode and enable the “-icds” option. You need to also choose "Resize partitional table proportionally" option. Your external should come back if you turn off, remove cable, reboot, wait then reconect the external. Type 'lsblk' to confirm it's there.

Answer 4

I run SuperDuper to create a bootable clone on an external hard disk. Is it possible to do the same thing on Ubuntu?

Although there is no tool out there like CCC (Bombich) or Super Duper for Mac OS, I think I have done it roughly, at least with EFI booting and a single user.

Introduction

If you make a complete clone with dd then it will take a very long time so you can't do it too often (like every day). Moreover you may have trouble using both devices at the same time as they have initially the same identifiers (UUID).

I have read in Clonezilla documentation that it does not have the incremental feature so I did not search further. You need an incremental copy program, that is rsync.

Identity must be set consistently on each device for booting.

Also, in order to copy from original to clone, at least, both should be mounted simultaneously, so they should have different UUIDs.

Technical details

Device identity lies in exactly three locations:

1. /etc/fstab.
2. /boot/efi/EFI/ubuntu/grub.cfg
3. /boot/grub/grub.cfg.

/boot/efi is just a mount of the EFI system partition = EFS.

How to

On the intended clone, create a FAT partition about same size as the original EFS. Copy the EFS with cp -r: there are few small files and they will rarely change. As FAT does not know about permissions, so you don't have to worry about them.

Also create a Linux partition, similar to the one you want to clone (e. g. ext4).

Create a Ubuntu live USB, as explained elsewhere. You can use Startup Disk Creator.

Power off. Disconnect you original device from your computer, to make sure it won´t be corrupted (this requires some screwdriver playing).

Connect the intended clone. Install Ubuntu thereon from the Ubuntu live USB, creating one user with the same name as original.

If you have files from a previous attempt, in order to save time, choose "Something else" in the install menu and do not reformat.

The Ubuntu installer with hopefully fill correctly the EFS, /etc/fstab, /boot/efi and /boot/grub.

Check that Ubuntu boots on the intended clone.

Power off. Disconnect the intended clone. Reconnect the original. Reboot. Reconnect the intended clone (externally, by USB or Firewire). With Disks, check the intended clone mount and set $MOUNT to it. Fill the intended clone with

sudo rsync -aEHuvX --progress --exclude={"/etc/fstab","/export",".cache","/dev","/proc","/sys","/tmp","/run","/mnt","/media","/lost+found","/cdrom","/srv","/swapfile","/boot"} / $MOUNT

/etc/fstab /boot must be excluded to preserve clone boot-ability. Other excludes concern temporary, duplicate, auxiliary files that should not be copied because that will take time or may cause trouble. You can add --delete to save space.

Power off. Replace original by intended clone. Check reboot and clone functionality.

Defects found on the clone

• Evolution fails : "evolution-alarm-notify crashed" and "settings schema not installed".

• Login shell on clone is bash, whereas it was zsh on original.

• Many Gnome desktop shortcuts don't work.

I missed something to copy correctly user database or settings.

Please improve.

Answer 5

I don't know if you have a big backup disk, but if you have, you can copy by ISO using GNOME Disks.

step 1: create a disk image on your backup disk

step 2: restore disk image on your new disk

I hope this helps

Answer 6

What the original poster wanted and what I actually did is a bootable BACKUP, not exactly a clone, because the UUIDs on the backup differ from the UUIDs on the original. On Mac OS, CCC and Super Duper must work likewise, because both the original must be mounted simultaneously to partly update the backup with rsync (or cp).

If you really you want a clone, you can use dd but it can take long (depending on your interface speed and the data size -- through Firewire so it takes hours to move 300 GB). The outcome of dd is uncertain because it is very low level and it can be dangerous. Also, it will reproduce the original with all sorts of useless or annoying details, while rsync will somehow clean the file system (e. g. not copy a broken file). The clone should be good to fully replace the original but not good to mount alongside to recover a lost file.

For some reason that I do not understand, few Linux people are interested in bootable backup or want to communicate about it.

However, backing up in a few seconds and having a backup that can be booted in a minute is a great security. Reinstalling a complete Linux with all my customizations and programs takes me maybe one or two days, or can be a kill, if I am on a trip or if I have a deadline.

The backup script I have posted is roughly correct (although it can be improved greatly and lacks generality). It worked with Ubuntu until July 2022 (up to this date, it was 22.04).

I moved to Fedora 36 in August 2022 for other reasons. I adapted the backup script. At the beginning of September, it ceased to work. I checked again in Ubuntu (up to date) and it did not work either, while it still worked on a July Ubuntu backup.

I upgraded Fedora to 37 (F37) some weeks ago. I returned to the backup problem, making tests with two machines other than my principal machine, as these tests are very dangerous, all the more because I don't have a bootable backup! This can be a costly hobby, as three computers are needed to be comfortable.

I discovered than I could no more install F37 on my Macbook pros. I also discovered that F37 is not portable, that is, if I move it to another machine, it won't boot. Also, Fedora puts a lot of things in the EFS and uses the HFS+ format for the EFS (while even Apple uses FAT).

Considering the complex situation in Fedora, I began trying Manjaro a few days ago. Manjaro's EFS is very simple (containing only two useful files). The backup script works again on a fresh installation of Manjaro. I have moved all my stuff to Manjaro and checked that the backup script still works.

The bootable backup topic is not specific to Ubuntu, so it should go to the Unix Stack Exchange. I may have some time to share on this topic.

Answer 7

Thanks for your update. I'm glad to see that there is some interest in creating a bootable backup. I also thought your comments on Fedora and Manjaro were timely.

Frankly, I haven't played with Ubuntu since my original post. I haven't had much luck with getting things like the DVD player (the one with the orange traffic cone) to work properly, and if I can't make a bootable backup, then, frankly, there's not much point to continuing ...

... although I would like to see an article on taking an image. If I can create an installation DVD with the software to restore an image from an external drive to the hard disk, that would be acceptable, because then the process would work like Windows. Not great, mind you, but acceptable.