/etc/hosts file overwritten in persistent live USB

Question

This page https://help.ubuntu.com/community/mkusb encourages to create persisten live bootable USB. So that's what I suggested to my class, when I had to suggest a way how they could enjoy Ubuntu Linux without disturbing the school's computer default OS. So now they can boot Ubuntu 18.04 from their USB pen drives.

When teaching them how to set up Apache web server, it was discovered that the changes made to /etc/hosts file were not preserved between boots. The config files under /etc/apache2/sites-enabled and the site files under /var/www where preserved, though.

What could be going on here? Mkusb is using casper-rw partition in order to establish the persistence. Why would the persistence work on one file and not work on another? Or perhaps the /etc/hosts file is rewritten by some default live USB instruction that should be removed?

Some system settings are set up and some programs are started before the overlay system for persistence has started. This happens for the linux kernel and for the kernel's hardware device drivers. Maybe this is also what is happening to the /etc/hosts file. I have tampered with that file in installed systems, but not in persistent live systems, so I don't really know, if there is an easy solution to the problem. — sudodus, Feb 08 '19 at 23:43
An obvious, but not so easy solution is to install Ubuntu (install like into an internal drive) into a USB drive. Such a system will behave like any installed system. See this link and links from it. -- Another alternative might be to [automatically] backup /etc/hosts to a 'safe place', and at boot restore it [also automatically]. This could be done with scripts and maybe involve cron. — sudodus, Feb 08 '19 at 23:51
If you don't want identical installed systems, it should be possible to set up a master installed system with the OEM method, clone it and then finish the installation in each computer making the systems unique with user names and passwords, host names etc. — sudodus, Feb 09 '19 at 00:09

score 1 · Answer 1 · edited Jun 12 '20 at 14:37

General discussion about persistent live systems

Some system settings are set up and some programs are started before the overlay system for persistence has started. This happens for the linux kernel and for the kernel's hardware device drivers. Maybe this is also what is happening to the /etc/hosts file. I have tampered with that file in installed systems, but not in persistent live systems, so I don't really know, if there is an easy solution to the problem.

Possible solutions

An obvious, but not so easy solution is to install Ubuntu (install like into an internal drive) into a USB drive. Such a system will behave like any installed system. See this link and links from it.

If you don't want identical installed systems, it should be possible to set up a master installed system with the OEM method, clone it and then finish the installation in each computer making the systems unique with user names and passwords, host names etc.
Another alternative might be to [automatically] backup /etc/hosts to a 'safe place', and at boot restore it [also automatically]. This could be done with scripts and maybe involve cron. But there should be a reason for /etc/hosts to bounce back (unless it is a bug). Maybe things involving the network and /etc/hosts happen early about in the boot process, before the overlay system is activated, and in that case, this workaround will not work.
Yet another alternative is to try a persistent live system made from another linux distro in order to check if the /etc/hosts bounces back to default after reboot.

Test with Debian 9.6.0

mkusb can create persistent live drives from Ubuntu and Debian iso files. So it is natural for me to try a Debian persistent live system. I have the iso file

debian-live-9.6.0-amd64-cinnamon.iso

which is new (downloaded January 2019). I made and tested a persistent live system, and it preserves /etc/hosts after reboot :-)

user@debian:~$ cat /etc/hosts
127.0.0.1   localhost debian
192.168.0.4 xw8400
::1     localhost ip6-localhost ip6-loopback
ff02::1     ip6-allnodes
ff02::2     ip6-allrouters
user@debian:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.6 (stretch)
Release:    9.6
Codename:   stretch
user@debian:~$ uname -a
Linux debian 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux
user@debian:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            7.8G     0  7.8G   0% /dev
tmpfs           1.6G  9.4M  1.6G   1% /run
/dev/sdb4       2.0G  2.0G     0 100% /lib/live/mount/persistence/sdb4
/dev/loop0      1.9G  1.9G     0 100% /lib/live/mount/rootfs/filesystem.squashfs
tmpfs           7.8G     0  7.8G   0% /lib/live/mount/overlay
/dev/sdb5       8.4G  2.3G  5.8G  28% /lib/live/mount/persistence/sdb5
overlay         8.4G  2.3G  5.8G  28% /
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
tmpfs           7.8G  4.0K  7.8G   1% /tmp
tmpfs           1.6G   20K  1.6G   1% /run/user/1000
/dev/sdb1       4.2G   23M  4.2G   1% /media/user/usbdata
user@debian:~$ sudo lsblk -fm
NAME   FSTYPE   LABEL                 UUID                                 MOUNTPOINT                                 NAME     SIZE OWNER GROUP MODE
loop0  squashfs                                                            /lib/live/mount/rootfs/filesystem.squashfs loop0    1.8G root  disk  brw-rw----
sda                                                                                                                   sda    238.5G root  disk  brw-rw----
├─sda1 ext4     root                  2093f8d6-7840-4256-8edc-4db97e865784                                            ├─sda1 232.5G root  disk  brw-rw----
└─sda2 swap     swap                  4b882c9f-4867-4c5c-8eb7-c84ef03f4786                                            └─sda2     6G root  disk  brw-rw----
sdb                                                                                                                   sdb     14.9G root  disk  brw-rw----
├─sdb1 ntfs     usbdata               51B99954568550BA                     /media/user/usbdata                        ├─sdb1   4.2G root  disk  brw-rw----
├─sdb2                                                                                                                ├─sdb2     1M root  disk  brw-rw----
├─sdb3 vfat     usbboot               3FAC-E416                                                                       ├─sdb3   244M root  disk  brw-rw----
├─sdb4 iso9660  d-live 9.6.0 ci amd64 2018-11-10-11-54-14-00               /lib/live/mount/persistence/sdb4           ├─sdb4     2G root  disk  brw-rw----
└─sdb5 ext4     persistence           9d044926-15cd-4e1b-911d-ceb8e7101cf3 /lib/live/mount/persistence/sdb5           └─sdb5   8.5G root  disk  brw-rw----
sr0                                                                                                                   sr0     1024M root  cdrom brw-rw----

I don't know if things will work as they should with Debian persistent live and your Apache web server,

if /etc/hosts and the corresponding network settings work as intended, and
if other things in Debian, for example hardware drivers work with your computers,

but I think it is worth trying with Debian.

There's something called casper-snapshot, which creates a casper-sn file which may do the trick. It's part of the casper package, check the man pages for it. — ubfan1, Feb 09 '19 at 05:20
Installing Ubuntu on USB stick as on a normal hdd could be an option, but carrying it out ir much much painful as the simple mkusb operation. And there are many ways how one can screw up their computer while doing it. That's the main reason I chose the LiveUSB+overlay approach. — Passiday, Feb 09 '19 at 13:35
@Passiday, 1. Will you try with Debian 9.6.0 persistent live? 2. You can make one installed Ubuntu system, create a compressed image using dd | xz or similar, and let the students clone to their USB pendrives (clone with mkusb), which should make the student task as safe as making a persistent live drive. — sudodus, Feb 09 '19 at 17:52
Well, I am hesitant to propose them a solution that would mean reinstalling and reconfiguring the apps they have already started using. So I am looking for a way how to deal with the specific problem at hand. Perhaps you are right about the simplicity of cloning the bootable USBs, once one is succesfully created and configured, maybe I'll give it a try. — Passiday, Feb 09 '19 at 22:19
@Passiday, I might have some already prepared compressed image file for you; 1. What size (in GB) pendrives are your students using for this purpose? 2. Are the student computers running in UEFI mode or BIOS/CSM/legacy mode or both? — sudodus, Feb 09 '19 at 22:36
@sudodus I appreciate your initiative, please don't take this personally, but that would be against reasonable security policy to accept a ready-made image and pass it to my students. — Passiday, Feb 09 '19 at 23:01
@Passiday, I understand and accept that. Good luck trying along the two tracks that you intend to try :-) — sudodus, Feb 09 '19 at 23:21

score 1 · Accepted Answer · answered Apr 05 '19 at 06:14

For a live/install iso the system boots before persistent data is mounted, so you might need to copy those /etc/hostname and /etc/hosts to the home directory, edit them and add commands in /etc/rc.local to copy them back to /etc and set hostname dynamically

It's actually interesting so many files are persistent and hosts and hostname are not

sudo -i
cp /etc/hosts /home/
cp /etc/hostname /home/
  # edit both files and save 
nano /home/hosts
nano /home/hostname
  # make sure they are root
chown root:root /home/hosts
chown root:root /home/hostname

nano /etc/rc.local
  # add those lines before line exit 0
cp /home/hostname /etc/
cp /home/hosts /etc/
hostname $(cat /etc/hostname)
  #save file

Reboot and check, should be fine.

There are similar solutions that keep files in user specific dir, but with my approach you can delete ubuntu or mint user (after creating new one and adding to sudo group) and you don't have to worry about losing hosts files

That's exactly what I did in my case, thanks! – Passiday Apr 08 '19 at 10:51 — Passiday, Apr 08 '19 at 10:51

/etc/hosts file overwritten in persistent live USB

2 Answers2

General discussion about persistent live systems

Possible solutions

Test with Debian 9.6.0