The neat part
If there is no way to restore the list of packages, I could recreate
the list of installed packages by reinstalling them all.
There are multiple copies/backups(automatic) of the dpkg status/database.
Is there maybe a scanner which loops over the available packages and
searches for their files on the local system?
There is a more efficient log file at /var/log/dpkg.log that contains packages ever handled by dpkg along with their status and if you filter that file with like grep “install” you get a list of all the packages installed by dpkg.
Or is there a list of packages which are commonly installed?
openSuSE's YAST for example has patterns which are groups of packages
to install eg. base, Laptop, Network and Server
Yes, one meta package ubuntu-desktop.
I know that if everything fails I could click through all packages in
Synaptic and guess what might have been installed, but I'm not very
keen on doing so.
No need for that, there are dpkg options to fix database corruption.
Is there a way to recreate the list of installed packages?
(you mean verify/check as the database/files are still there) Yes,in most corruption cases.
You need to, first, run:
sudo dpkg --configure -a
then, run:
sudo dpkg --verify
then, run:
sudo dpkg --audit
finally, run:
sudo apt update
From man dpkg:
--configure package...|-a|--pending Configure a package which has been unpacked but not yet configured. If -a or --pending is
given instead of package, all unpacked but unconfigured packages are
configured.
-V, --verify [package-name...] Verifies the integrity of package-name or all packages if omitted, by comparing information
from the files installed by a package with the files metadata
information stored in the dpkg database (since dpkg 1.17.2). The
origin of the files metadata information in the database is the
binary packages themselves. That metadata gets collected at package
unpack time during the installation process.
Currently the only functional check performed is an md5sum
verification of the file contents against the stored value in the
files database. It will only get checked if the database contains
the file md5sum. To check for any missing metadata in the database,
the --audit command can be used.
-C, --audit [package-name...] Performs database sanity and consistency checks for package-name or all packages if omitted (per
package checks since dpkg 1.17.10). For example, searches for
packages that have been installed only partially on your system or
that have missing, wrong or obsolete control data or files. dpkg
will suggest what to do with them to get them fixed.
The messy part
Last resort, if the above didn't work, then the dpkg database(status) is seriously damaged beyond repair or even empty or missing. In this situation first backup whatever you have if any like so:
sudo mv /var/lib/dpkg/status /var/lib/dpkg/my-old-status
Then, try restoring from /var/lib/dpkg/status-old like so:
sudo cp /var/lib/dpkg/status-old /var/lib/dpkg/status
That file holds a near recent copy(around one day old).
If that was not successful, then look under /var/backups/ like so:
ls -lh /var/backups/ | grep status
Start by restoring the most recent one dpkg.status.0 like so:
sudo cp /var/backups/dpkg.status.0 /var/lib/dpkg/status
Then see how things go and if needed restore older backups(these are compressed .gz files so extract/decompress them first) one by one until things are back to normal(no errors when running sudo apt update)
Please, notice as well that other related files are also backed up and might be restored as well if restoring status alone is not enough to resolve the issue. You can list them with:
ls -lh /var/backups/ | grep dpkg
To know what you are missing, here are the files and directories that normally live under /var/lib/dpkg/:
$ ls -lh /var/lib/dpkg/
total 5.4M
drwxr-xr-x 2 root root 4.0K Jun 11 12:41 alternatives
-rw-r--r-- 1 root root 11 Jan 8 02:35 arch
-rw-r--r-- 1 root root 178K Aug 19 2021 available
-rw-r--r-- 1 root root 8 Aug 19 2021 cmethopt
-rw-r--r-- 1 root root 616 Jan 8 02:38 diversions
-rw-r--r-- 1 root root 671 Jan 8 02:38 diversions-old
drwxr-xr-x 2 root root 436K Jun 16 10:54 info
-rw-r----- 1 root root 0 Jun 16 14:36 lock
-rw-r----- 1 root root 0 Jun 16 14:36 lock-frontend
drwxr-xr-x 2 root root 4.0K Mar 23 2020 parts
-rw-r--r-- 1 root root 270 Jan 8 13:40 statoverride
-rw-r--r-- 1 root root 309 Jan 8 13:40 statoverride-old
-rw-r--r-- 1 root root 2.4M Jun 16 10:54 status
-rw-r--r-- 1 root root 2.4M Jun 16 10:54 status-old
drwxr-xr-x 2 root root 4.0K Jun 11 12:40 triggers
drwxr-xr-x 2 root root 4.0K Jun 16 14:36 updates
The dirty part
If you get here, you are most likely out of ideas/choices and on the verge of committing a fresh Ubuntu install ... Let's reach an agreement first; I don't know you and you don't know me (you are totally on your own) ... This is a dirty business(yours) and you should deal with it alone :) ... The process ahead will parse the contents of /var/log/dpkg.log(assuming you haven't blown up this one yet) into package names and feed them to apt for re-installation ... This should rebuild your dpkg status/database even if it is severely damaged(which it is) ... If you have other choices, please take them now ... "no other choices" ... let's get to business:
Run all the four commands in the neat part above, one by one and in the same order (again).
Reinstall the supposedly(assuming you did not install then remove a package/s) installed/half-installed packages by parsing the contents of /var/log/dpkg.log into package names and feeding them to apt install -y like so(This is working but, not optimal ... I strongly urge you to improve it if you can):
awk -F' |:' '/installed/ { if ( !seen[$7]++ ) print $7 }' /var/log/dpkg.log | xargs sudo apt install -y
Deal with errors(oh yeah, there will be errors), fix them and start over until all packages are installed with no errors ... Good luck(you're gonna need it).
sudo dpkg --configure -aneeds to be run. – Raffa Jun 16 '22 at 09:17sudo dpkg --verifyor evensudo dpkg --auditshould be run in that order. – Raffa Jun 16 '22 at 09:25/var/lib/dpkg/status-old– nobody Jun 16 '22 at 11:02/var/lib/dpkg/status-oldcontains the same as dpkg -l returns – Arigion Jun 16 '22 at 11:45