after upgrade from Ubuntu 18.04 to 20.04: 'apt-get update' fails with dpkg "pre-removal script subprocess was killed by signal (Terminated)"

Question

After a upgrade from 18.04 to 20.04, many 'apt' commands fail: apt-get update, apt-get remove pritunl-client-gtk (same for purge), apt-get --fix-broken install, apt-get autoremove.

They all fail with: installed pritunl-client-gtk package pre-removal script subprocess was killed by signal (Terminated).

Here are some outputs (shortened with [...]):

:~$ sudo apt-get update
Hit:1 http://ubuntu.ethz.ch/ubuntu focal InRelease
Ign:2 http://dl.google.com/linux/chrome-remote-desktop/deb stable InRelease                                                                                                           
[...]
Get:26 http://ubuntu.ethz.ch/ubuntu focal-security/universe amd64 DEP-11 Metadata [55.7 kB]
Fetched 3,242 kB in 2s (1,802 kB/s)                                        
appstreamcli: symbol lookup error: appstreamcli: undefined symbol: AS_APPSTREAM_METADATA_PATHS
Reading package lists... Done
N: Skipping acquire of configured file 'stable/binary-i386/Packages' as repository 'https://download.docker.com/linux/ubuntu focal InRelease' doesn't support architecture 'i386'
N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://download.mono-project.com/repo/ubuntu stable-focal InRelease' doesn't support architecture 'i386'
E: Problem executing scripts APT::Update::Post-Invoke-Success 'if /usr/bin/test -w /var/cache/app-info -a -e /usr/bin/appstreamcli; then appstreamcli refresh-cache > /dev/null; fi'
E: Sub-process returned an error code

:~$ sudo apt-get remove pritunl-client-gtk
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  [...]
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED
  pritunl-client-gtk
0 to upgrade, 0 to newly install, 1 to remove and 974 not to upgrade.
After this operation, 1,397 kB disk space will be freed.
Do you want to continue? [Y/n] 
(Reading database ... 335530 files and directories currently installed.)
Removing pritunl-client-gtk (1.0.1865.25-0ubuntu1~xenial) ...
dpkg: error processing package pritunl-client-gtk (--remove):
 installed pritunl-client-gtk package pre-removal script subprocess was killed by signal (Terminated)
dpkg: too many errors, stopping
Errors were encountered while processing:
 pritunl-client-gtk
Processing was halted because there were too many errors.
E: Sub-process /usr/bin/dpkg returned an error code (1)

:~$ sudo apt --fix-broken install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
   [...]
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED
  pritunl-client-gtk
0 to upgrade, 0 to newly install, 1 to remove and 974 not to upgrade.
After this operation, 1,397 kB disk space will be freed.
Do you want to continue? [Y/n] 
(Reading database ... 335530 files and directories currently installed.)
Removing pritunl-client-gtk (1.0.1865.25-0ubuntu1~xenial) ...
dpkg: error processing package pritunl-client-gtk (--remove):
 installed pritunl-client-gtk package pre-removal script subprocess was killed by signal (Terminated)
dpkg: too many errors, stopping
Errors were encountered while processing:
 pritunl-client-gtk
Processing was halted because there were too many errors.
E: Sub-process /usr/bin/dpkg returned an error code (1)

As suggested in other questions, I tried to fix libappstream:

:~$ dpkg -l '*libappstream*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                     Version      Architecture Description
+++-========================-============-============-==========================================
ii  libappstream-glib8:amd64 0.7.7-2      amd64        GNOME library to access AppStream services
ii  libappstream4:amd64      0.12.10-2    amd64        Library to access AppStream services
:~$ sudo apt install --reinstall libappstream4 
Reading package lists... Done
Building dependency tree

Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies.
 pritunl-client-gtk : Depends: python-appindicator but it is not installable
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

I had already tried --fix-broken install (see above) so I tried:

~$ sudo apt-get install python-appindicator
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package python-appindicator is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'python-appindicator' has no installation candidate

So there is definitely a problem with a missing python-appindicator (also suggested here). I cannot tell whether this is related to the terminating --remove.

Questions (answering one of them would probably be sufficient):

• How can I find out why the pre-remove fails? Can I skip the pre-remove?
• How can I force remove a package (pritunnel-clinet-gtk)?
• Where can I find python-appindicator for Ubuntu 20.04?

I found several similar problem, but neither of them appears to be failing during pre-remove, so their solutions don appear to be applicable, see for example here, here or here.

EDIT

As requested in the comments, here is my repo listing:

:~$ grep -r deb /etc/apt/sources.list /etc/apt/sources.list.d
/etc/apt/sources.list:# deb cdrom:[Ubuntu 18.04.1 LTS _Bionic Beaver_ - Release amd64 (20180725)]/ bionic main restricted
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal main restricted
/etc/apt/sources.list:# deb-src http://gb.archive.ubuntu.com/ubuntu/ bionic main restricted
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal-updates main restricted
/etc/apt/sources.list:# deb-src http://gb.archive.ubuntu.com/ubuntu/ bionic-updates main restricted
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal universe
/etc/apt/sources.list:# deb-src http://gb.archive.ubuntu.com/ubuntu/ bionic universe
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal-updates universe
/etc/apt/sources.list:# deb-src http://gb.archive.ubuntu.com/ubuntu/ bionic-updates universe
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal multiverse
/etc/apt/sources.list:# deb-src http://gb.archive.ubuntu.com/ubuntu/ bionic multiverse
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal-updates multiverse
/etc/apt/sources.list:# deb-src http://gb.archive.ubuntu.com/ubuntu/ bionic-updates multiverse
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal-backports main restricted universe multiverse
/etc/apt/sources.list:# deb-src http://gb.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse
/etc/apt/sources.list:# deb http://archive.canonical.com/ubuntu bionic partner
/etc/apt/sources.list:# deb-src http://archive.canonical.com/ubuntu bionic partner
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal-security main restricted
/etc/apt/sources.list:# deb-src http://security.ubuntu.com/ubuntu bionic-security main restricted
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal-security universe
/etc/apt/sources.list:# deb-src http://security.ubuntu.com/ubuntu bionic-security universe
/etc/apt/sources.list:deb http://ubuntu.ethz.ch/ubuntu/ focal-security multiverse
/etc/apt/sources.list:# deb-src http://security.ubuntu.com/ubuntu bionic-security multiverse
/etc/apt/sources.list:# deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main # disabled on upgrade to focal
/etc/apt/sources.list:# deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main
/etc/apt/sources.list.d/puppet5.list.save:# deb http://apt.puppetlabs.com focal puppet5 # disabled on upgrade to focal
/etc/apt/sources.list.d/puppet5.list.save:# deb-src http://apt.puppetlabs.com bionic puppet5
/etc/apt/sources.list.d/pc_repo.list:# deb https://apt.puppet.com focal puppet5 # disabled on upgrade to focal
/etc/apt/sources.list.d/mono-xamarin.list.distUpgrade:deb https://download.mono-project.com/repo/ubuntu stable-bionic main
/etc/apt/sources.list.d/google-chrome.list.distUpgrade:deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main
/etc/apt/sources.list.d/hnakamur-ubuntu-libarchive-bionic.list.save:# deb http://ppa.launchpad.net/hnakamur/libarchive/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-libarchive-bionic.list.save:# deb-src http://ppa.launchpad.net/hnakamur/libarchive/ubuntu bionic main
/etc/apt/sources.list.d/buildkite-agent.list:# deb https://apt.buildkite.com/buildkite-agent stable main # disabled on upgrade to focal
/etc/apt/sources.list.d/iovisor.list.distUpgrade:deb https://repo.iovisor.org/apt/bionic bionic main
/etc/apt/sources.list.d/vscode.list.save:# deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main # disabled on upgrade to focal
/etc/apt/sources.list.d/pc_repo.list.save:# deb https://apt.puppet.com focal puppet5 # disabled on upgrade to focal
/etc/apt/sources.list.d/bazel.list.save:deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8
/etc/apt/sources.list.d/docker-ce.list.distUpgrade:deb https://download.docker.com/linux/ubuntu bionic stable
/etc/apt/sources.list.d/google-chrome.list.save:deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main
/etc/apt/sources.list.d/hnakamur-ubuntu-cmake-bionic.list.distUpgrade:deb http://ppa.launchpad.net/hnakamur/cmake/ubuntu bionic main
/etc/apt/sources.list.d/hnakamur-ubuntu-cmake-bionic.list.distUpgrade:# deb-src http://ppa.launchpad.net/hnakamur/cmake/ubuntu bionic main
/etc/apt/sources.list.d/iovisor.list.save:# deb https://repo.iovisor.org/apt/bionic focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/google-cloud-sdk.list:deb http://packages.cloud.google.com/apt cloud-sdk main
/etc/apt/sources.list.d/bluetooth-ubuntu-bluez-bionic.list.save:# deb http://ppa.launchpad.net/bluetooth/bluez/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/bluetooth-ubuntu-bluez-bionic.list.save:# deb-src http://ppa.launchpad.net/bluetooth/bluez/ubuntu bionic main
/etc/apt/sources.list.d/mono-xamarin.list.save:deb https://download.mono-project.com/repo/ubuntu stable-focal main
/etc/apt/sources.list.d/puppet5.list.distUpgrade:deb http://apt.puppetlabs.com bionic puppet5
/etc/apt/sources.list.d/puppet5.list.distUpgrade:# deb-src http://apt.puppetlabs.com bionic puppet5
/etc/apt/sources.list.d/libreoffice-ubuntu-ppa-bionic.list.distUpgrade:deb http://ppa.launchpad.net/libreoffice/ppa/ubuntu bionic main
/etc/apt/sources.list.d/libreoffice-ubuntu-ppa-bionic.list.distUpgrade:# deb-src http://ppa.launchpad.net/libreoffice/ppa/ubuntu bionic main
/etc/apt/sources.list.d/osquery_repo.list.save:# deb [arch=amd64] https://pkg.osquery.io/deb deb main # disabled on upgrade to focal
/etc/apt/sources.list.d/buildkite-agent.list.save:# deb https://apt.buildkite.com/buildkite-agent stable main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-libzstd-bionic.list.save:# deb http://ppa.launchpad.net/hnakamur/libzstd/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-libzstd-bionic.list.save:# deb-src http://ppa.launchpad.net/hnakamur/libzstd/ubuntu bionic main
/etc/apt/sources.list.d/pc_repo.list.distUpgrade:deb https://apt.puppet.com bionic puppet5
/etc/apt/sources.list.d/chrome-remote-desktop.list.save:deb [arch=amd64] http://dl.google.com/linux/chrome-remote-desktop/deb/ stable main 
/etc/apt/sources.list.d/pritunl.list:deb http://repo.pritunl.com/stable/apt xenial main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-libzstd-bionic.list.distUpgrade:deb http://ppa.launchpad.net/hnakamur/libzstd/ubuntu bionic main
/etc/apt/sources.list.d/hnakamur-ubuntu-libzstd-bionic.list.distUpgrade:# deb-src http://ppa.launchpad.net/hnakamur/libzstd/ubuntu bionic main
/etc/apt/sources.list.d/bazel.list:deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8
/etc/apt/sources.list.d/sbt.list:# deb http://dl.bintray.com/sbt/debian / # disabled on upgrade to focal
/etc/apt/sources.list.d/google-cloud-sdk.list.save:deb http://packages.cloud.google.com/apt cloud-sdk main
/etc/apt/sources.list.d/buildkite-agent.list.distUpgrade:deb https://apt.buildkite.com/buildkite-agent stable main
/etc/apt/sources.list.d/hnakamur-ubuntu-libarchive-bionic.list.distUpgrade:deb http://ppa.launchpad.net/hnakamur/libarchive/ubuntu bionic main
/etc/apt/sources.list.d/hnakamur-ubuntu-libarchive-bionic.list.distUpgrade:# deb-src http://ppa.launchpad.net/hnakamur/libarchive/ubuntu bionic main
/etc/apt/sources.list.d/libreoffice-ubuntu-ppa-bionic.list:# deb http://ppa.launchpad.net/libreoffice/ppa/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/libreoffice-ubuntu-ppa-bionic.list:# deb-src http://ppa.launchpad.net/libreoffice/ppa/ubuntu bionic main
/etc/apt/sources.list.d/mono-xamarin.list:deb https://download.mono-project.com/repo/ubuntu stable-focal main
/etc/apt/sources.list.d/google-cloud-sdk.list.distUpgrade:deb http://packages.cloud.google.com/apt cloud-sdk main
/etc/apt/sources.list.d/osquery_repo.list.distUpgrade:deb [arch=amd64] https://pkg.osquery.io/deb deb main
/etc/apt/sources.list.d/bluetooth-ubuntu-bluez-bionic.list.distUpgrade:deb http://ppa.launchpad.net/bluetooth/bluez/ubuntu bionic main
/etc/apt/sources.list.d/bluetooth-ubuntu-bluez-bionic.list.distUpgrade:# deb-src http://ppa.launchpad.net/bluetooth/bluez/ubuntu bionic main
/etc/apt/sources.list.d/chrome-remote-desktop.list.distUpgrade:deb [arch=amd64] http://dl.google.com/linux/chrome-remote-desktop/deb/ stable main
/etc/apt/sources.list.d/dotnet-sdk.list:# deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-libzstd-bionic.list:# deb http://ppa.launchpad.net/hnakamur/libzstd/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-libzstd-bionic.list:# deb-src http://ppa.launchpad.net/hnakamur/libzstd/ubuntu bionic main
/etc/apt/sources.list.d/hnakamur-ubuntu-cmake-bionic.list.save:# deb http://ppa.launchpad.net/hnakamur/cmake/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-cmake-bionic.list.save:# deb-src http://ppa.launchpad.net/hnakamur/cmake/ubuntu bionic main
/etc/apt/sources.list.d/dotnet-sdk.list.save:# deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/clang-format-7.list:# deb [arch=amd64] http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main # disabled on upgrade to focal
/etc/apt/sources.list.d/clang-format-7.list.save:# deb [arch=amd64] http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main # disabled on upgrade to focal
/etc/apt/sources.list.d/bazel.list.distUpgrade:deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8
/etc/apt/sources.list.d/pritunl.list.save:# deb http://repo.pritunl.com/stable/apt xenial main # disabled on upgrade to focal
/etc/apt/sources.list.d/pritunl.list.distUpgrade:deb http://repo.pritunl.com/stable/apt xenial main
/etc/apt/sources.list.d/hnakamur-ubuntu-cmake-bionic.list:# deb http://ppa.launchpad.net/hnakamur/cmake/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-cmake-bionic.list:# deb-src http://ppa.launchpad.net/hnakamur/cmake/ubuntu bionic main
/etc/apt/sources.list.d/docker-ce.list.save:deb https://download.docker.com/linux/ubuntu focal stable
/etc/apt/sources.list.d/bluetooth-ubuntu-bluez-bionic.list:# deb http://ppa.launchpad.net/bluetooth/bluez/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/bluetooth-ubuntu-bluez-bionic.list:# deb-src http://ppa.launchpad.net/bluetooth/bluez/ubuntu bionic main
/etc/apt/sources.list.d/osquery_repo.list:# deb [arch=amd64] https://pkg.osquery.io/deb deb main # disabled on upgrade to focal
/etc/apt/sources.list.d/puppet5.list:# deb http://apt.puppetlabs.com focal puppet5 # disabled on upgrade to focal
/etc/apt/sources.list.d/puppet5.list:# deb-src http://apt.puppetlabs.com bionic puppet5
/etc/apt/sources.list.d/dotnet-sdk.list.distUpgrade:deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main
/etc/apt/sources.list.d/iovisor.list:# deb https://repo.iovisor.org/apt/bionic focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/sbt.list.distUpgrade:deb http://dl.bintray.com/sbt/debian /
/etc/apt/sources.list.d/hnakamur-ubuntu-libarchive-bionic.list:# deb http://ppa.launchpad.net/hnakamur/libarchive/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/hnakamur-ubuntu-libarchive-bionic.list:# deb-src http://ppa.launchpad.net/hnakamur/libarchive/ubuntu bionic main
/etc/apt/sources.list.d/docker-ce.list:deb https://download.docker.com/linux/ubuntu focal stable
/etc/apt/sources.list.d/google-chrome.list:deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main
/etc/apt/sources.list.d/vscode.list:# deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main # disabled on upgrade to focal
/etc/apt/sources.list.d/sbt.list.save:# deb http://dl.bintray.com/sbt/debian / # disabled on upgrade to focal
/etc/apt/sources.list.d/vscode.list.distUpgrade:deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main
/etc/apt/sources.list.d/clang-format-7.list.distUpgrade:deb [arch=amd64] http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main
/etc/apt/sources.list.d/libreoffice-ubuntu-ppa-bionic.list.save:# deb http://ppa.launchpad.net/libreoffice/ppa/ubuntu focal main # disabled on upgrade to focal
/etc/apt/sources.list.d/libreoffice-ubuntu-ppa-bionic.list.save:# deb-src http://ppa.launchpad.net/libreoffice/ppa/ubuntu bionic main
/etc/apt/sources.list.d/chrome-remote-desktop.list:deb [arch=amd64] http://dl.google.com/linux/chrome-remote-desktop/deb/ stable main

EDIT 2 I managed to uninstall pritunl-client-gtk by inserting exit 0 in the beginning of the remove script: /var/lib/dpkg/info/pritunl-client-gtk.prerm

After that, apt --fix-broken remove / ... install as well as apt autoremove and apt reinstall libappstream4 all work fine and cleaned up things.

However, apt-get update still fails as described above.

Answer 1

So I fixed it.

In short, the key parts were

1. Identify the failing pre-remove and insert exit 0 to skip it.
2. Reinstall all libappstream packages.

In detail (not all of these may be required, but this is what I did):

1. As written in "EDIT 2" above, I inserted exit 0 in the beginning of the pre-remove script: /var/lib/dpkg/info/pritunl-client-gtk.prerm
2. apt --fix-broken remove
3. apt --fix-broken install
4. apt autoremove
5. List the appstream packages:

:~$ dpkg -l | grep -i appstream
ii  appstream                                                   0.12.10-2                                                   amd64        Software component metadata management
ii  libappstream-glib8:amd64                                    0.7.16-1ubuntu1                                             amd64        GNOME library to access AppStream services
ii  libappstream4:amd64                                         0.12.10-2                                                   amd64        Library to access AppStream services

6. Then I did apt reinstall xxx for each these three: libappstream4, appstream, libappstream-glib8

Now apt-get update works fine.

Answer 2

Oh so many ppa's.

for the first make a backup of your important data.

In

/etc/apt/sources.list.d/docker-ce.list:deb https://download.docker.com/linux/ubuntu focal stable

add [arch=amd64]

deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable

then remove the disruptive packages.

sudo dpkg -P pritunl-client-gtk 

Only if this work.

sudo apt update && sudo apt-get clean && sudo apt-get autoclean

the second command removes partial downloaded packages the third removes packages in /var/cache/apt/archives that can no longer be downloaded, and are largely useless

sudo dpkg --configure -a && sudo apt -f install

If you are lucky and this works make your upgrade ready

sudo apt full-upgrade