7

I want to upgrade my Ubuntu 16.04 server to Ubuntu 18.04 and I'm running the following commands to do so;

apt update -y
apt upgrade -y
do-release-upgrade

apt update command runs fine with the following output;

# apt update -y
Hit:1 https://esm.ubuntu.com/infra/ubuntu bionic-infra-security InRelease
Hit:2 https://esm.ubuntu.com/infra/ubuntu bionic-infra-updates InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
5 packages can be upgraded. Run 'apt list --upgradable' to see them.

However when running the apt upgrade command, it returns the following errors;

# apt upgrade -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done

*The following packages could receive security updates with UA Infra: ESM service enabled:
  libkrb5-3 libgssapi-krb5-2 libk5crypto3 libkrb5support0 libzstd1
Learn more about UA Infra: ESM service for Ubuntu 16.04 at https://ubuntu.com/16-04


Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.


The following packages have been kept back:
  libk5crypto3 libkrb5support0
The following packages will be upgraded:
  libzstd1
1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
1 esm-infra security update
Need to get 189 kB of archives.
After this operation, 132 kB of additional disk space will be used.
Err:1 https://esm.ubuntu.com/infra/ubuntu bionic-infra-security/main amd64 libzstd1 amd64 1.3.3+dfsg-2ubuntu1+esm1
  401  Unauthorized
E: Failed to fetch https://esm.ubuntu.com/infra/ubuntu/pool/main/libz/libzstd/libzstd1_1.3.3+dfsg-2ubuntu1+esm1_amd64.deb  401  Unauthorized


E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

I'm not sure what to do at this point. How can I solve this?

Marry Jane
  • 255
  • How long ago did you try? 401 can mean that a repo is down for whatever reason, it seems up now, alternatively you could try just removing esm.ubuntu.com form your sources list, since you are upgrading, you don't need extended support. – Mark Kirby Oct 24 '21 at 13:21
  • Well I tried a few minutes ago but I have multiple servers and I'm doing them all at the same time and while some of them works fine, some of them gives this error so I don't think the repo is down, I think it's related to my server. – Marry Jane Oct 24 '21 at 13:23
  • @MarkKirby also I don't have esm.ubuntu.com in my sources list at all. – Marry Jane Oct 24 '21 at 13:42
  • 1
    You say you're running 16.04 ESM, but those look like 18.04 ESM sources (they say bionic instead of xenial). 18.04 does not need to be ESM, since it's still in Community Support. This suggests that somebody already tried a do-release-upgrade that failed. Perhaps the upgrader left the altered sources in place (it happens). You can change the sources back to Xenial. – user535733 Oct 24 '21 at 13:50
  • @user535733 Yes, I tried that myself but it failed. How can I revert this back? – Marry Jane Oct 24 '21 at 14:04
  • The fact that it failed suggests that you might need to be prepared for a backup-and-reinstall. Depends upon the reason (check the logs!) Revert the sources change by simply substituting release names in /etc/apt/sources.list and /etc/apt/sources.list.d/* – user535733 Oct 24 '21 at 14:39

3 Answers3

12

Two ways exist depending on currently running Ubuntu version.

(a) Ubuntu 16.04 LTS as currently running version

You have to backup your sources.list by

sudo mv /etc/apt/sources.list ~/
sudo mv /etc/apt/sources.list.d/*.list ~/

and then fill main sources.list with correct URLs using one of commands below:

  • plain sources.list replace

    cat <<EOF | sudo tee /etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu/ xenial-backports main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ xenial-updates main universe multiverse restricted
deb http://security.ubuntu.com/ubuntu/ xenial-security main universe multiverse restricted
EOF

  • using add-apt-repository

    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial-backports main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ xenial-updates main universe multiverse restricted"
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu/ xenial-security main universe multiverse restricted"

and then resume upgrade to 18.04 LTS:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get purge ubuntu-advantage-tools --autoremove
sudo rm /etc/apt/sources.list.d/ubuntu-esm-infra.list
sudo do-release-upgrade

Then check ~/*.list files from previous release and partially move the lines to /etc/apt/sources.list.

(b) Ubuntu 18.04 LTS as currently running version

You have to backup your sources.list by

sudo mv /etc/apt/sources.list ~/
sudo mv /etc/apt/sources.list.d/*.list ~/

and then fill main sources.list with correct URLs using one of commands below:

  • plain sources.list replace

    cat <<EOF | sudo tee /etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu/ bionic-backports main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ bionic main universe multiverse restricted
deb http://archive.ubuntu.com/ubuntu/ bionic-updates main universe multiverse restricted
deb http://security.ubuntu.com/ubuntu/ bionic-security main universe multiverse restricted
EOF

  • using add-apt-repository

    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic-backports main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic main universe multiverse restricted"
sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu/ bionic-updates main universe multiverse restricted"
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu/ bionic-security main universe multiverse restricted"

and then install all necessary upgrades by:

sudo apt-get purge ubuntu-advantage-tools --autoremove
sudo rm /etc/apt/sources.list.d/ubuntu-esm-infra.list

sudo apt-get update
sudo apt-get upgrade

Then check ~/*.list files from previous release and partially move the lines to /etc/apt/sources.list while replacing xenial with bionic.

N0rbert
  • 99,918
  • 3
    Thank you for your answer. It is very detailed. For my case, only running the command sudo apt-get purge ubuntu-advantage-tools --autoremove alone solved my issue. I didn't have to do the other changes. – Marry Jane Oct 25 '21 at 12:58
  • On an 18.04 system, the source of the problem was the /etc/apt/sources.list.d/ubuntu-esm-infra.list file. Commenting out the sources in there did the trick. I'm sure deleting it would have worked, too. – Lambart Jun 07 '23 at 19:37
4

On an 18.04 system (at least), look for a /etc/apt/sources.list.d/ubuntu-esm-infra.list file. That file erroneously lists sources for ESM updates that you would need to pay for, and apt is dutifully trying to download them.

I disabled those sources by putting a # before each of the two deb lines. The file now looks like this:

# Written by ubuntu-advantage-tools
#deb https://esm.ubuntu.com/infra/ubuntu bionic-infra-security main
# deb-src https://esm.ubuntu.com/infra/ubuntu xenial-infra-security main

#deb https://esm.ubuntu.com/infra/ubuntu bionic-infra-updates main


deb-src https://esm.ubuntu.com/infra/ubuntu xenial-infra-updates main

I think deleting the file would have worked, too.

Now, when I update the system again, using my usual update command:

$ sudo apt update && sudo apt autoremove && sudo apt upgrade

...the apt upgrade portion doesn't try to download those unauthorized sources, and the upgrade is able to finish!

Lambart
  • 2,760
  • 1
  • 22
  • 27
3

When you register a system with Ubuntu Pro a /etc/apt/auth.conf.d/90ubuntu-advantage file is created. This file contains credentials tied to your Ubuntu account which gives that system permission to retrieve ESM updates.

I registered a system and initially it had no issues, but after some weeks attempts to retrieve packages from https://esm.ubuntu.com/ gave a HTTP 401 "Unauthorized" error.

I resolved the issue by detaching and reattaching the machine using the current Token value from https://ubuntu.com/pro/dashboard.

This regenerated the /etc/apt/auth.conf.d/90ubuntu-advantage file with current credentials.

  1. sudo pro detach
  2. sudo pro attach TOKEN_VALUE_HERE
  3. sudo apt-get update

At this point I could apply updates (e.g., via apt, apt-get, aptitude) as before.

deoren
  • 373
  • 2
  • 9