Ubuntu 20.04.5 live USB flash fails to boot (Failed to set MokListRT: Out of Resources) on 2011 Dell Optiplex 990

Question

Had Ubuntu 18.04 Desktop on a Dell Optiplex 990. Tried to upgrade it to 20.04 over the network but that had periodic errors and eventually failed. Could get GUI login screen but could not login. After following steps such as in https://ostechnix.com/how-to-fix-broken-ubuntu-os-without-reinstalling-it/ (which ran with errors) I could login, but video was limited to 800x600, it couldn't recognize my display, and updates failed because of unmet dependencies. In this report I'm skipping the details of those errors because I decided to try to reinstall from scratch.

So, the problem is every Ubuntu 20.04.5 live USB flash drive I've made with Startup Disk Creator cannot be booted. I've made the flash drives on another Ubuntu 20.04 system, an Ubuntu 18.04.5 system, and an Ubuntu 16.04 system (the later 2 being booted from old bootable flash drives I still have). I get the error "Failed to Set MokListRT: Out of resources."
Error message: Failed to Set MokListRT: Out of resources

Researched that issue, e.g.:
Failed to set MokListRT: out of Resources : import_mok_state() Ubuntu 20.04 Failed to Set MokListRT: Invalid Parameter and several people who have gotten a similar error after updating their Mac or Intel laptop with Ubuntu suggest fixing it by either:

1. overwriting the shim executeable with a copy of the grub executeable (e.g. cd /boot/efi/EFI/ubuntu, cp grubx64.efi shimx64.efi). But the problem with this, after searching, is shimx64.efi doesn't exist on the live flash, except perhaps inside the shim...deb package,

OR

2. overwriting mmx64.efi with grubx64.efi (which on the live flash are both in /EFI/BOOT/). I attempted the later but it turned out that since the flash drive has an ISO9660 file system, it's read only and can't be mounted rw, so I can't "cp grubx64.efi mmx64.efi".

So, again, the problem is every Ubuntu 20.04.5 live USB flash drive I've made with Startup Disk Creator cannot boot on my 2011 Dell Optiplex 990, or my 2011 Dell Latitude E6420. Both of these have the latest BIOS, and have GPT disks so I boot in UEFI mode (since, from recalling upgrade difficulties ~2 years ago, I think if I boot the live flash in MBR mode but install on a GPT disk, it doesn't work). There is no secure boot setting in BIOS. FYI I have Ubuntu and Windows 10 installed on the boot disk. FYI I have used at least 2 different flash drives, have formatted them as GPT before running Startup Disk Creator, but after Startup Disk Creator is finished they appear (in the Disks utility) as MBR not GPT. Also FYI per this article https://web.sas.upenn.edu/jasonrw/2016/04/07/uefi-and-a-dell-optiplex-990/ Dell 990s support either BIOS/MBR or UEFI, but not both simultaneously (no UEFI-CSM compatibility mode).

Answer 1

I have several thoughts on this....

First, I recommend against installing Ubuntu 20.04 at this time. Ubuntu 22.04 is the current LTS release. Ubuntu 22.04 will be supported for longer than 20.04, and 22.04 includes more up-to-date packages. That said, I doubt if using Ubuntu 22.04 would help you fix your problem; this is more a matter of general advice.

Second, the error message you're seeing suggests that your computer's NVRAM storage may be either full or so fragmented that Shim can't create a new variable. You might be able to improve matters a bit by removing redundant or unwanted EFI boot variables. The easiest way for you to do this is probably to boot my rEFInd boot manager; there's a USB flash drive image available that should boot directly (that image does not use Shim, which also means that you'll have to disable Secure Boot to boot it). Once in rEFInd, there's a second-row option to adjust NVRAM-based boot entries. You should be able to use that to delete unwanted boot entries. Be careful, though; delete only options that you're certain are unnecessary or redundant! Ultimately, though, I suspect that the problem is Secure Boot db and dbx keys, and/or Shim MOKs. Your firmware probably has an option to reset the Secure Boot db and dbx keys to their defaults, but precisely how varies from one computer to another. I couldn't find instructions on doing this for your Dell, but I did find this article about how to do it on an HP computer. Maybe that'll be helpful, or at least point you in the right direction. It's also conceivable that your firmware includes an option to reset everything to factory defaults, which might (or might not) clear the NVRAM of all the variables it's acquired over time. Note, however, that Secure Boot db and dbx keys exist for a reason, and deleting them can cause security problems. If you can't boot your computer, though, that can be an even bigger problem, so you may have little choice but to accept an increased security risk and either make do with a slimmer dbx or disable Secure Boot entirely.

Concerning the fact that shimx64.efi doesn't exist on the live image, that's because it has a different name. Removable media (CD-Rs, USB flash drives, etc.) boot using the fallback filename of EFI\BOOT\bootx64.efi; shimx64.efi is renamed to that filename on them. On an Ubuntu installer, EFI\BOOT\bootx64.efi (Shim) then launches grubx64.efi in the same directory; so with Secure Boot disabled, you can launch GRUB directly by renaming grubx64.efi to bootx64.efi. Note, however, that if this works, the installer with put Shim on the hard disk (as EFI\ubuntu\shimx64.efi), so you'll need to create a non-Shim boot path (using efibootmgr; see here for details) or rename grubx64.efi to shimx64.efi in that directory. (The former is the better option, if it works....) A further complication is that, given the nature of your problem, the installer might not be able to create a new boot entry, so rebooting after the install might fail. This leads back to the above solution -- cleaning up the (presumably) over-full NVRAM.

Finally, as @guiverc notes, switching the computer to boot in BIOS/CSM/legacy mode is an option. Most UEFI-based PCs (those made since 2011 or so) are UEFI-based, but most of them also include a Compatibility Support Module (CSM), which enables them to boot legacy BIOS OSes. Hence, the three terms BIOS, CSM, and legacy are more-or-less interchangeable in this context. All three rely on the CSM compatibility layer of the UEFI, so the computer remains UEFI-based when booting a BIOS-mode OS; but any given OS installation will be one or the other. Thus, you can probably work around these problems by doing a BIOS-mode install of Ubuntu. The trick is forcing the computer to boot in BIOS mode rather than in EFI mode. This is most easily accomplished by removing the EFI-mode boot loaders, which are files in the EFI directory on the boot medium. Whatever tool you used to create the boot medium will have to have included a BIOS-mode boot loader, too. Note also that this approach works best if the computer is to boot ONLY BIOS-mode OSes, because GRUB 2 can't switch between boot modes. That is, the BIOS-mode GRUB 2 can launch only BIOS-mode OSes, and the EFI-mode GRUB 2 can launch only EFI-mode OSes. Thus, if you want to dual-boot an existing EFI-mode Windows alongside Ubuntu, it's best to stick with an EFI-mode Ubuntu installation, if at all possible. If you must mix boot modes, then my rEFInd boot manager can do the trick, but you'll need to enable support on the scanfor line in its refind.conf file. Using rEFInd in your case has the disadvantage that you'd still need to create an EFI boot entry for it, and that might not be possible, given the overcrowded state of your NVRAM. Also, no matter how you do it, booting in BIOS/CSM/legacy mode means giving up the protections of Secure Boot, so this approach is not preferable to deleting the dbx from a security point of view.

So in sum, there are a number of possible solutions and workarounds to your problem, but ultimately, my suspicion is that your best option is to clean up what I suspect is an overcrowded NVRAM.

Answer 2

1. I completed and fixed my broken upgrade to Ubuntu 20.04 desktop by first disabling SAV Sophos Anti Virus

sudo /opt/sophos-av/bin/savdctl disable

Apparently the anti-virus interferes with some, but not most, of the updates! (boy did I feel dumb!)
Then I ran the commands in https://ostechnix.com/how-to-fix-broken-ubuntu-os-without-reinstalling-it/ namely:

sudo rm /var/lib/apt/lists/lock
sudo rm /var/lib/dpkg/lock
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/cache/apt/archives/lock
sudo dpkg --configure -a
sudo apt clean
sudo apt update --fix-missing
sudo apt install -f
sudo dpkg --configure -a 
sudo apt upgrade 
sudo apt dist-upgrade 
sudo reboot

2a. Regarding the issue of Startup Disk Creator making Ubuntu 20.04 desktop USB flash drives that won't boot on my Dell Optiplex 990, that is not fixed. However as a workaround I discovered that USB flash with:

• Ubuntu 18.04.5 boots with no errors
• Ubuntu 18.04.6 gets the MokListRT Out of Resources error, but continues to "boot since secure mode is disabled": Fig 1. Ubuntu 18.04.6 live USB flash errors but still boots on Dell 990
• Ubuntu 20.04.5 gets the MokListRT error and fails to boot

So I'd recommend to someone getting this error on old hardware that does not support Secure Boot, to install Ubuntu 18.04.6 or earlier, and then upgrade over the net.

2b. To try and free up space in NVRAM as Rod Smith suggested, I used rEFInd boot manager to delete unnecessary boot entries, but my Dell Optiplex 990 kept adding the Legacy boot entries back in:

Fig 2. Initial boot entries and shown by rEFInd, including unnecessary Legacy entries (diskette drive doesn't even exist!)

Fig 3. After 4 Legacy boot entries deleted

Fig 4. ...But after reboot my Dell 990 BIOS kept putting unnecessary Legacy boot entries back in. Note the higher numbers on the left hand side 1A 1B 1C

And then to clear CMOS NVRAM I followed instructions in https://www.dell.com/support/kbdoc/en-us/000124377/how-to-perform-a-bios-or-cmos-reset-and-clear-the-nvram-on-dell-systems by removing the battery, moving the motherboard jumper to RTCRST (which on a Dell Optiplex 990 mini-tower is hidden under the SATA connectors), and -- after taking pictures of my UEFI boot entries in BIOS -- using BIOS Load defaults. But, you guessed it, my Dell Optiplex 990 kept adding the Legacy boot entries back in:
Fig 4. Final boot entries after BIOS NVRAM cleared many times

So then I researched the update dependencies problem and discovered that disabling anti-virus is necessary to allow SOME updates to work, as described above.