How do I fix boot of Ubuntu 17.04 from external USB HDD after update to Windows 10 Creator Update broke it?

Question

Note: This is subtly different from many apparently similar existing questions.

I have Windows 10 installed on the internal drive of my note PC and Ubuntu 17.04 on an external USB HDD. Everything was working tolerably until the Windows 10 Creators Update came along. By tolerably I mean, if the HDD was plugged in it would boot Ubuntu, if not, grub would complain about not finding Ubuntu. I could type "exit" and eventually get into the EFI menu from where I could select Windows Boot Manager and boot Windows.

Since W10 Creators Update it always boots directly into Windows, even when the USB HDD is connected and even though the EFI boot order menu still shows "Ubuntu" as the first item. I tried moving the USB HDD to the first item but it continues to boot directly into Windows.

I want to take the opportunity of this problem to fix this correctly so if the USB HDD is plugged in it will boot into Ubuntu and if not it will boot into Windows without any interaction.

I booted Ubuntu Live, 16.04 because that's what I already had on a USB stick. Using gparted, I see that the Ubuntu USB HDD drive has

  /dev/sdc1 ext4        139.70 GiB boot
  /dev/sdc2 extended     11.17 GiB
  /dev/sdc5 unknown      11.17 GiB
unallocated unallocated 314.89 GiB

I am looking at the instructions in Windows 10 upgrade kills grub and boot-repair doesn't help attempting to modify them for my situation and have the following questions:

1. The root partition with 17.04 in it is /dev/sdc1. The instructions suggest /dev/sd*1 would be an EFI partition and /dev/sd*2 would be the root partition. How do I determine if I need to have an EFI partition?

2. I thought about trying sudo grub_install /dev/sdc per the above linked instructions, but I'd like some confidence it won't make things worse before I try it. Is this a sensible thing to do?

3. Do I even need Grub for this situation? Since disk has only one OS on it I don't need a way to choose the OS on power-up.

ADDENDUM

The original failure to boot when the USB HDD is unplugged is due to grub-installbug 1173457. This also explains how Windows update was able to break the Ubuntu boot now - it was using grub mistakenly installed on the internal disk and explains why the USB HDD is not "bootable" on its own.

The answer over in Install Ubuntu 17.04 on external usb drive without modifying the internal hard drive has suggestions how to fix the buggy grub-installation. I can't apply those suggestions directly because I do not have a good grub.cfg and because my USB HDD is not using EFI. /boot/efi is empty. Furthermore I need the answer to question 3 before I consider whether to install grub to the USB HDD.

boot-repair Findings

The boot-repair report URL is http://paste.ubuntu.com/25148428/.

/dev/sda is the internal disk with Windows, /dev/sdb the USB stick from which Ubuntu Live was booted and /dev/sdc the USB HDD with my 17.04 installation. From my limited knowledge of boot, it looks like there is indeed no boot file on /dev/sdc and Windows installed an EFI/Boot/bootx64.efi on /dev/sda which I'm guessing is taking precedence over EFI/ubuntu/grubx64.efi.

So I need to know how to install a boot file on /dev/sdc without also writing stuff to /dev/sda.

BTW why isn't boot-repair a standard part of Ubuntu Live?

First Attempt at Repair

I made a new 550MiB fat32 partition at the end of the unallocated space on /dev/sdc. The gparted on my Ubuntu Live USB stick does not support the ESP flag so I set just the boot flag and I added a label "ESP". I then attempted to copy the boot files from the internal disk but I got errors from copy: e.g

cp: error reading '/boot/efi/EFI/ubuntu/shimx64.efi’: Input/output error
cp: failed to extend ‘/mnt/EFI/ubuntu/shimx64.efi’: Input/output error`

Thinking about possible ESP filesystem corruption I booted Windows disabled Fast Start and shutdown Windows. I booted Ubuntu Live again and again tried the copy. Same thing.

So then I followed the instructions in the second answer at Windows 10 upgrade kills grub and boot-repair doesn't help, using /dev/sdc1 for the linux partition on my USB HDD and mounting /dev/sdc3 for the ESP. The core of these instructions is running grub-install /dev/sdc.

When I tried to reboot I got a secure boot error. When I dismissed this, Windows booted, running some disk checks in the process. I shut it down went into EFI, turned off secure boot, and this time Ubuntu booted.

I shutdown, unplugged the USB HDD and rebooted. It booted into Windows without any interaction from me. I then repeated with the USB HDD plugged back in and Ubuntu booted.

Progress indeed.

However I see 2 peculiar things:

1. /dev/sda3 (internal disk ESP) is mounted on /boot/efi. /dev/sdb3 (USB HDD) is mounted as /media/mark/ESP.
2. /media/mark/ESP/EFI/ubuntu has a subdirectory also called ubuntu with files of the same name as those in the parent (grub.cfg, grubx64.efi, mmx64.efi shimx64.efi, plus a directory fw and a file fwupx64. The contents of grub.cfg, grubx64.efi &simx64.efi` differ.

I have not yet done steps 5 & 6 in Rod Smith's answer. Step 6 looks like it should fix no. 1.

UPDATE 1 I have done step 6, edited /etc/fstab and now /dev/sdb3 is mounted on /boot/efi. I still see those extra files, of course.

UPDATE 2 I have now done step 5, copy the ubuntu files to EFI/boot and renaming the copied shimx64.cfg to bootx64.cfg. This has made it compatible with secure boot which is now renabled.

Will it be ok to just delete the extra files in the /dev/sdb ESP? What are fw and fwupx64? Do I need them?

Answer 1

The Initial Problem

Fundamentally, your initial problem was caused by the intersection of a few issues:

• Your computer is booting Windows, and was booting Ubuntu, in EFI mode; however, your /dev/sdc, where Ubuntu is installed, is an MBR disk. EFI-mode boots are best done from GPT disks. (It's possible to boot in EFI mode from MBR disks, but yours isn't set up with an MBR EFI System Partition [ESP], which would be needed for that.) This isn't really a critical issue, since your GRUB is installed to /dev/sda, which is GPT; but it could become a complication if you needed to boot more directly from /dev/sdc.
• When installing in EFI mode to a computer with multiple disks, the Ubuntu installer installs GRUB to the first ESP it finds. This is normally the ESP on /dev/sda -- /dev/sda3 in your case. As your /dev/sdc doesn't have an ESP, there's really nothing else it could have done.
• When installing in EFI mode, Ubuntu installs GRUB as EFI/ubuntu/grubx64.efi and Shim as EFI/ubuntu/shimx64.efi. To boot from removable media, it's better to install the boot loader (Shim, normally) as EFI/BOOT/bootx64.efi. Since your /dev/sdc lacks an ESP, this would have been impossible. I mention it because renaming GRUB could be part of a solution to the initial problem -- but perhaps not the easiest solution, as noted later.
• GRUB, at least as Ubuntu configures it, relies on files on two partitions: The ESP and whatever partition holds the Linux kernel. In your case, these partitions are on two physical disks -- /dev/sda and /dev/sdc. Thus, when you unplug the external disk, GRUB switches from its menu to its grub> prompt, since it can no longer find its configuration files on the external disk. Getting both parts of GRUB on one physical disk is one way to resolve this aspect of the problem -- but there are other fixes, too....

The net result of these issues is that the computer worked as you described, with boots through GRUB working only if the external disk was plugged in. There are other ways to get it working, but they require some knowledge to set up. (More on this shortly....)

What Updating Windows May Have Done, and How to Fix It

I'm not sure what happened with the Windows update. Ordinarily, I'd guess that it changed the boot order, making Windows the default; but your efibootmgr output (lines 1000-1024 of your Boot Repair output) shows that ubuntu is still first in the boot order (after the USB drive that you presumably don't normally use). This boot order change hypothesis also does not explain the fact that you can't get the system to boot even by selecting the ubuntu option in the boot menu.

Thus, I'm left with a second hypothesis: You may have a damaged ESP filesystem. This can happen if you leave the Windows Fast Startup and/or Hibernate features enabled. These features turn shutdown operations into suspend-to-disk operations, which leaves disk filesystems in an inconsistent state, which can prevent EFIs from reading some files on them. Thus, I recommend you disable both of these features, as described here and here. Note that you may need to reboot at least once to get this change to take effect.

Another possibility is that you're running into a Secure Boot problem of some sort. I wouldn't expect a Windows update to affect this, unless Microsoft has decided to blacklist Ubuntu, and I've not heard of them doing such a thing. Nonetheless, it could be that Secure Boot is malfunctioning. If this is the case, disabling it should fix the problem. Unfortunately, the details of how to do this vary from one system to another, so I can't provide succinct instructions. See this page of mine for several examples, though.

Fixing the Original Problem

At best, disabling Fast Startup, Hibernate, and/or Secure Boot will get the system booting as it did before. That's not optimal, though. There are several possible ways to improve the configuration. The simplest to get working is likely to be to install my rEFInd boot manager. Given your current configuration, rEFInd should install to the ESP and launch the first time you boot it. If you leave Secure Boot enabled, though, this will actually bypass into MokManager, which is a utility for managing Machine Owner Keys (MOKs), which are Secure Boot keys used by Shim to authorize booting. You'll need to use the primitive MokManager user interface to locate the refind.cer and/or refind_local.cer keys, probably in the EFI/refind/keys directory, and add them to the MOK list. See the rEFInd Secure Boot documentation for further details.

Once you get rEFInd to boot, it should show you options to boot both Windows and Ubuntu. Since rEFInd doesn't rely on configuration files on the external disk, it won't do the equivalent of dropping to a grub> prompt when you unplug the external disk; it will simply stop showing you the option to boot the Linux kernels directly. (It will likely still show an option to boot Ubuntu via GRUB, but that option will just generate the grub> prompt.)

Another way to fix the problem is to create a separate /boot partition on your internal disk. Yet another is to create an ESP on the external disk and move GRUB there, using the filename EFI/BOOT/bootx64.efi. Both these options are likely to be more work than installing rEFInd -- although installing the MOK may be frustrating if you're not familiar with the operation.

---

EDIT:

In response to your comments:

• Many EFIs have very delicate FAT drivers that object to filesystem damage that doesn't bother Windows or Linux. Also, note that the filesystem damage to which I refer would be invisible to Windows, since it's caused by the fact that, with Fast Startup turned on, Windows leaves the filesystem in an inconsistent state when it hibernates ("shuts down" -- but it's really not a shutdown). If nothing else touches the filesystem, it will then be in the expected state when Windows starts up again -- but both the EFI and any other OS will see it as being in an inconsistent state. This is a very real and very serious problem in a dual-boot scenario. I STRONGLY advise you to take this issue seriously! Disabling Fast Startup and Hibernate, as suggested above, is a requirement for safe dual-booting when one OS is Windows 8 or later. The problem you're experiencing is just one possible symptom of a failure to disable this feature; there may be other problems you're experiencing now or will experience in the future, and these problems can affect partitions other than the ESP.
• The second paragraph of your question refers to a boot menu. I assumed you meant the boot menu that most EFIs offer, typically by hitting a function key, Esc, or Enter (but which key varies from one computer to another). You might also get this boot menu after exiting GRUB in some situations. This menu should enable you to select the boot program to run. It can be manipulated in Ubuntu with the efibootmgr utility, and your Boot Repair output shows a listing of what's on your system on lines 1000-1024.
• If you create an ESP on /dev/sdc, its location on that disk is unimportant. Some documentation suggests putting it at the start of the disk, but that's just to keep it out of the way of subsequent partition changes. The end of the disk works as well for that, and may be preferable in your case, since resizing from the end of a partition is faster and safer than resizing by changing the start of a partition.
• On an MBR disk, recent versions of parted and GParted can set the ESP type code (0xEF) by setting the "esp" flag. Older versions of parted and GParted cannot set this type code on MBR disks; you'd need to use fdisk to set the type code to 0xEF. If you convert to GPT with gdisk, you can set its type code to EF00 in that tool, or set the "boot/esp" flag in parted or GParted. (Older versions lack the "esp" flag, so you'd use "boot" with them.) Note that the "boot" flag in parted and GParted has entirely different meanings for MBR vs. GPT disks.
• I recommend making the ESP 550 MiB (or larger, but that's generally large enough). The reasons are complex and have to do with flaky EFI FAT filesystem drivers on some computers.
• If you insist on putting an ESP on /dev/sdc, you may want to consider converting it from MBR to GPT. This is probably not necessary, but there's a chance that it is. (It depends on your EFI implementation.) See this page of mine for details of how to do this.
• If you create an ESP on /dev/sdc and move GRUB there, you should copy Shim to the fallback filename (EFI/BOOT/bootx64.efi) on the ESP. That is:
  1. Mount the /dev/sdc ESP somewhere. Let's say it's /mnt.
  2. Prepare a directory tree: sudo mkdir -p /mnt/EFI/ubuntu.
  3. Copy GRUB from the ESP: sudo cp -r /boot/efi/EFI/ubuntu/* /mnt/EFI/ubuntu/.
  4. Copy the new ubuntu directory: sudo cp -r /mnt/EFI/ubuntu /mnt/EFI/BOOT.
  5. Rename Shim: sudo mv /mnt/EFI/BOOT/shimx64.efi /mnt/EFI/BOOT/bootx64.efi.
  6. Edit /etc/fstab to mount the ESP from /dev/sdc rather than the one on /dev/sda to /boot/efi. This will simplify subsequent software updates and maintenance.
  7. When you reboot, you'll need to select the option to boot from /dev/sdc in your computer's built-in boot manager menu. This will likely be tedious on a long-term basis.

With all that laid out, I think it should be clear why I recommended installing rEFInd -- there's no need, with that course of action, to repartition /dev/sdc or to deal with the manual file-copying. These actions are tedious at best and they can easily go wrong. Also, if you rely on GRUB on the fallback filename (EFI/BOOT/bootx64.efi) on the external disk, you'll also need to use the computer's built-in boot manager to boot Ubuntu. This could be tedious. You might be able to explicitly register the EFI/ubuntu/shimx64.efi boot loader on the external disk with efibootmgr, but many EFIs will delete that entry when you unplug the disk, so this may not work as expected. OTOH, if you leave Secure Boot active, using rEFInd does require registering a MOK, which is a nuisance; but that's less risky and, IMHO, slightly easier than all the file-juggling. You'll also have a simple, consistent boot menu every time you boot, without having to mess with hitting a function key at just the right moment when you boot.

Answer 2

Microsoft Windows Creator's Update does it again. The same thing happened in the Windows Anniversary Update. It overwrote GRUB with its own loader, and it seems to want to wipe out Linux (Ubuntu) partitions.

Looking at your /dev/sdc drive, it looks like:

/dev/sdc1 may have been your / (root)
/dev/sdc5 may have been your swap partition
then there's 314GB of unallocated space which may have been your /home

Does this configuration sound familiar? If it does, then:

• boot to a Ubuntu Live DVD/USB
• install testdisk
• see http://www.cgsecurity.org/wiki/TestDisk_Step_By_Step for info on how to recover a lost partition.
• you may also have a corrupt swap partition /dev/sdc5