1

I urgently need help with this here!

4 years old server.
Ubuntu 14.04 server i686.
Linux 3.13.0-149-generic was the last version to run flawlessly.
10 days ago, I upgraded to 3.13.0.151.
Server crashes then on boot.

Screen shows ...

Loading Linux 3.13.0-151-generic ...
Loading initial ramdisk ...

1 second later ... reboot.

Same with 3.13.0-151 recovery mode.
Same with 3.13.0-153 (most recent as of today, normal and recovery mode).

How can I find out, after subsequent successful 3.13.0-149 bootup, what exactly generates the crash?

Thanks!

----- later -----

@heynnema tried to help me out by telling me how to construct a new initrd.img-* for 151 (update-initramfs -c -k 3.13.0-151-generic). See below. This didn't work. 151 still didn't make the system boot. My fatal mistake was then to say update-initramfs -c -k 3.13.0-149-generic (the only working kernel). After that, I was stuck. No kernel to boot from anymore! Same probem with ramdisk as for 151 and 153.

After that, I started a Live DVD (ubuntu-14.04.5-desktop-i386.iso) on the stuck system, mounted an old 14.05.5 VM with 3.13 kernels on another computer, updated these (apt-get dist-upgrade), copied the resulting initrd.img-3.13.0-153-generic (latest kernel) to the stuck system ('/boot') and the it booted again (with 153)! This was to my big surprise, not knowing that the initrd.img-* from a VM would work on a real hardware! However I was still unable to boot from 149 and 151 (which makes sense).

All of the above was just to get the system up and working again. The problem itself is not solved!

Bottomline: update-initramfs uses data (files) on the system to build initrd.img-*. On my box, this makes it impossible to get further than "Loading initial ramdisk ...".

Questions:
Which files are used by update-initramfs?
Can I (?!) do something to create a build again a working initrd.img-3.13.0-153-generic?

As long as this problem is not solved, future built initrd-img-* files will almost certainly also crash!

geohei
  • 153
  • 1
    Boot into -149 and in terminal type sudo update-initramfs -c -k 3.13.0-151, then reboot to -151, and see if that fixes the problem. Report back to @heynnema. – heynnema Jul 04 '18 at 20:33
  • Thanks for the answer, but I feel safer to wait until 4.15.0-24 has a fix, hoping that it will be implemented as well in 3.13.0-151. Meanwhile, I changed grub that it keeps on booting from 3.13.0-149. – geohei Jul 05 '18 at 03:43
  • You're running 3.13... not 4.15. My command will do no harm, and might easily fix your problem with -151 and -153. Please begin comments to me with @heynnema or I may miss them. – heynnema Jul 05 '18 at 12:51
  • @heynnema - update-initramfs -c -k 3.13.0-151-generic didn't work. Actually no surprise since /boot/config-3.13.0-*-generic (all 3 config files - 149, 151 and 153) don't show any differences (except version numbers). – geohei Jul 05 '18 at 15:16
  • You lost me. What does "config-anything" have to do with my **update-initramfs" command? – heynnema Jul 05 '18 at 15:18
  • @heynnema - Typo at my end. I forgot to post the first part of the command. I corrected the comment above! – geohei Jul 05 '18 at 15:26
  • @heynnema - Now, the system doen't boot anymore. Not even from 149. Well ... thanks a lot! – geohei Jul 05 '18 at 15:27
  • That wasn't very nice. There may be something more wrong with your system. Try booting -149 in recovery mode, or try booting an earlier -14x kernel, if you have one. – heynnema Jul 05 '18 at 15:30
  • Let us continue this discussion in chat. – geohei Jul 05 '18 at 15:32
  • Maybe lsinitramfs <initramfsfile> will show what you want to see. – heynnema Jul 05 '18 at 19:58
  • this helped me https://linoxide.com/linux-how-to/fixing-broken-initrd-image-linux/ in a similar situation – Ferroao Sep 05 '19 at 19:12

1 Answers1

2

Again an idea from @heynnema (thanks!)

lsinitramfs didn't work on all 3 non-working initrd.img files (149, 151, 153).

root@gan:~# lsinitramfs /boot/initrd.img-3.13.0-153
/boot/initrd.img-3.13.0-153

gzip: /boot/initrd.img-3.13.0-153: not in gzip format
cpio: premature end of archive

Then this here this morning ...

root@gan:~# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  linux-headers-3.13.0-151 linux-headers-3.13.0-151-generic
  linux-image-3.13.0-151-generic linux-image-extra-3.13.0-151-generic
Use 'apt-get autoremove' to remove them.
The following packages will be upgraded:
  amd64-microcode
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 26.3 kB of archives.
After this operation, 2,048 B disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 http://xx.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64-microcode i386 3.20180524.1~ubuntu0.14.04.2+really20130710.1 [26.3 kB]
Fetched 26.3 kB in 0s (387 kB/s)
(Reading database ... 132952 files and directories currently installed.)
Preparing to unpack .../amd64-microcode_3.20180524.1~ubuntu0.14.04.2+really20130710.1_i386.deb ...
Unpacking amd64-microcode (3.20180524.1~ubuntu0.14.04.2+really20130710.1) over (3.20180524.1~ubuntu0.14.04.1) ...
Setting up amd64-microcode (3.20180524.1~ubuntu0.14.04.2+really20130710.1) ...
Updating microcode on all online processors...
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools (0.103ubuntu4.11) ...
update-initramfs: Generating /boot/initrd.img-3.13.0-153-generic

Booting worked again !!!

lsinitramfs now as well!

root@gan:~# lsinitramfs /boot/initrd.img-3.13.0-153-generic
/boot/initrd.img-3.13.0-153-generic
.
sbin
sbin/biosdevname
...

Updated other initrd.img files also (149 and 151).

root@gan:/boot# update-initramfs -c -k 3.13.0-151-generic
update-initramfs: Generating /boot/initrd.img-3.13.0-151-generic
root@gan:/boot# update-initramfs -c -k 3.13.0-149-generic
update-initramfs: Generating /boot/initrd.img-3.13.0-149-generic

All 3 were accepted now by lsinitranfs.
All 3 could be used for booting.

Hence, the source of the issue was amd64-microcode. This took 2 weeks for a fix to appear.

For testing purpose, I manually built initrd.img-3.13.0-153-generic using update-initramfs. The result was not exactly the same as the one built by apt-get update, but it worked as well.

Thanks for all the help!

geohei
  • 153