47

I have read every thread I could find -- even the ones that say that the question has been answered elsewhere but none of them address the specific issue I am having. Update Manager ran and produced errors relating to unmet dependency and suggested using apt-get install -f which fails with this message:

Unpacking linux-headers-3.5.0-36 (from .../linux-headers-3.5.0-36_3.5.0-36.57~precise1_all.deb) ...
dpkg: error processing /var/cache/apt/archives/linux-headers-3.5.0-36_3.5.0-36.57~precise1_all.deb (--unpack):
 unable to create `/usr/src/linux-headers-3.5.0-36/arch/arm/mach-iop32x/include/mach/glantank.h.dpkg-new' (while processing `./usr/src/linux-headers-3.5.0-36/arch/arm/mach-iop32x/include/mach/glantank.h'): No space left on device

I suspect that the last 5 words No space left on device are significant but df and du both show adequate space. So I am wondering, what device needs more space?

Ouput from df -h

Filesystem 
Size Used Avail Use% 
Mounted on /dev/sda1 5.5G 4.4G 786M 86% 
/ udev 996M 4.0K 996M 1% 
/dev tmpfs 402M 880K 401M 1% 
/run none 5.0M 0 5.0M 0% /run/lock none 1004M 156K 1004M 1% 
/run/shm /dev/sdb1 30G 1.4G 27G 5% 
/home /dev/sdc1 299G 31G 268G 11% /media/HD-PCTU2
Mitch
  • 107,631
Don A
  • 571
  • 1
  • 4
  • 4
  • Do you have space on the root partition? – don.joey Jul 08 '13 at 13:38
  • 1
    Please paste (i) the output of df -hand (ii) the output of mount. dpkg will clean up the files after it finished working, so it might appear that you have enough space when in fact you don't. Consider running apt-get clean before you start dpkg. – January Jul 08 '13 at 13:41
  • 1
    to monitor do watch -n 1 df -h in a terminal while doing the apt-get install (it will slow down a bit you i/o but you will know which file system is too small) – Emmanuel Jul 08 '13 at 13:51
  • Problem occurring also with Ubuntu 14.04 and kernel version 3.13.0.108 in one computer out of three. I started off with http://askubuntu.com/questions/223143/broken-package-after-update-linux-headers-error-brokencount-0 but the answers shown here are in the same line. – XavierStuvw Feb 05 '17 at 16:58
  • https://askubuntu.com/questions/911865/no-more-disk-space-how-can-i-find-what-is-taking-up-the-space – Deepak Kumar Singh Oct 06 '20 at 07:29

5 Answers5

52

I just encountered this same problem. I elsewhere came across a mention of inodes, and ran in my terminal to check inode usage : 

  df -i

This showed inode use at 99%. So, while my disk had plenty of space left, I wasn't able to create more files because of the limit in the number of inodes. Some disk cleanup was the solution to my problem.

Wilf
  • 30,194
  • 17
  • 108
  • 164
Kevin
  • 621
  • 5
  • 3
  • 8
    I had the same problem caused by kernel updates which do not remove old kernels automatically. Linux headers of every kernel update occupy a considerable number of inodes! --- My "safe" solution was to move two kernel headers directories to a different filesystem: cd /usr/src ; cp -a linux-headers-3.2.0-3{2,3}* /home/tmp-hdr ; rm -r linux-headers-3.2.0-3{2,3}*, apt-get-install -f, remove some other headers the right way: apt-get purge linux-headers-..., move the headers from the tmp back: cd /usr/src ; cp -a /home/tmp-hdr/* .. Remove more of the old kernels using apt-get purge. – pabouk - Ukraine stay strong Jun 29 '14 at 20:40
  • 1
    Please consider participating on the bug reports with low activity: Bug #1089195 linux-headers will eat your inodes on LTS., Bug #690911 Installation without formatting fails to remove old kernels – pabouk - Ukraine stay strong Jun 29 '14 at 20:43
  • @pabouk I wonder, if you could have used dpkg --purge linux-headers-3.2.0-3{2,3}*; apt-get -f install so that no copying was needed? – jarno Aug 18 '16 at 08:18
  • @jarno It was a long time about but I think that all the commands using the dpkg or apt database were failing. I think it was because of broken dependencies and other inconsistencies caused by insufficient number of inodes during an automatic upgrade. – pabouk - Ukraine stay strong Aug 18 '16 at 13:21
  • The required disk cleanup could be done by sudo linux-purge --fix once the script is published. – jarno Mar 25 '17 at 14:02
  • How do you remove packages since dpkg won't work because of no space left? I think only pabouk's solution (manually mv cp etc) is possible – chefarov May 11 '18 at 11:00
  • pabouk's solution does work: dpkg --purge <pkg> does remove straight away, even if there is no space left on the device. – Adrian Zaugg Mar 22 '21 at 22:48
48

I'm thinking your boot partition is full of old kernel images, leaving no room for the new one apt-get is trying to install. You can type 

dpkg -l linux-headers-\* linux-image-\* | grep ^ii

into a terminal window. When I do this, I get

ii  linux-headers-3.8.0-18                    3.8.0-18.28                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-18-generic            3.8.0-18.28                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-19                    3.8.0-19.30                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-19-generic            3.8.0-19.30                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-21                    3.8.0-21.32                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-21-generic            3.8.0-21.32                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-22                    3.8.0-22.33                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-22-generic            3.8.0-22.33                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-23                    3.8.0-23.34                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-23-generic            3.8.0-23.34                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-3.8.0-25                    3.8.0-25.37                            all          Header files related to Linux kernel version 3.8.0
ii  linux-headers-3.8.0-25-generic            3.8.0-25.37                            amd64        Linux kernel headers for version 3.8.0 on 64 bit x86 SMP
ii  linux-headers-generic                     3.8.0.25.43                            amd64        Generic Linux kernel headers

Since I have quite a few, I could delete some of the oldest with

sudo apt-get remove linux-headers-3.8.0-18

If apt-get fails because it misses some package dependencies (which you can't install due to the lack of free inodes), you can resort to dpkg directly instead:

sudo dpkg --remove linux-headers-3.8.0-18

Do that a couple of times to your oldest linux-headers-* and you should have room.

David Foerster
  • 36,264
  • 56
  • 94
  • 147
Marc
  • 9,980
  • 4
    This brings to light the other issue. Every time I attempt to run dpkg it tells me that it has an unmet dependency The following packages have unmet dependencies: linux-headers-3.5.0-36-generic : Depends: linux-headers-3.5.0-36 but it is not going to be installed and that is the thing that won't install because there isn't enough space – Don A Jul 08 '13 at 14:41
  • Perhaps a backup of your user files is in order, before you mess with your boot partition? I've never had trouble doing this, but if I did, I'd have everything I needed for a reinstall. Of course, I do a backup every day anyway. – Marc Jul 08 '13 at 14:55
  • I was hoping I wouldn't need to mess with the boot partition and that there was a way to break out of the endless cycle of dependency. I see now that it's probably going to be easier to backup data and start over. Now that I know the importance of cleaning out older items I'll make it a habit. Thanks to all for their assistance. – Don A Jul 08 '13 at 16:44
  • Not at all . . . backing up is always a good idea, and you're not really messing with the boot partition very much, just uninstalling some packages with apt-get. It should be a piece of cake. I just suggested backing up your data because I always suggest backing up data if you're having a problem (or even if you're not!) I actually ran those commands on my own system to test them. It took me three minutes to remove linux-headers-3.8.0-18. – Marc Jul 08 '13 at 16:54
  • 1
    If it fails again for sudo dpkg --remove linux-headers-3.8.0-18, remove sudo dpkg --remove linux-headers-3.8.0-18-generic and it will take the other header with it. – Sam Dec 12 '17 at 14:43
  • you can't use dpkg if no space is left – chefarov May 11 '18 at 10:59
21

I used sudo apt-get autoremove and it removed a bunch of old kernel-headers packages. Good to go after that.

Eliah Kagan
  • 117,780
Frank Sfalanga Jr.
  • 211
  • 2
  • 2
  • In my case apt-get refused to do autoremove giving precedence to the fact that there was a broken package to sort out. Which broken package had been caused by the inode saturation. See also this AskUbuntu thread – XavierStuvw Feb 05 '17 at 16:57
  • Well, this just saved me! After an update my x session was never entering, just asking to log in again and I oft got the no space left message. I thought I had to format my hd, but your trick just saved me :) – user5193682 May 17 '17 at 21:44
  • 1
    Lol I can't run this because I have too many files... – Brian Leishman Dec 14 '18 at 17:21
13

As far as I know the limit of the inodes in Linux ext* file systems is around 4 billion, but not less, which is huge amount of files. So your problem is that something is generating a lot of files. I can propose you to check first this: http://blog.scoutapp.com/articles/2014/10/08/understanding-disk-inodes So you will find what is filling the inodes of your FS. I can propose also to clean the apt cache:

sudo apt-get clean

and

sudo apt-get autoclean
Stefan
  • 389
0

If you ran out of disk space, apt-get clean / apt-get autoclean doesn't give you enough space, and you have some RAM to spare, try my answer at:

https://unix.stackexchange.com/a/726355/250404

I used a ramdisk to squeeze out enough space to complete apt --fix-broken install and recover from this.

karel
  • 114,770
Cedar
  • 101
  • 2