9

My /var/log folder has taken 56.6 GB. So large that I don't have too much space left. How can I clean up the folder and make more space for the system?

Here's the output for ls -alh

zedd@zedd-kubuntu:/var/log$ ls -alh
total 53G
drwxrwxr-x  17 root   syslog          4.0K Jun 25 00:00 .
drwxr-xr-x  15 root   root            4.0K Jun 16 21:52 ..
-rw-r--r--   1 root   root             36K Jun 23 03:00 alternatives.log
-rw-r--r--   1 root   root             15K Jun  2 21:44 alternatives.log.1
-rw-r--r--   1 root   root             298 Sep  6  2020 alternatives.log.10.gz
-rw-r--r--   1 root   root             127 Jul 18  2020 alternatives.log.11.gz
-rw-r--r--   1 root   root             194 Jul 18  2020 alternatives.log.12.gz
-rw-r--r--   1 root   root             455 Feb 21 10:55 alternatives.log.2.gz
-rw-r--r--   1 root   root             175 Sep 13  2021 alternatives.log.3.gz
-rw-r--r--   1 root   root            5.4K Aug 31  2021 alternatives.log.4.gz
-rw-r--r--   1 root   root             692 Aug 29  2021 alternatives.log.5.gz
-rw-r--r--   1 root   root             126 Mar 22  2021 alternatives.log.6.gz
-rw-r--r--   1 root   root             851 Mar  6  2021 alternatives.log.7.gz
-rw-r--r--   1 root   root             133 Feb 22  2021 alternatives.log.8.gz
-rw-r--r--   1 root   root             231 Oct  3  2020 alternatives.log.9.gz
drwxr-x---   2 root   adm             4.0K Jun 23 02:28 apache2
-rw-r-----   1 root   adm                0 Jun 25 00:00 apport.log
-rw-r-----   1 root   adm              621 Jun 24 11:12 apport.log.1
-rw-r-----   1 root   adm              869 Jun 22 05:28 apport.log.2.gz
-rw-r-----   1 root   adm             3.4K Jun 17 21:15 apport.log.3.gz
-rw-r-----   1 root   adm              419 Jun 14 18:46 apport.log.4.gz
-rw-r-----   1 root   adm              587 Jun 13 14:23 apport.log.5.gz
-rw-r-----   1 root   adm              577 Jun 11 04:11 apport.log.6.gz
-rw-r-----   1 root   adm             7.3K Jun 10 20:39 apport.log.7.gz
drwxr-xr-x   2 root   root            4.0K Jun 24 11:12 apt
-rw-r-----   1 syslog adm              56K Jun 25 00:17 auth.log
-rw-r-----   1 syslog adm             116K Jun 17 22:04 auth.log.1
-rw-r-----   1 syslog adm             7.2K Jun 13 07:44 auth.log.2.gz
-rw-r-----   1 syslog adm              11K Jun  6 09:35 auth.log.3.gz
-rw-r-----   1 syslog adm              788 May 29 19:33 auth.log.4.gz
-rw-------   1 root   root               0 Jun 25 00:00 boot.log
-rw-------   1 root   root             16K Jun 25 00:00 boot.log.1
-rw-------   1 root   root             50K Jun 24 09:32 boot.log.2
-rw-------   1 root   root            7.5K Jun 22 05:02 boot.log.3
-rw-------   1 root   root            7.3K Jun 21 21:55 boot.log.4
-rw-------   1 root   root             23K Jun 20 16:51 boot.log.5
-rw-------   1 root   root             29K Jun 17 11:20 boot.log.6
-rw-------   1 root   root            8.0K Jun 16 00:00 boot.log.7
-rw-r--r--   1 root   root             56K Aug  5  2019 bootstrap.log
-rw-rw----   1 root   utmp             384 Jun 20 16:55 btmp
-rw-rw----   1 root   utmp             768 May 30 22:55 btmp.1
drwxr-xr-x   2 root   root            4.0K Jun 25 00:00 cups
drwxr-xr-x   5 root   root            4.0K Jun  3 00:43 dist-upgrade
-rw-r-----   1 root   adm              86K Jun 24 21:07 dmesg
-rw-r-----   1 root   adm              89K Jun 24 11:13 dmesg.0
-rw-r-----   1 root   adm              22K Jun 24 09:32 dmesg.1.gz
-rw-r-----   1 root   adm              22K Jun 23 03:46 dmesg.2.gz
-rw-r-----   1 root   adm              22K Jun 23 03:15 dmesg.3.gz
-rw-r-----   1 root   adm              22K Jun 23 02:58 dmesg.4.gz
-rw-r--r--   1 root   root            1.6M Jun 24 11:12 dpkg.log
-rw-r--r--   1 root   root            1.9M Jun  3 00:19 dpkg.log.1
-rw-r--r--   1 root   root            3.2K Mar  6  2021 dpkg.log.10.gz
-rw-r--r--   1 root   root             17K Feb 22  2021 dpkg.log.11.gz
-rw-r--r--   1 root   root            5.8K Oct  5  2020 dpkg.log.12.gz
-rw-r--r--   1 root   root             572 May 30 03:36 dpkg.log.2.gz
-rw-r--r--   1 root   root             695 Mar 28 15:54 dpkg.log.3.gz
-rw-r--r--   1 root   root             11K Feb 21 11:50 dpkg.log.4.gz
-rw-r--r--   1 root   root             233 Nov  2  2021 dpkg.log.5.gz
-rw-r--r--   1 root   root            2.2K Sep 13  2021 dpkg.log.6.gz
-rw-r--r--   1 root   root            336K Aug 31  2021 dpkg.log.7.gz
-rw-r--r--   1 root   root             15K Aug 29  2021 dpkg.log.8.gz
-rw-r--r--   1 root   root            2.3K Mar 22  2021 dpkg.log.9.gz
-rw-r--r--   1 root   root             32K Aug 29  2021 faillog
drwxr-xr-x   2 root   root            4.0K Mar  3  2020 firebird
-rw-r--r--   1 root   root             16K Jun 23 03:02 fontconfig.log
-rw-r--r--   1 root   root            2.4K Jun 24 21:07 gpu-manager.log
-rw-r--r--   1 root   root            2.4K Jun 24 11:28 gpu-manager-switch.log
drwxrwxr-x   3 root   root            4.0K Dec 29  2019 installer
drwxr-sr-x+  3 root   systemd-journal 4.0K Dec 29  2019 journal
-rw-r-----   1 syslog adm             2.3M Jun 25 00:21 kern.log
-rw-r-----   1 syslog adm              27G Jun 17 21:59 kern.log.1
-rw-r-----   1 syslog adm             261K Jun 13 07:44 kern.log.2.gz
-rw-r-----   1 syslog adm             524K Jun  6 09:35 kern.log.3.gz
-rw-r-----   1 syslog adm              22K May 29 19:33 kern.log.4.gz
-rw-rw-r--   1 root   utmp            286K Aug 29  2021 lastlog
-rw-r-----   1 syslog adm                0 Sep 13  2021 mail.log
-rw-r-----   1 syslog adm              971 Aug 29  2021 mail.log.1
drwxr-xr-x   2 mpd    audio           4.0K Jun  6 09:35 mpd
drwxr--r--   2 root   nordvpn         4.0K Jan 23  2020 nordvpn
-rw-r--r--   1 root   root              20 Jun 24 21:07 prime-supported.log
drwx------   2 root   root            4.0K Aug 29  2021 private
drwxr-x---   3 root   adm             4.0K Jun 20 16:51 samba
-rw-r--r--   1 sddm   sddm               0 Aug 29  2021 sddm.log
-rw-r-----   1 syslog adm             1.1M Jun 25 00:21 syslog
-rw-r-----   1 syslog adm              27G Jun 17 22:04 syslog.1
-rw-r-----   1 syslog adm             647K Jun 13 07:44 syslog.2.gz
-rw-r-----   1 syslog adm             1.3M Jun  6 09:35 syslog.3.gz
-rw-r-----   1 syslog adm              52K May 29 19:33 syslog.4.gz
-rw-r-----   1 syslog adm             142K Mar 22  2021 syslog.6.gz
-rw-r-----   1 syslog adm              39K Mar 11  2021 syslog.7.gz
-rw-------   1 root   root             63K Aug 29  2021 tallylog
-rw-r--r--   1 root   root             685 Jun  3 00:43 ubuntu-advantage.log
-rw-------   1 root   root             685 Jun  2 21:26 ubuntu-advantage.log.1
-rw-------   1 root   root             210 Feb 21 10:52 ubuntu-advantage.log.2.gz
-rw-------   1 root   root             406 Jan 21 03:30 ubuntu-advantage.log.3.gz
-rw-------   1 root   root             175 Jan 11 15:17 ubuntu-advantage.log.4.gz
-rw-------   1 root   root             178 Nov  3  2021 ubuntu-advantage.log.5.gz
-rw-------   1 root   root             192 Sep 13  2021 ubuntu-advantage.log.6.gz
-rw-------   1 root   root            3.0K Jun 24 21:54 ubuntu-advantage-timer.log
-rw-------   1 root   root             314 May 30 23:22 ubuntu-advantage-timer.log.1
-rw-------   1 root   root             110 Apr 20 18:03 ubuntu-advantage-timer.log.2.gz
-rw-------   1 root   root             137 Mar 28 15:53 ubuntu-advantage-timer.log.3.gz
drwxr-x---   2 root   adm             4.0K Jun  4 01:28 unattended-upgrades
drwxr-xr-x   2 root   root            4.0K Aug 29  2021 upgrade
drwxr-xr-x   2 root   root            4.0K Jun 24 21:07 vmware
-rw-r--r--   1 root   root             43K Jun 11 04:08 vmware-installer
-rw-r--r--   1 root   root             73K Jun 23 02:58 vnetlib
drwxrwxrwx   2 root   root            4.0K Dec 30  2019 wpslog
-rw-rw-r--   1 root   utmp            295K Jun 25 00:15 wtmp
-rw-rw-r--   1 root   utmp            6.0K Aug 29  2021 wtmp.1
-rw-r--r--   1 root   root             55K Jun 24 23:21 Xorg.0.log
-rw-r--r--   1 root   root             43K Jun 24 11:28 Xorg.0.log.old

It turns out the largest files are kern.log.1; syslog.1

These 2 files are 27G each. Can I remove them safely without causing problems?

-Update-

I don't know why, but the size of these two files has drastically reduced and I haven't done anything to decrease it.

zedd@zedd-kubuntu:/var/log$ ls -alh
total 1.5G
drwxrwxr-x  17 root   syslog          4.0K Jun 28 08:27 .
drwxr-xr-x  15 root   root            4.0K Jun 16 21:52 ..
-rw-r--r--   1 root   root             36K Jun 23 03:00 alternatives.log
-rw-r--r--   1 root   root             15K Jun  2 21:44 alternatives.log.1
-rw-r--r--   1 root   root             298 Sep  6  2020 alternatives.log.10.gz
-rw-r--r--   1 root   root             127 Jul 18  2020 alternatives.log.11.gz
-rw-r--r--   1 root   root             194 Jul 18  2020 alternatives.log.12.gz
-rw-r--r--   1 root   root             455 Feb 21 10:55 alternatives.log.2.gz
-rw-r--r--   1 root   root             175 Sep 13  2021 alternatives.log.3.gz
-rw-r--r--   1 root   root            5.4K Aug 31  2021 alternatives.log.4.gz
-rw-r--r--   1 root   root             692 Aug 29  2021 alternatives.log.5.gz
-rw-r--r--   1 root   root             126 Mar 22  2021 alternatives.log.6.gz
-rw-r--r--   1 root   root             851 Mar  6  2021 alternatives.log.7.gz
-rw-r--r--   1 root   root             133 Feb 22  2021 alternatives.log.8.gz
-rw-r--r--   1 root   root             231 Oct  3  2020 alternatives.log.9.gz
drwxr-x---   2 root   adm             4.0K Jun 23 02:28 apache2
-rw-r-----   1 root   adm                0 Jun 25 00:00 apport.log
-rw-r-----   1 root   adm              621 Jun 24 11:12 apport.log.1
-rw-r-----   1 root   adm              869 Jun 22 05:28 apport.log.2.gz
-rw-r-----   1 root   adm             3.4K Jun 17 21:15 apport.log.3.gz
-rw-r-----   1 root   adm              419 Jun 14 18:46 apport.log.4.gz
-rw-r-----   1 root   adm              587 Jun 13 14:23 apport.log.5.gz
-rw-r-----   1 root   adm              577 Jun 11 04:11 apport.log.6.gz
-rw-r-----   1 root   adm             7.3K Jun 10 20:39 apport.log.7.gz
drwxr-xr-x   2 root   root            4.0K Jun 28 08:12 apt
-rw-r-----   1 syslog adm              15K Jun 28 08:32 auth.log
-rw-r-----   1 syslog adm              61K Jun 27 13:58 auth.log.1
-rw-r-----   1 syslog adm              11K Jun 17 22:04 auth.log.2.gz
-rw-r-----   1 syslog adm             7.2K Jun 13 07:44 auth.log.3.gz
-rw-r-----   1 syslog adm              11K Jun  6 09:35 auth.log.4.gz
-rw-------   1 root   root             16K Jun 28 08:27 boot.log
-rw-------   1 root   root            6.5K Jun 28 08:02 boot.log.1
-rw-------   1 root   root            6.0K Jun 27 13:53 boot.log.2
-rw-------   1 root   root             16K Jun 25 00:00 boot.log.3
-rw-------   1 root   root             50K Jun 24 09:32 boot.log.4
-rw-------   1 root   root            7.5K Jun 22 05:02 boot.log.5
-rw-------   1 root   root            7.3K Jun 21 21:55 boot.log.6
-rw-------   1 root   root             23K Jun 20 16:51 boot.log.7
-rw-r--r--   1 root   root             56K Aug  5  2019 bootstrap.log
-rw-rw----   1 root   utmp             384 Jun 20 16:55 btmp
-rw-rw----   1 root   utmp             768 May 30 22:55 btmp.1
drwxr-xr-x   2 root   root            4.0K Jun 28 08:02 cups
drwxr-xr-x   5 root   root            4.0K Jun  3 00:43 dist-upgrade
-rw-r-----   1 root   adm              88K Jun 28 08:27 dmesg
-rw-r-----   1 root   adm              86K Jun 28 08:02 dmesg.0
-rw-r-----   1 root   adm              22K Jun 27 13:53 dmesg.1.gz
-rw-r-----   1 root   adm              22K Jun 24 21:07 dmesg.2.gz
-rw-r-----   1 root   adm              22K Jun 24 11:13 dmesg.3.gz
-rw-r-----   1 root   adm              22K Jun 24 09:32 dmesg.4.gz
-rw-r--r--   1 root   root            1.6M Jun 28 08:12 dpkg.log
-rw-r--r--   1 root   root            1.9M Jun  3 00:19 dpkg.log.1
-rw-r--r--   1 root   root            3.2K Mar  6  2021 dpkg.log.10.gz
-rw-r--r--   1 root   root             17K Feb 22  2021 dpkg.log.11.gz
-rw-r--r--   1 root   root            5.8K Oct  5  2020 dpkg.log.12.gz
-rw-r--r--   1 root   root             572 May 30 03:36 dpkg.log.2.gz
-rw-r--r--   1 root   root             695 Mar 28 15:54 dpkg.log.3.gz
-rw-r--r--   1 root   root             11K Feb 21 11:50 dpkg.log.4.gz
-rw-r--r--   1 root   root             233 Nov  2  2021 dpkg.log.5.gz
-rw-r--r--   1 root   root            2.2K Sep 13  2021 dpkg.log.6.gz
-rw-r--r--   1 root   root            336K Aug 31  2021 dpkg.log.7.gz
-rw-r--r--   1 root   root             15K Aug 29  2021 dpkg.log.8.gz
-rw-r--r--   1 root   root            2.3K Mar 22  2021 dpkg.log.9.gz
-rw-r--r--   1 root   root             32K Aug 29  2021 faillog
drwxr-xr-x   2 root   root            4.0K Mar  3  2020 firebird
-rw-r--r--   1 root   root             16K Jun 23 03:02 fontconfig.log
-rw-r--r--   1 root   root            2.4K Jun 28 08:27 gpu-manager.log
-rw-r--r--   1 root   root            2.4K Jun 27 21:57 gpu-manager-switch.log
drwxrwxr-x   3 root   root            4.0K Dec 29  2019 installer
drwxr-sr-x+  3 root   systemd-journal 4.0K Dec 29  2019 journal
-rw-r-----   1 syslog adm             323K Jun 28 08:34 kern.log
-rw-r-----   1 syslog adm             2.4M Jun 27 13:56 kern.log.1
-rw-r-----   1 syslog adm             730M Jun 17 21:59 kern.log.2.gz
-rw-r-----   1 syslog adm             261K Jun 13 07:44 kern.log.3.gz
-rw-r-----   1 syslog adm             524K Jun  6 09:35 kern.log.4.gz
-rw-rw-r--   1 root   utmp            286K Aug 29  2021 lastlog
-rw-r-----   1 syslog adm                0 Sep 13  2021 mail.log
-rw-r-----   1 syslog adm              971 Aug 29  2021 mail.log.1
drwxr-xr-x   2 mpd    audio           4.0K Jun  6 09:35 mpd
drwxr--r--   2 root   nordvpn         4.0K Jan 23  2020 nordvpn
-rw-r--r--   1 root   root              20 Jun 28 08:27 prime-supported.log
drwx------   2 root   root            4.0K Aug 29  2021 private
drwxr-x---   3 root   adm             4.0K Jun 27 14:01 samba
-rw-r--r--   1 sddm   sddm               0 Aug 29  2021 sddm.log
-rw-r-----   1 syslog adm             954K Jun 28 08:35 syslog
-rw-r-----   1 syslog adm             1.5M Jun 27 13:58 syslog.1
-rw-r-----   1 syslog adm             731M Jun 17 22:04 syslog.2.gz
-rw-r-----   1 syslog adm             647K Jun 13 07:44 syslog.3.gz
-rw-r-----   1 syslog adm             1.3M Jun  6 09:35 syslog.4.gz
-rw-r-----   1 syslog adm             142K Mar 22  2021 syslog.6.gz
-rw-r-----   1 syslog adm              39K Mar 11  2021 syslog.7.gz
-rw-------   1 root   root             63K Aug 29  2021 tallylog
-rw-r--r--   1 root   root             685 Jun  3 00:43 ubuntu-advantage.log
-rw-------   1 root   root             685 Jun  2 21:26 ubuntu-advantage.log.1
-rw-------   1 root   root             210 Feb 21 10:52 ubuntu-advantage.log.2.gz
-rw-------   1 root   root             406 Jan 21 03:30 ubuntu-advantage.log.3.gz
-rw-------   1 root   root             175 Jan 11 15:17 ubuntu-advantage.log.4.gz
-rw-------   1 root   root             178 Nov  3  2021 ubuntu-advantage.log.5.gz
-rw-------   1 root   root             192 Sep 13  2021 ubuntu-advantage.log.6.gz
-rw-------   1 root   root            3.3K Jun 27 21:23 ubuntu-advantage-timer.log
-rw-------   1 root   root             314 May 30 23:22 ubuntu-advantage-timer.log.1
-rw-------   1 root   root             110 Apr 20 18:03 ubuntu-advantage-timer.log.2.gz
-rw-------   1 root   root             137 Mar 28 15:53 ubuntu-advantage-timer.log.3.gz
drwxr-x---   2 root   adm             4.0K Jun  4 01:28 unattended-upgrades
drwxr-xr-x   2 root   root            4.0K Aug 29  2021 upgrade
drwxr-xr-x   2 root   root            4.0K Jun 28 08:27 vmware
-rw-r--r--   1 root   root             43K Jun 11 04:08 vmware-installer
-rw-r--r--   1 root   root             73K Jun 23 02:58 vnetlib
drwxrwxrwx   2 root   root            4.0K Dec 30  2019 wpslog
-rw-rw-r--   1 root   utmp            302K Jun 28 08:35 wtmp
-rw-rw-r--   1 root   utmp            6.0K Aug 29  2021 wtmp.1
-rw-r--r--   1 root   root             62K Jun 28 08:29 Xorg.0.log
-rw-r--r--   1 root   root             49K Jun 28 08:23 Xorg.0.log.old

Note that the kern.log.2.gz and syslog.2.gz are now 730MB and 731MB respectfully. Is it because of the rotation? It's so much smaller than it was a few days ago.

Do you think it is okay to simply remove them?

Zedd
  • 141
  • 2
    Step 1: Identify the largest files in that directory. – user535733 Jun 24 '22 at 18:06
  • journalctl --vacuum-size=10M – Archisman Panigrahi Jun 25 '22 at 05:02
  • Hi. Thank you for the help. @ArchismanPanigrahi. I've tried the command you suggested. Unfortunately it didn't clean up anything. zedd@zedd-kubuntu:~$ journalctl --vacuum-size=10M Vacuuming done, freed 0B of archived journals from /var/log/journal. Vacuuming done, freed 0B of archived journals from /run/log/journal. Vacuuming done, freed 0B of archived journals from /var/log/journal/266dca3619c3437085714dcd32ae788d. – Zedd Jun 25 '22 at 05:12
  • @user535733 I have identified the largest files in the directory and updated it in the post. Do you think I could remove these 2 files without causing problems? – Zedd Jun 25 '22 at 07:31
  • Take a manual backup to an external HDD and delete the unwanted files ? – Benin Jun 25 '22 at 07:34
  • 13
    If I were you, I would be less concerned about the size of the log files and more about the quality of your incident monitoring, reporting, alerting, and alarming: clearly, something significant has happened sometime between June, 13th and June, 17th, and you are only finding out about this on June, 24th? That sounds like a problem to me. – Jörg W Mittag Jun 25 '22 at 14:38
  • 3
    27G Jun 17 22:04 syslog.1 that's suspicious all by itself. – Mast Jun 25 '22 at 22:32
  • 6
    @JörgWMittag For all we know, this may just be an unimportant private desktop machine, where OP noticed this problem because disk space ran low. "Incident monitoring" might be a bit much to expect. For a production server? Sure, but not every Linux machine is that. – marcelm Jun 26 '22 at 08:48

8 Answers8

11

First, have a look at the two files you identified. (Some graphical editors might choke at a 27GB file, but less should have no problem displaying the file contents). From there, you should identify which program is causing the log files to grow to such enormous size. My guess is it's one program logging the same thing over and over again. Feel free to ask another question on this site on how to fix whatever is causing this.

Once you've done this, feel free to remove the huge files. (Or backup and remove, if you want to dig into this further.) They have already been rotated by logrotate (you can see that by the .1 extension), so there's nothing writing to them anymore.

David Z
  • 10,301
Sören
  • 223
  • Thanks for the advice. Let me see what I can find. I'll report here about the problem. – Zedd Jun 25 '22 at 11:09
  • 3
    Good catch … looking at the dates, it’s obvious that whatever issue caused the kern.log and syslog files to bloat was temporary and is not an issue anymore … so it appears that if dealt with these existing two files, it shouldn’t occur again. – Raffa Jun 25 '22 at 11:15
  • I am guessing this is due to a failed python installation I had a while back. I downloaded a Python-3.10.5 package and installed the program according to the README.rst file inside. The system was frozen before the installation had been completed, and the next time I logged in, the whole partition was full. What do you think about simply deleting these 2 files? – Zedd Jun 28 '22 at 15:51
9

It will be - most likely - the journal that eats up your space. If you don't mind not getting the logs from last year you might try:

journalctl --vacuum-time=2d

which will remove all logs older than 2 days. Your mileage may vary, so replace the "2d" with the number of days you'd like to retain.

kanehekili
  • 6,402
  • 4
    Thank you for the reply. This is helpful and I have freed 3.9 GB from the archived journals. But 3.9 GB is relatively small compared to 56.6 GB in total. Do you have other ways to remove more files in this case? – Zedd Jun 25 '22 at 04:19
6

In addition to @kanehehili's answer, you may want to automatically remove old log files from /var/log with the package logrotate.

If you enable the compress and compresscmd xz options in /etc/logrotate.conf you will be able to store a large amount of log data with a small footprint. For more info: man logrotate.conf

You can also search for files based on modification date (greater than 7 days) and automatically remove them with find /var/log -type f -mtime +7 -exec rm {} \;

From your update, we can see that kern.log is huge, and all of this info also exists in syslog. You can prevent this duplication by removing the kern.log line from /etc/rsyslog.d/50-default.conf

ThankYee
  • 1,708
2

Since kern.log.1 and syslog.1 both have the .1 suffix, it seems they have been already been rotated once by logrotate. On the next log rotatíon, those files would apparently get gzipped and would receive the .2.gz suffix.

The corresponding "active" log files are kern.log and syslog respectively. So you could remove the files with the .1 suffix, but given that the older gzipped logs are in the single-digit megabytes range or below, in your position I would be really interested in knowing what caused such a huge flood of log messages between 13th and 17th of June.

In the best case, it was caused by a known software bug and a fix for it has already been automatically installed (see dpkg.log to find any recent package updates). In the worst case, the flood of messages might be the first warning sign of an impending hardware failure of some sort.

(Of course you have up-to-date backups of any important files, don't you?)

telcoM
  • 294
  • Does this answer your question? https://askubuntu.com/questions/436051/i-cannot-clear-syslog-but-i-can-remove-it and https://askubuntu.com/questions/239455/how-do-i-stop-var-log-kern-log-1-from-consuming-all-my-disk-space – newcat1000 Jun 25 '22 at 22:04
  • Hi Thank you for the reply. Just as you mentioned, the files now have the .2.gz suffix. However, they have become much smaller than they were before. I think the reason for the gigantic file size is due to a failed software installation a while back. The system had frozen before the installation was completed. Do you think it is OK to simply remove the log files with the number suffix? Since the system won't need it in the future – Zedd Jun 28 '22 at 16:09
  • Certainly you can remove it, especially now that you have a good theory on the probable cause, so you hopefully aren't missing any important warning signs. – telcoM Jun 28 '22 at 16:32
  • 1
    @Zedd, if they had a lot of identical lines then the compression (which results in the .gz extension) could go from GB to kB even (more likely MB though). On a home system you can literally delete the log files with no effect other than then being missing if you have a problem later. I'd set them to be limited to say 10M and let the system rotate them. – pbhj Jul 14 '22 at 15:57
0

As @Sören has already pointed out, find out what happend and filled your logs.

Logrotate

Then check your logrotate config, since syslog logs should have been compressed days ago, seems yours is weekly ?

A default config of /etc/logrotate.d/rsyslog should look like this:

    /var/log/syslog
{
        rotate 7
        daily
        missingok
        notifempty
        delaycompress
        compress
        postrotate
                /usr/lib/rsyslog/rsyslog-rotate
        endscript
}

/var/log/mail.info /var/log/mail.warn /var/log/mail.err /var/log/mail.log /var/log/daemon.log /var/log/kern.log /var/log/auth.log /var/log/user.log /var/log/lpr.log /var/log/cron.log /var/log/debug /var/log/messages { rotate 4 weekly missingok notifempty compress delaycompress sharedscripts postrotate /usr/lib/rsyslog/rsyslog-rotate endscript }

As you can see, syslog should be rotated daily and should look something like this

ls /var/log/syslog* -lha
-rw-r----- 1 syslog adm 340K Jun 27 08:22 /var/log/syslog
-rw-r----- 1 syslog adm 219K Jun 21 06:25 /var/log/syslog-20220621.gz
-rw-r----- 1 syslog adm 217K Jun 22 06:25 /var/log/syslog-20220622.gz
-rw-r----- 1 syslog adm 214K Jun 23 06:25 /var/log/syslog-20220623.gz
-rw-r----- 1 syslog adm 218K Jun 24 06:25 /var/log/syslog-20220624.gz
-rw-r----- 1 syslog adm 216K Jun 25 06:25 /var/log/syslog-20220625.gz
-rw-r----- 1 syslog adm 208K Jun 26 06:25 /var/log/syslog-20220626.gz
-rw-r----- 1 syslog adm 4.1M Jun 27 06:25 /var/log/syslog-20220627

If your config file looks like the one above, you can try to find out what went wrong via the logs. Logrotate itself doesn't have its own log (by default) but you should see "CRON" entries for logrotatein your syslog, which should report errors if something failed.

Other than that, there is also /var/lib/logrotate/status which should tell you the last time a log has been rotated.

Robert Riedl
  • 4,351
0

I have this nasty feeling that this problem can (will?) recur. Before nuking log files, I would go through them to find out what caused them to grow this big in the first place. You don't have to cat the whole multi-gb file - just tailing the last 1000 lines (tail /var/log/syslog.1) or so should give you an idea of the package/software that caused the problem.

0

You can safely remove anything that's a regular file (so not folders and symlink) with rm, the program will automatically create a new log file when needed.

In particular, only the file that do not end with .[number] are currently used: the .[number] ones contain very old logs that have been rotated out, and likely aren't going to be very useful for their age alone, so, what you can do, it to make a root cron job that periodically deletes them, for example this:

@weekly find /var/log -regex '.*\.[0-9]+\(\.gz\)?$' -exec rm {} \;

This cron job will look run every week, and will delete all the regular files in /var/log (and subfolders) that end with ".[number]" or ".[number].gz".

lSON4X
  • 57
-1

First, Set a reasonable maximum log size for systemd

sudo journalctl --vacuum-size=50M

to remove current system logs launch this command in terminal:

sudo rm -v /var/log/*log*
Alexif_47
  • 157
  • 3
  • 2
    That might not be a good idea … while you might delete already rotated log files … you shouldn’t do that for current log files … emty them with like > logfile instead and only if you have to. – Raffa Jun 26 '22 at 04:16
  • Never do rm on a log file. The file might be oipen. Do sudo > /var/log/syslog and sudo > /var/log/messages.log etc (the files ending in gz and/or a number are archives so safe to rm) – Rinzwind Jun 28 '22 at 16:04