What is soaking up my free memory?

Question

I have computer handling various functions for our community radio station, including processing the program audio for broadcast. It's running remotely and sends daily health emails for me to keep an eye on its status. I am seeing free memory drop - by quite a lot - over the course of a few days and can't account where it is going.

The report contains the output of "free" and the top 20 processes using memory:

ps -eo comm:15,pid,ppid,cmd,%mem,%cpu --sort=-%mem | head -n 20

Two days ago:

Memory stats as of 10/17/2023
               total        used        free      shared  buff/cache   available
Mem:            12Gi       3.1Gi       7.4Gi       160Mi       2.1Gi       9.0Gi
Swap:             0B          0B          0B
Total:          12Gi       3.1Gi       7.4Gi
MEM stats
COMMAND             PID    PPID CMD                         %MEM %CPU
ST_FM              2214       1 /home/kpov_tech/Desktop/Ste  4.6 24.1
ST_Stream          2256       1 /home/kpov_tech/Desktop/Ste  4.5 21.6
plasmashell        1759    1689 /usr/bin/plasmashell         2.0  0.2
kwin_x11           1721    1689 /usr/bin/kwin_x11            1.8 10.7
fluidsynth         1538    1528 /usr/bin/fluidsynth -is /us  1.8  0.0
kded5              1718    1689 /usr/bin/kded5               1.8  0.0
audacious        871581  827042 /usr/bin/audacious --play    1.6  1.6
Xorg               1259    1218 /usr/lib/xorg/Xorg -noliste  1.3 16.8
dolphin          774584    1759 /usr/bin/dolphin             1.2  0.1
kwrite            44322       1 /usr/bin/kwrite /home/kpov_  1.1  0.1
konsole            2097       1 konsole --noclose --profile  1.0  0.1
raysession         2307       1 /usr/bin/python3 -u /usr/sh  1.0  0.3
jackd              2111    2097 /usr/bin/jackd -v -d alsa -  1.0  0.7
zita-mu1           2173       1 /usr/bin/zita-mu1 -name MON  0.9  0.4
ebumeter           2191       1 ebumeter -name ebu_Raw_MCR   0.9  1.0
ebumeter           2207       1 ebumeter -name ebu_ListenLi  0.9  0.9
ebumeter           2200       1 ebumeter -name ebu_ST_FM     0.9  0.9
ebumeter           2184       1 ebumeter -name ebu_ST_Strea  0.9  0.9
ray-jackpatch_t    2330       1 /usr/bin/python3 -u /usr/sh  0.9  0.1

Today:

Memory stats as of 10/19/2023
               total        used        free      shared  buff/cache   available
Mem:            12Gi       3.8Gi       529Mi       155Mi       8.2Gi       8.2Gi
Swap:             0B          0B          0B
Total:          12Gi       3.8Gi       529Mi
MEM stats
COMMAND             PID    PPID CMD                         %MEM %CPU
ST_FM              2214       1 /home/kpov_tech/Desktop/Ste  4.6 24.1
ST_Stream          2256       1 /home/kpov_tech/Desktop/Ste  4.5 21.8
plasmashell        1759    1689 /usr/bin/plasmashell         2.4  0.2
fluidsynth         1538    1528 /usr/bin/fluidsynth -is /us  1.8  0.0
kwin_x11           1721    1689 /usr/bin/kwin_x11            1.8  9.5
audacious        871581  827042 /usr/bin/audacious --play    1.5  1.6
Xorg               1259    1218 /usr/lib/xorg/Xorg -noliste  1.2 16.4
dolphin         3467064    1759 /usr/bin/dolphin             1.1  0.1
kwrite            44322       1 /usr/bin/kwrite /home/kpov_  1.0  0.1
kded5              1718    1689 /usr/bin/kded5               1.0  0.0
jackd              2111    2097 /usr/bin/jackd -v -d alsa -  1.0  0.7
konsole            2097       1 konsole --noclose --profile  1.0  0.0
raysession         2307       1 /usr/bin/python3 -u /usr/sh  1.0  0.3
xdg-desktop-por 3450540    1528 /usr/lib/x86_64-linux-gnu/l  0.9  0.0
zita-mu1           2173       1 /usr/bin/zita-mu1 -name MON  0.9  0.4
ebumeter           2191       1 ebumeter -name ebu_Raw_MCR   0.9  1.0
ebumeter           2207       1 ebumeter -name ebu_ListenLi  0.9  0.9
ebumeter           2200       1 ebumeter -name ebu_ST_FM     0.9  0.9
ebumeter           2184       1 ebumeter -name ebu_ST_Strea  0.9  0.9

It looks to me that something like 7Gi of Free memory has evaporated. How can I find out what is using it?

Comment: I should know better to post a question before finishing my first cup of coffee. The headline should read "WHAT is soaking up my Free memory?" — thatjackelliott, Oct 19 '23 at 14:18
You can always [edit] your own posts. That said, is there an actual problem here? You still have 8+GB of available memory, actual use seems to have only gone by a few 100 MBs. — muru, Oct 19 '23 at 14:23
Try these sync; echo 1 > /proc/sys/vm/drop_caches will drop cache in the " buff/cache" column. Change to 2 or 3 for more aggressive clearing. 3 is likely to slow your system down for a couple of seconds. If that clears free up don't worry about it. It is what it says... a buffer/cache ;) The "available" column is the important one :) — Rinzwind, Oct 19 '23 at 14:28
The 8.2Gi "buff/cache" is technically free considering used memory pages ... It's used for caching disk I/O operations, but is readily available when needed ... So, consider it as free. — Raffa, Oct 19 '23 at 14:30
Does this answer your question? What is available memory while using free command? — muru, Oct 19 '23 at 14:50
For Rinzwind's suggestion to drop cache, yes that added about 7Gi to the Free memory. So, it's just buffered/cached memory that's sitting around with stuff in it, but stuff that can cleared if needed. Thank you. — thatjackelliott, Oct 19 '23 at 17:03

score 6 · Accepted Answer · answered Oct 19 '23 at 14:29

6

From the output of the "free" application it is apparent that you have approximately 8-9Gi of memory available.

What might be confusing to you is that under the column labeled "free" you see a very small amount of memory - this is normal behavior.

You have to factor into your available memory the amount under the "buff/cache" column - Linux uses available memory as buffers to help speed up other processes when possible, and this memory is not unavailable in a traditional sense - you can still use it for other applications if you need to.

You can see the total available memory on your system listed under the "available" column in your logs.

answered Oct 19 '23 at 14:29

kishkovert

168

1

It's like hiring 12 workers and complaining that everybody is trying to do something useful. They're not "used up", they're just doing something. As soon as there's more important work for the RAM workers, they will drop the less important stuff and go for it. – Eike Oct 19 '23 at 15:55
1

Thank you. So I am fretting about a non-problem. But history with this machine (a previous freeze) suggests that I need to keep an eye on memory usage as the sys logs suggested that lack of memory preceded the crash. So, for my use-case, I do want to get an emailed warning if available memory starts looking low. Maybe < 1Gi would be a reasonable amount to warn of possible trouble. Does that seem not unreasonable? – thatjackelliott Oct 19 '23 at 16:57
@thatjackelliott The amount used did rise by ~700 MB, so yes, there might be something. Only you know the system usage patterns though. If memory should basically be a flat line, sending a mail if there's more than say 7 GB used might be fine as well. If it's supposed to go up and down, having only 1 GB free might be a limit. – Eike Oct 19 '23 at 17:46
Thank you. I'll see how it goes! – thatjackelliott Oct 19 '23 at 23:26
Oh darn. Once again I can't find how to mark a topic "solved." – thatjackelliott Oct 19 '23 at 23:28
Oh wait -- before I go. The slow leak, ~700 MB, is going to be a problem because this PC is meant to run unattended 24/7/365 (we hope). Some process or processes below the Top 20 are responsible. It would be smart for me to figure out a way to figure out what's up. Any suggestions? Thank you! – thatjackelliott Oct 20 '23 at 15:43
You can profile the memory usage of the application. If you have access to the source code or your app generates logs, combine logs from profiling the process with detailed logs from the application. If you need more information on that, please check this question on U&L on profiling a single process.
As for marking the question as "solved" - you can mark this answer as accepted if it answered your question..
– kishkovert Oct 22 '23 at 08:09

What is soaking up my free memory?

1 Answers1