6

Currently I have set the scrollback setting of gnome terminal (Edit -> Profile Preferences -> Scrolling) to 10000 lines. I am tempted to tick Unlimited, but I am afraid this could lead to memory crashes (if I leave a program with lots of output running overnight, I wouldn't like it to crash because of this).

My questions are:

  1. Are these scrollback lines stored in memory, or are they cached in a file on disk?

  2. If I change from 10000 to 100000 or to 1 million, am I still safe? What parameters (RAM, free disk space, number of open terminals) I need to take into account)? Is selecting Unlimited safe?

muru
  • 197,895
  • 55
  • 485
  • 740
a06e
  • 13,223
  • 26
  • 70
  • 104

2 Answers2

6

It's apparently stored in deleted files on disk, according to very interesting blog post:

I could just look at what files gnome-terminal had open, so lsof to the rescue. Then I found it was being sneaky, it had a number of files called /tmp/vteXYZ1tv open, but it had already deleted them. Thus you can't see them when browsing, and they will be removed when the program closes. This makes sense, it means that when the process is closed, it doesn't matter how (at least I think), the space of the files can be reclaimed, i.e. we won't get leftover files on a program crash, or a kill -9. They can be restored though, my way (there are probably others), was to do a ls -l /proc/<gnome-terminal pid>/fd to see what they point to. Then you can cat these to make a new file. These are just a verbatim copy of the terminal output. No compression. No nothing. As it turns out, one of my terminal history's was almost 900 MB! But that was only after random data being spat out very quickly for quite a while, unlikely to happen in ordinary usage.

And I'll stress this again: The poster had run base64 < /dev/urandom for a while, looking for increases in memory or disk usage - 900MB would be exceptional usage.

Raffa
  • 32,237
muru
  • 197,895
  • 55
  • 485
  • 740
  • So those 900mb were not in RAM memory, but in the disk? – a06e Mar 27 '15 at 20:26
  • After reading the blog post (nice find, +1), if I understand it correctly, the terminal history goes to a temporary file on the disk, not to the RAM. I guess that when you scroll up, it loads the history from that file back in to memory. Do you agree? – a06e Mar 27 '15 at 20:35
  • @becko I think so too. It might be helped by Linux automatically caching an open file, but I don't think the Terminal itself keeps it in memory. – muru Mar 27 '15 at 20:39
  • I've already accepted your answer, but now I am using xfce terminal. I am wondering if the same reasoning applies (or maybe I should ask a new question?)? – a06e Aug 03 '15 at 18:13
  • @becko try lsof on an xfce-terminal instance. Though there's no reason why XFCE can't implement their own way, this seems a safe and easy way to do it. – muru Aug 03 '15 at 18:16
  • @becko on the other hand, if it's vte that does this, the xfce-terminal will have the same behaviour. Most terminal emulators on Linux use vte. – muru Aug 03 '15 at 18:17
  • 2
    Looks like these are now encrypted (see comment on the blog post above). – jhclark Feb 07 '19 at 22:29
4

gnome-terminal (actually vte) indeed stores the scrollback in an immediately-deleted file under /tmp (more precisely: under the standard tempfile location influenced by $TMPDIR and maybe a few similar ones).

This design was chosen mainly for unlimited scrollback not to cause OOM errors. The disk is way less likely to get filled up, and even if that happens, it's less harmful to the whole system than running out of memory.

vte-0.40 (which will most likely appear in Ubuntu 15.10 W.W.) will compress and encrypt these files. This will shrink the required storage to approx third–fourth of its size (if your app produces X amount of data as plain text, somewhere between X/4 .. X/3 is a reasonable estimate for the storage that'll be required), and also gets rid of the privacy/security issue in case someone gets raw access to the hard drive.

We're planning to add an option to store the scrollback in memory, pretty much equally to as if your /tmp was on tmpfs. If everything goes as planned, this will appear in vte-0.42 and in turn in Ubuntu 16.04 X.X. LTS. I can't promise it, though.

sotirov
  • 3,169
egmont
  • 8,225
  • What about xfce-terminal? I know that was not my original question, but I don't know if both use vte? If they do, the same reasoning applies to both, right? – a06e Aug 03 '15 at 18:15
  • @becko yes, both use vte. Have a look at apt-cache rdepends libvte9. – muru Aug 03 '15 at 18:19
  • xfce-terminal uses the ancient gtk+2 based libvte9 a.k.a. vte-0.28. I can't tell how the scrollback was stored there, but probably the same as in later (but pre-encryption) versions. – egmont Aug 03 '15 at 22:14