a process started with sudo and put in background still exist after ssh disconnect

Question

Here is the procedure I use:

I'm connecting to Ubuntu 22.04 using ssh.
I'm editing a file with vim (or emacs) using sudo
I'm leaving the editor with Ctrl-Z

Then, I close the ssh connection. Either I force to close the ssh connection or the connection is closed for network reasons.

Before I close the ssh connection, this is what I have on my Ubuntu:

userxxx   468394  468393  0 11:04 pts/2    00:00:00 -bash
root      469168  265462  0 12:43 ?        00:00:00 sshd: [accepted]
sshd      469169  469168  0 12:43 ?        00:00:00 sshd: [net]
root      469177  468394  0 12:43 pts/2    00:00:00 sudo vim hello-world.txt
root      469180  265462  0 12:43 ?        00:00:00 sshd: [accepted]
root      469181  469177  0 12:43 pts/0    00:00:00 sudo vim hello-world.txt
root      469182  469181  0 12:43 pts/0    00:00:00 vim hello-world.txt

After I come back with another ssh connection, this is what I have:

root      469168  265462  0 12:43 ?        00:00:00 sshd: [accepted]
sshd      469169  469168  0 12:43 ?        00:00:00 sshd: [net]
root      469177       1  0 12:43 ?        00:00:00 sudo vim hello-world.txt
root      469180  265462  0 12:43 ?        00:00:00 sshd: [accepted]
root      469181  469177  0 12:43 ?        00:00:00 sudo vim hello-world.txt
root      469182  469181  0 12:43 ?        00:00:00 vim hello-world.txt

When I left, the process parent for 469177 was changed from 468394 to 1?

I tried on a Debian and I have a different behavior: the process sudo vim is killed. When I use an editor without sudo, the process vim is killed.

QUESTIONS:

Can someone confirm that the process sudo vim is expected to be killed upon termination of the ssh connection?
Can someone give an explanation on the reason it's not killed on Ubuntu?

Please let me know if more details are needed.

All processes need a parent except process 1 which is the top of the process tree. On ssh disconnect (tty closes) all your processes get the signal "HUP" (Hangup). But "Stopped" processes can not handle the "SIGHUP" to exit. They are left and as they need a new parent, they get process 1 as parent. As the ttys are gone, you will never be able to attach to the stopped process again. Better use screen or tmux or similar. Other login options are configured in /etc/systemd/logind.conf, see man logind.conf — Marco, Jun 02 '23 at 13:39
Thanks for the interesting and useful comment. However, my use-case is only happening for process started with sudo: if I use vim without sudo, the "stopped" process will not be there after ssh disconnect. It will not get process 1 as parent. Also, on debian, a sudo "stopped" process will not be there after ssh disconnect. — AymericM, Jun 02 '23 at 14:30
@user535733 i would not accept it because it doesn't answer the question. For example, a "stopped" process which is not started with sudo is killed, even if it cannot handle the signal. I feel like the answer cannot be related to signal handling. (And I can use kill -9 to destroy a stopped process.) Thanks! — AymericM, Jun 03 '23 at 14:21
Welcome to the Ask Ubuntu community. As @Marco calls out, /etc/systemd/logind.conf can be configured, through the KillUserProcesses boolean option, to either keep or kill a process when a user logs out. From the man page: "Configures whether the processes of a user should be killed when the user logs out." While I do know that Debian defaults that bool to yes, I don't know if current releases of Ubuntu do the same. — richbl, Jun 05 '23 at 00:47
Hello @richbl! I was confused by the advise to use tmux or screen so I missed the man logind.conf advise. That's most probably the knowledge I was missing and most probably the way to answer my question? Feel free to try an answer! Thanks a lot. — AymericM, Jun 05 '23 at 08:42
@Marco Just a friendly minor nitpicking comment ... "you will never be able to attach to the stopped process again." might not be quiet the verdict ... Please see for example https://askubuntu.com/a/1419821 — Raffa, Jun 05 '23 at 15:46

score 3 · Accepted Answer · answered Jun 05 '23 at 12:13

3

As @Marco calls out in the comments section, /etc/systemd/logind.conf is your friend. User processes can be configured through the KillUserProcesses boolean option in this conf file, to either keep or kill a process when a user logs out.

From the Ubuntu 22.04 manpage: "Configures whether the processes of a user should be killed when the user logs out."

As you had noted in Debian's behavior, Debian defaults that bool to yes, while the manpage for Ubuntu indicates that the default is set to no.

answered Jun 05 '23 at 12:13

richbl

2,303

Thanks again! I was able to have my sudo vi hello.txt being destroyed by using KillUserProcesses=yes on Ubuntu. man logind.conf on Debian indicates KillUserProcesses=no as default. But I found /usr/share/doc/systemd/README.Debian.gz some wording that seems to indicate KillUserProcesses=yes is the default. I have now understood the behavior difference between Ubuntu and Debian. I still do not understand why: when I use sudo vi, the process is not destroyed when leaving the session VS use vi, process is destroyed. May be there is no parent for vi available? – AymericM Jun 05 '23 at 16:55
1

@AymericM Not an answer, but a few eye openers … vim is a userspace application/program and therefore requires a user runtime-environment … That one is handled/created by systemd units … User processes are grouped via scope units which are in turn created using DBUS which provides an inter-process-communication-path socket as well and is a part of the user's runtime environment … Normally login will initiate a user runtime-environment and logout will terminate it … Using sudo, your process gets owned by root away from your user's scope. – Raffa Jun 05 '23 at 17:32
Thanks @Raffa, I do consider that your comment complete the answer to the question I had in mind! I should have asked why vim without sudo was destroyed instead of why sudo vim was not destroyed. I'm still a little surprised that using KillUserProcesses=yes and KillExcludeUsers=root introduce a different behavior for a sudo process compared to KillUserProcesses=no and KillExcludeUsers=root. If it's a sudo process, then it's root and thus, process should be excluded from being killed, no matter KillUserProcesses.... That's how I read man logind.conf.... – AymericM Jun 05 '23 at 19:30
2

@AymericM Added that in an answer because it doesn't fit in a comment ... I hope it helps ... And richbl +1 for a straight to the point answer. – Raffa Jun 06 '23 at 16:44

Raffa · Answer 2 · 2023-06-06T18:46:08.060

This is not a direct answer to your main question, but rather to your comment:

Thanks @Raffa, I do consider that your comment complete the answer to the question I had in mind! I should have asked why vim without sudo was destroyed instead of why sudo vim was not destroyed. I'm still a little surprised that using KillUserProcesses=yes and KillExcludeUsers=root introduce a different behavior for a sudo process compared to KillUserProcesses=no and KillExcludeUsers=root. If it's a sudo process, then it's root and thus, process should be excluded from being killed, no matter KillUserProcesses.... That's how I read man logind.conf....

In as limited scope as possible and as simple terminology as possible and with demonstration.

Lets look behind the scenes of when you SSH to your server.

test@localhost is my test demonstration user ... So, lets connect:

ubuntu@Lenovo:~$ ssh test@localhost
test@localhost's password: 
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.19.0-43-generic x86_64)

Documentation:  https://help.ubuntu.com
Management:     https://landscape.canonical.com
Support:        https://ubuntu.com/advantage

Expanded Security Maintenance for Applications is not enabled.
9 updates can be applied immediately.
To see these additional updates run: apt list --upgradable
21 additional security updates can be applied with ESM Apps.
Learn more about enabling ESM Apps service at https://ubuntu.com/esm
Last login: Tue Jun  6 17:22:18 2023 from 127.0.0.1
test@Lenovo:~$

Now connected and logged-in ... Lets see what login looks like behind the scenes(remember our scope is limited to and focused on the behavior in question):

test@Lenovo:~$ loginctl user-status --no-pager test
test (1004)
       Since: Tue 2023-06-06 18:22:53 +03; 4s ago
       State: active
    Sessions: *455
      Linger: no
        Unit: user-1004.slice
          ├─session-455.scope
          │ ├─60990 "sshd: test [priv]" "" "" ""
          │ ├─61052 "sshd: test@pts/2" "" "" "" ""
          │ ├─61055 -bash
          │ └─61114 loginctl user-status --no-pager test
          └─user@1004.service
            ├─app.slice
            │ ├─dbus.service
            │ │ ├─61024 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
            │ │ ├─61097 /usr/libexec/goa-daemon
            │ │ └─61104 /usr/libexec/goa-identity-service
            │ ├─gvfs-afc-volume-monitor.service
            │ │ └─61084 /usr/libexec/gvfs-afc-volume-monitor
            │ ├─gvfs-daemon.service
            │ │ └─61037 /usr/libexec/gvfsd
            │ ├─gvfs-goa-volume-monitor.service
            │ │ └─61093 /usr/libexec/gvfs-goa-volume-monitor
            │ ├─gvfs-gphoto2-volume-monitor.service
            │ │ └─61080 /usr/libexec/gvfs-gphoto2-volume-monitor
            │ ├─gvfs-mtp-volume-monitor.service
            │ │ └─61089 /usr/libexec/gvfs-mtp-volume-monitor
            │ └─gvfs-udisks2-volume-monitor.service
            │   └─61075 /usr/libexec/gvfs-udisks2-volume-monitor
            ├─background.slice
            │ ├─tracker-extract-3.service
            │ │ └─61007 /usr/libexec/tracker-extract-3
            │ └─tracker-miner-fs-3.service
            │   └─61061 /usr/libexec/tracker-miner-fs-3
            ├─init.scope
            │ ├─60995 /lib/systemd/systemd --user
            │ └─60996 "(sd-pam)"
            └─session.slice
              ├─pipewire-media-session.service
              │ └─61005 /usr/bin/pipewire-media-session
              ├─pipewire-pulse.service
              │ └─61006 /usr/bin/pipewire-pulse
              └─pipewire.service
                └─61004 /usr/bin/pipewire
يونيو 06 18:22:53 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Activating service name='org.gnome.Identity' requested by ':1.12' (uid=1004 pid=61097 comm="/usr/libexec/goa-daemon " label="unconfined")
يونيو 06 18:22:53 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Successfully activated service 'org.gnome.OnlineAccounts'
يونيو 06 18:22:53 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Successfully activated service 'org.gtk.vfs.GoaVolumeMonitor'
يونيو 06 18:22:53 Lenovo systemd[60995]: Started Virtual filesystem service - GNOME Online Accounts monitor.
يونيو 06 18:22:53 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Successfully activated service 'org.gnome.Identity'
يونيو 06 18:22:54 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Successfully activated service 'org.freedesktop.Tracker3.Miner.Files'
يونيو 06 18:22:54 Lenovo systemd[60995]: Started Tracker file system data miner.
يونيو 06 18:22:54 Lenovo systemd[60995]: Started Tracker metadata extractor.
يونيو 06 18:22:54 Lenovo systemd[60995]: Reached target Main User Target.
يونيو 06 18:22:54 Lenovo systemd[60995]: Startup finished in 726ms.
test@Lenovo:~$

From all that you need to know three things:

Linger: no means your login session(s) will be terminated on logout(or ending your session by other means) and as a result all the above listed services, slices, scopes and other units will be stopped, killed, destroyed or "abandoned".
session-455.scope is where user run services will live(not really but rather grouped by reference and governed ... in other words that "scope" is actually(among other things) a definition of what/how those services can "see" or can "be seen").
user@1004.service is what creates your user run-time environment that is essential for userspace applications/programs to run.

Let's zoom-in on session-455.scope:

test@Lenovo:~$ systemctl --no-pager status session-455.scope
● session-455.scope - Session 455 of User test
     Loaded: loaded (/run/systemd/transient/session-455.scope; transient)
  Transient: yes
     Active: active (running) since Tue 2023-06-06 18:22:53 +03; 23min ago
      Tasks: 4
     Memory: 3.5M
        CPU: 166ms
     CGroup: /user.slice/user-1004.slice/session-455.scope
             ├─60990 "sshd: test [priv]" "" "" ""
             ├─61052 "sshd: test@pts/2" "" "" "" ""
             ├─61055 -bash
             └─61470 systemctl --no-pager status session-455.scope
test@Lenovo:~$

Notice how it reports as active (running) ... These are your current running user processes ... So, let's add one more then suspend it with Ctrl + z:

test@Lenovo:~$ vim
[1]+  Stopped                 vim
test@Lenovo:~$

And check the user scope unit again:

test@Lenovo:~$ systemctl --no-pager status session-455.scope
● session-455.scope - Session 455 of User test
     Loaded: loaded (/run/systemd/transient/session-455.scope; transient)
  Transient: yes
     Active: active (running) since Tue 2023-06-06 18:22:53 +03; 28min ago
      Tasks: 6
     Memory: 10.3M
        CPU: 275ms
     CGroup: /user.slice/user-1004.slice/session-455.scope
             ├─60990 "sshd: test [priv]" "" "" ""
             ├─61052 "sshd: test@pts/2" "" "" "" ""
             ├─61055 -bash
             ├─61515 vim
             └─61537 systemctl --no-pager status session-455.scope
test@Lenovo:~$

Nothing dramatic ... Just the new user process is now added to the scope.

Let's now run one more process, but with sudo then suspend it:

test@Lenovo:~$ sudo vim
[sudo] password for test:
[2]+  Stopped                 sudo vim
test@Lenovo:~$

And check again:

test@Lenovo:~$ systemctl --no-pager status session-455.scope
● session-455.scope - Session 455 of User test
     Loaded: loaded (/run/systemd/transient/session-455.scope; transient)
  Transient: yes
     Active: active (running) since Tue 2023-06-06 18:22:53 +03; 34min ago
      Tasks: 10
     Memory: 18.7M
        CPU: 432ms
     CGroup: /user.slice/user-1004.slice/session-455.scope
             ├─60990 "sshd: test [priv]" "" "" ""
             ├─61052 "sshd: test@pts/2" "" "" "" ""
             ├─61055 -bash
             ├─61515 vim
             ├─61610 sudo vim
             ├─61611 sudo vim
             ├─61612 vim
             └─61637 systemctl --no-pager status session-455.scope
يونيو 06 18:56:27 Lenovo sudo[61610]:     test : TTY=pts/2 ; PWD=/home/test ; USER=root ; COMMAND=/usr/bin/vim
يونيو 06 18:56:27 Lenovo sudo[61610]: pam_unix(sudo:session): session opened for user root(uid=0) by test(uid=1004)
test@Lenovo:~$

Now, something dramatic happened ... A new user session is opened for root(technically a sudo session) ... This session is opened from within your user scope, but in reality it's somebody else's(i.e. the user root) session ... Now your user's scope is no more entirely yours alone and therefore tied with that user's session as well (waiting for it to close to be all yours again) ... In optimal situations, that root session should close after your user application i.e. vim exits ... But, what if situations are not optimal ... Let's see how our processes look:

test@Lenovo:~$ ps -ejf | { head -n 1; grep vim; }
UID          PID    PPID    PGID     SID  C STIME TTY          TIME CMD
test       61515   61055   61515   61055  0 18:49 pts/2    00:00:00 vim
root       61610   61055   61610   61055  0 18:56 pts/2    00:00:00 sudo vim
root       61611   61610   61611   61611  0 18:56 pts/3    00:00:00 sudo vim
root       61612   61611   61612   61611  0 18:56 pts/3    00:00:00 vim
test@Lenovo:~$

Then just kill the ssh session and give it the SIGHUP:

test@Lenovo:~$ pkill --signal SIGHUP "ssh"
pkill: killing pid 1782 failedConnection to localhost closed by remote host.
Connection to localhost closed.
ubuntu@Lenovo:~$

Then, connect and login back again to see what happened:

ubuntu@Lenovo:~$ ssh test@localhost
test@localhost's password: 
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.19.0-43-generic x86_64)

Documentation:  https://help.ubuntu.com
Management:     https://landscape.canonical.com
Support:        https://ubuntu.com/advantage

Expanded Security Maintenance for Applications is not enabled.
9 updates can be applied immediately.
To see these additional updates run: apt list --upgradable
21 additional security updates can be applied with ESM Apps.
Learn more about enabling ESM Apps service at https://ubuntu.com/esm
Last login: Tue Jun  6 18:22:53 2023 from 127.0.0.1
test@Lenovo:~$ loginctl user-status --no-pager test
test (1004)
       Since: Tue 2023-06-06 18:22:53 +03; 1h 2min ago
       State: active
    Sessions: *479 455
      Linger: no
        Unit: user-1004.slice
          ├─session-455.scope
          │ ├─61610 sudo vim
          │ ├─61611 sudo vim
          │ └─61612 vim
          ├─session-479.scope
          │ ├─62030 "sshd: test [priv]" "" "" ""
          │ ├─62107 "sshd: test@pts/4" "" "" "" ""
          │ ├─62108 -bash
          │ └─62137 loginctl user-status --no-pager test
          └─user@1004.service
            ├─app.slice
            │ ├─dbus.service
            │ │ ├─61024 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
            │ │ ├─61097 /usr/libexec/goa-daemon
            │ │ └─61104 /usr/libexec/goa-identity-service
            │ ├─gvfs-afc-volume-monitor.service
            │ │ └─61084 /usr/libexec/gvfs-afc-volume-monitor
            │ ├─gvfs-daemon.service
            │ │ └─61037 /usr/libexec/gvfsd
            │ ├─gvfs-goa-volume-monitor.service
            │ │ └─61093 /usr/libexec/gvfs-goa-volume-monitor
            │ ├─gvfs-gphoto2-volume-monitor.service
            │ │ └─61080 /usr/libexec/gvfs-gphoto2-volume-monitor
            │ ├─gvfs-mtp-volume-monitor.service
            │ │ └─61089 /usr/libexec/gvfs-mtp-volume-monitor
            │ └─gvfs-udisks2-volume-monitor.service
            │   └─61075 /usr/libexec/gvfs-udisks2-volume-monitor
            ├─background.slice
            │ └─tracker-miner-fs-3.service
            │   └─61061 /usr/libexec/tracker-miner-fs-3
            ├─init.scope
            │ ├─60995 /lib/systemd/systemd --user
            │ └─60996 "(sd-pam)"
            └─session.slice
              ├─pipewire-media-session.service
              │ └─61005 /usr/bin/pipewire-media-session
              ├─pipewire-pulse.service
              │ └─61006 /usr/bin/pipewire-pulse
              └─pipewire.service
                └─61004 /usr/bin/pipewire
يونيو 06 18:22:53 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Successfully activated service 'org.gtk.vfs.GoaVolumeMonitor'
يونيو 06 18:22:53 Lenovo systemd[60995]: Started Virtual filesystem service - GNOME Online Accounts monitor.
يونيو 06 18:22:53 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Successfully activated service 'org.gnome.Identity'
يونيو 06 18:22:54 Lenovo dbus-daemon[61024]: [session uid=1004 pid=61024] Successfully activated service 'org.freedesktop.Tracker3.Miner.Files'
يونيو 06 18:22:54 Lenovo systemd[60995]: Started Tracker file system data miner.
يونيو 06 18:22:54 Lenovo systemd[60995]: Started Tracker metadata extractor.
يونيو 06 18:22:54 Lenovo systemd[60995]: Reached target Main User Target.
يونيو 06 18:22:54 Lenovo systemd[60995]: Startup finished in 726ms.
يونيو 06 18:56:27 Lenovo sudo[61610]:     test : TTY=pts/2 ; PWD=/home/test ; USER=root ; COMMAND=/usr/bin/vim
يونيو 06 18:56:27 Lenovo sudo[61610]: pam_unix(sudo:session): session opened for user root(uid=0) by test(uid=1004)
test@Lenovo:~$

The user session is still alive(see the login time) and we still have the user scope session-455.scope:

test@Lenovo:~$ systemctl --no-pager status session-455.scope
● session-455.scope - Session 455 of User test
     Loaded: loaded (/run/systemd/transient/session-455.scope; transient)
  Transient: yes
     Active: active (abandoned) since Tue 2023-06-06 18:22:53 +03; 1h 5min ago
      Tasks: 4
     Memory: 8.5M
        CPU: 1.385s
     CGroup: /user.slice/user-1004.slice/session-455.scope
             ├─61610 sudo vim
             ├─61611 sudo vim
             └─61612 vim
يونيو 06 18:56:27 Lenovo sudo[61610]:     test : TTY=pts/2 ; PWD=/home/test ; USER=root ; COMMAND=/usr/bin/vim
يونيو 06 18:56:27 Lenovo sudo[61610]: pam_unix(sudo:session): session opened for user root(uid=0) by test(uid=1004)
test@Lenovo:~$

With processes that we recognize and still reports as Active, but now with (abandoned) instead of (running) ... That's when your user session lost ultimate control on that scope ... KillUserProcesses=true is for that type of situation i.e. "abandoned" processes or entire scopes as clearly stated in man logind.conf:

KillUserProcesses=
   Takes a boolean argument. Configures whether the processes of a user should be killed
   when the user logs out. If true, the scope unit corresponding to the session and all
   processes inside that scope will be terminated. If false, the scope is "abandoned",
   see systemd.scope(5), and processes are not killed. Defaults to "no", but see the
   options KillOnlyUsers= and KillExcludeUsers= below ...

KillExcludeUsers=root would save an "abandoned" .scope unit under root login session and not yours … It works on whole user scope units and not on single user processes unless those processes are running under the user main session without a scope unit.

Let's have a closer look at our processes again:

test@Lenovo:~$ ps -ejf | { head -n 1; grep vim; }
UID          PID    PPID    PGID     SID  C STIME TTY          TIME CMD
root       61610       1   61610   61055  0 18:56 ?        00:00:00 sudo vim
root       61611   61610   61611   61611  0 18:56 ?        00:00:00 sudo vim
root       61612   61611   61612   61611  0 18:56 ?        00:00:00 vim
test@Lenovo:~$

voila.

I will keep the original selected answer because you said it's not a direct answer to my question. However, this was exactly what I was looking for!!!! — AymericM, Jun 06 '23 at 21:54

a process started with sudo and put in background still exist after ssh disconnect

2 Answers2

Linked