Best way to make a shutdown hook?

Question

Since Ubuntu relies on upstart for some time now, I would like to use an upstart job to gracefully shutdown certain applications on system shutdown or reboot. It is essential that the system's shutdown or reboot is stalled until these applications are shut down.

The applications will be started manually on occasion, and on system shutdown should automatically be ended by a script (which I already have). Since the applications can't be ended reliably without (nearly all) other services running, ending the applications has to be done before the rest of the shutdown begins.

I think I can solve this by an upstart job which will be triggered on shutdown, but I am unsure which events I should use in which manner. So far, I have read the following (partly contradicting) statements:

• There is no general shutdown event in upstart
• Use a stanza like start on starting shutdown in the job definition
• Use a stanza like start on runlevel [06S] in the job definition
• Use a stanza like start on starting runlevel [06S] in the job definition
• Use a stanza like start on stopping runlevel [!06S] in the job definition

From these recommendations, the following questions arise:

• Is there or is there not a general shutdown event in Ubuntu's upstart?
• What is the recommended way to implement a "shutdown hook"?
• When are the events runlevel [x] triggered; is this when having entered the runlevel or when entering the runlevel?
• Can we use something like start on starting runlevel [x] or start on stopping runlevel [x]?
• What would be the best solution for my problem?

Thank you very much

Answer 1

In order to stop the shutdown from proceeding while your job stops, you will want to use this:

stop on starting rc RUNLEVEL=[016]

This will work because the first thing that happens when you type 'shutdown' is tunlevel 0 is emitted. rc starts on runlevel, and the transition from stopped -> starting will block completely until any jobs which also must change state do complete that state.

You'll want to make sure that your process responds quickly to SIGTERM. If it does not respond within 5 seconds, upstart will send it SIGKILL. You can raise that with 'kill timeout X'.

The 1 in there, btw is a bit tricky, you need to make sure your start on includes something that starts on runlevel [2345] at that point, so that a user going down for single user mode maintenance gets their job started again. Luckily a lot of work has gone into making this the suggested usual start on

start on runlevel [2345]

Also in some cases you need something to keep running right up until the network is brought down (such as dbus/network-manager). For that you want

stop on deconfiguring-networking

That is an event emitted later in the shutdown that will also be blocked on until any jobs that use it fully complete their transitions in state.

Answer 2

starting and runlevel are separate events, so you can't meaningfully say starting runlevel N.

The runlevel N event is emitted at the start of entering the runlevel. If you start on runlevel N then your task runs on entry. The way to run when entry to the runlevel is completed is run on started rc RUNLEVEL=N.

As I understand it, you need a start on runlevel [06S] to do what you want; it should in theory run before anything else is stopped. For finer control you might use start on stopping apache or stopping mysql or ... so that your task runs before any of them are allowed to be shut down.

---

Edited to change runlevel 5 to S.

Answer 3

Geekosaur, thank you very much for your help.

In the meantime, I have tried the start on runlevel [016] method, but it did not work, and I think I understand why:

The job has been started indeed, but the shutdown process has not been blocked until the job's task was finished. I am quite sure now that the events starting and stopping are the only events which can be used in a job definition to block other jobs, and I think that this is what Upstart's manuals try to tell us. Therefore, using the runlevel event will never lead to blocking other jobs or the shutdown process; thus, it is useless for my purpose.

Instead, I seem to have two possibilities:

1. Following one of your propositions, find out all jobs which the repective applications need, and include all of them in the start event for the script like that:

   start on stopping job1 or stopping job2 or ...
   

   This is so much work that I am seriously thinking of dumping the job list and running it through sed to automatically produce a start stanza for my job which includes all jobs which normally are running on the system.

   The advantage would be that the respective applications would be shut down even when somebody stops one of the prerequisites manually (as opposed to stopping them by a runlevel change / shutdown / reboot).

2. Find the one job which will be stopped at first when rebooting / shutting down the system (let's call this job "FirstJob") and use that job in a stanza like:

   start on stopping FirstJob
   

   The main disadvantages would be that I don't know if such a job exists at all and if that job really depends on all other jobs which the repective application actually depends on ("depends on other job" in this case means "will be stopped completely before other job begins to stop").

I am not sure which of the two possibilities is the better one...