Graceful shutdown

Having done a little research on application server shutdown in order to answer a customer query I thought I would post my findings. WebSphere Application Server supports three shutdown modes: stop, stop immediate and terminate (in order of immediacy).

A normal stop begins by preventing new inbound HTTP and IIP requests and then has a quiesce period in which in-flight requests are allowed to complete. The maximum time allowed for these requests to complete is 180 seconds (configurable via a JVM property). This maximum is enforced regardless of whether or not a request is part of a global transaction. At the end of this period the application server components (including any in-process messaging engine) then begin to shutdown.

An immediate stop proceeds as for a normal stop except that there is no quiesce period i.e. in-flight requests are given no time to complete before the server components begin to shutdown. A request to terminate a server effectively kills the server process without any shutdown logic running.

Part of the reason for this investigation was the presence of long-running operations in the particular scenario we were looking at. We were therefore looking for a mechanism to notify those operations when a server quiesce began so that they might perform their own graceful shutdown. I had hoped that this might be achieved via the release method of the CommonJ Work interface however this appears to be called at the end of the quiesce period.

In the end I suggested two possible options. The first, was to use JMX to listen for the “server stopping” notification. Shawn Lauzon’s developerWorks article provides some good example code for achieving this. The second option, was to implement an application startup bean, the stop method of which is called at the start of the quiesce period.

The use of the CommonJ WorkManager API would still provide one potential benefit to the long-running operations. The name of the configured WorkManager is included as part of the thread name. Hence, when the hung thread detection logic output its warning message, it would be possible to tell that, in the case of these threads, it could be safely ignored! (Also note that the timeout for hung thread detection can be modified or, indeed, stopped completely.)

Leave a Reply