Many programs must address the problem of handling a series of incoming requests. One simple concurrency strategy is the Thread-Per-Message design pattern, which uses a new thread for each request [Lea 2000a]. This pattern is generally preferred over sequential executions of time-consuming, I/O-bound, session-based, or isolated tasks.
However, the pattern also introduces overheads not seen in sequential execution, including the time and resources required for thread creation and scheduling, for task processing, for resource allocation and deallocation, and for frequent context switching [Lea 2000a]. Furthermore, an attacker can cause a denial of service (DoS) by overwhelming the system with too many requests at once, causing the system to become unresponsive rather than degrading gracefully. From a safety perspective, one component can exhaust all resources because of an intermittent error, consequently starving all other components.
Thread pools allow a system to limit the maximum number of simultaneous requests that it processes to a number that it can comfortably serve rather than terminating all services when presented with a deluge of requests. Thread pools overcome these issues by controlling the maximum number of worker threads that can execute concurrently. Each object that supports thread pools accepts a
Callable<T> task and stores it in a temporary queue until resources become available. Additionally, thread life-cycle management overhead is minimized because the threads in a thread pool can be reused and can be efficiently added to or removed from the pool.
Programs that use multiple threads to service requests should—and programs that may be subjected to DoS attacks must—ensure graceful degradation of service during traffic bursts. Use of thread pools is one acceptable approach to meeting this requirement.
Noncompliant Code Example (Thread-Per-Message)
This noncompliant code example demonstrates the Thread-Per-Message design pattern. The
RequestHandler class provides a public static factory method so that callers can obtain a
RequestHandler instance. The
handleRequest() method is subsequently invoked to handle each request in its own thread.
The thread-per-message strategy fails to provide graceful degradation of service. As threads are created, processing continues normally until some scarce resource is exhausted. For example, a system may allow only a limited number of open file descriptors even though additional threads can be created to serve requests. When the scarce resource is memory, the system may fail abruptly, resulting in a DoS.
Compliant Solution (Thread Pool)
This compliant solution uses a fixed thread pool that places a strict limit on the number of concurrently executing threads. Tasks submitted to the pool are stored in an internal queue. Storing tasks in a queue prevents the system from being overwhelmed when attempting to respond to all incoming requests and allows it to degrade gracefully by serving a fixed maximum number of simultaneous clients [Java Tutorials].
Executoris] an object that executes submitted
Runnabletasks. This interface provides a way of decoupling task submission from the mechanics of how each task will be run, including details of thread use, scheduling, etc. An
Executoris normally used instead of explicitly creating threads.
ExecutorService interface used in this compliant solution derives from the
java.util.concurrent.Executor interface. The
ExecutorService.submit() method allows callers to obtain a
Future<V> object. This object both encapsulates the as-yet unknown result of an asynchronous computation and enables callers to perform additional functions such as task cancellation.
The choice of
newFixedThreadPool is not always appropriate. Refer to the Java API documentation [API 2014] for guidance on choosing among the following methods to meet specific design requirements:
Chapter 8, "Applying Thread Pools"
Section 4.1.3, "Thread-Per-Message"