Jetty Continuations for Quality of Service.

Jetty Continuations can be used to prevent resource starvation
caused by JDBC Connection pools or similar slow and/or restricted resources.
Jetty Continuations have mostly been discussed in the context of web 2.0, Ajax Push and Comet. However there are many other use cases and the ThrottlingFilter,
written by Jetty committer Tim Vernum, is an excellent example of how Continuations can be used to provide a consistent quality of service within a web application.
JDBC Connection pools are frequently used to limit the
number of simultaneous requests to a database. The theory
being that databases will give better overall throughput
with fewer simultaneous requests, even if requests have
to wait for a connection from the pool.
This common approach can make your web
application very unstable. Threads queuing on the JDBC
pool can quickly exhaust the entire resources allocated
and deny service to the entire webapp, even if only
few requests actually need the database.
Consider a webapp that receives 1000 requests per second,
95% of which requests are served without a database in 20ms.
The other 5% require a database and take 100ms each.
Crunching the numbers gives:

25 simultaneous requests
5 simultaneous JDBC connections.

But as we are conservative, we will allocate 200 threads
to the servlet container and a 20 connections to the
JDBC pool. The webapp runs well and typically consumes only 25% of the resources allocated.
But what happens if the database runs slow for a while (because the administrator is running a report or an attacker has submitted a overly complex query, etc. etc.)?
If the DB request take 500ms instead of 100ms,
then 10 requests a second will accumulate to those already
blocked on the JDBC connection pool. If the condition lasts
for 20s, then the entire thread pool will be exhausted. No requests will be served for any resource, even those that do not use the database will be starved of threads.
If the database actually fails (or the DBAs report does a table lock), then it will only take 4 seconds before the entire thread pool is blocked on the JDBC pool.
But starvation can occur even without any failures, for example if the URLs that use the database become more popular than normal (due to marketing, slashdot, biorhythms, etc). If 25% of requests hit the database instead of 5%, then 50 requests per seconds will be added to the queue for JDBC connections and starvation will occur in only 4 seconds of increased load.
The solution is the use the thread-less waiting capability of continuations to allow requests to wait for JDBC connections without consuming a thread from the threadpool.
Thus an under performing DB will only slow the parts of the
webapp that use that database. The rest of the webapp will function normally even if the DB fails.
The ThrottlingFilter
can be mapped to arbitrary URLs in web.xml and will limit the number of threads past the filter and the size and timeout of the queue. If you know there are only 20 connections in the JDBC queue, then the ThrottlingFilter can be configured to only allow 20 requests past. Any requests in excess are suspended without a thread allocated and resumed when a connection becomes available as a previous request exists the filter.
The filter may also be used as a base class to provide more sophisticated policy than a simple count.