Jetty-6 Continuations introduced the concept of asynchronous servlets to provide scalability and quality of service to web 2.0 applications such as chat, collaborative editing, price publishing, as well as powering HTTP based frameworks like cometd, apache camel, openfire XMPP and flex BlazeDS. wt58jhp2an

With the introduction of similar  asynchronous features in Servlet-3.0, some have suggested that the Continuation API would be deprecated.  Instead, the Continuation API has been updated to provide a simplified portability run asynchronously on any servlet 3.0 container as well as on Jetty (6,7 & 8).  Continuations will work synchronously (blocking) on any 2.5 servlet container. Thus programming to the Continuations API allows your application to achieve asynchronicity today without waiting for the release of stable 3.0 containers (and needing to upgrade all your associated infrastructure).

Continuation Improvements

The old continuation API threw an exception when the continuation was suspended, so that the thread to exit the service method of the servlet/filter. This caused a potential race condition as a continuation would need to be registered with the asynchronous service before the suspend, so that service could do a resume before the actual suspend, unless a common mutex was used.

Also, the old continuation API had a waiting continuation that would work on non-jetty servers.  However the behaviour of this the waiting continuation was a little different to the normal continuation, so code had to be carefully written to work for both.

The new continuation API does not throw an exception from suspend, so
the continuation can be suspended before it is registered with any
services and the mutex is no longer needed. With the use of a ContinuationFilter for non asynchronous containers, the continuation will now behaive identically in all servers.

Continuations and Servlet 3.0

The servlet 3.0 asynchronous API introduced some additional asynchronous features not supported by jetty 6 continuations, including:

  • The ability to complete an asynchronous request without dispatching
  • Support for wrapped requests and responses.
  • Listeners for asynchronous events
  • Dispatching asynchronous requests to specific contexts and/or resources

While powerful, these additional features may also be very complicated and confusing. Thus the new Continuation API has cherry picked the good ideas and represents a good compromise between power and complexity.  The servlet 3.0 features adopted are:

  • The completing a continuation without resuming.
  • Support for response wrappers.
  • Optional listeners for asynchronous events.


Using The Continuation API

The new continuation API
is available in Jetty-7 and is not expected to significantly change in
future releases.  Also the continuation library is intended to be
deployed in WEB-INF/lib and is portable.  Thus the jetty-7 continuation
jar will work asynchronously when deployed in jetty-6, jetty-7, jetty-8
or any servlet 3.0 container.

Obtaining a Continuation

The ContinuationSupport factory class can be used to obtain a continuation instance associated with a request: 

    Continuation continuation = ContinuationSupport.getContinuation(request);

Suspending a Request

The suspend a request, the suspend method is called on the continuation: 

  void doGet(HttpServletRequest request, HttpServletResponse response)  {    ...    continuation.suspend();    ...  }

After this method has been called, the lifecycle of the request will be extended beyond the return to the container from the Servlet.service(…) method and Filter.doFilter(…) calls. After these dispatch methods return to, as suspended request will not be committed and a response will not be sent to the HTTP client.

Once a request is suspended, the continuation should be registered with an asynchronous service so that it may be used by an asynchronous callback once the waited for event happens.

The request will be suspended until either continuation.resume() or continuation.complete() is called. If neither is called then the continuation will timeout after a default period or a time set before the suspend by a call to continuation.setTimeout(long). If no timeout listeners resume or complete the continuation, then the continuation is resumed with continuation.isExpired() true.
There is a variation of suspend for use with request wrappers and the complete lifecycle (see below):


Suspension is analogous to the servlet 3.0 request.startAsync() method. Unlike jetty-6 continuations, an exception is not thrown by suspend and the method should return normally. This allows the registration of the continuation to occur after suspension and avoids the need for a mutex. If an exception is desirable (to bypass code that is unaware of continuations and may try to commit the response), then continuation.undispatch() may be called to exit the current thread from the current dispatch by throwing a ContinuationThrowable.

Resuming a Request

Once an asynchronous event has occurred, the continuation can be resumed: 

  void myAsyncCallback(Object results)  {    continuation.setAttribute("results",results);    continuation.resume();  }

Once a continuation is resumed, the request is redispatched to the servlet container, almost as if the request had been received again. However during the redispatch, the continuation.isInitial() method returns false and any attributes set by the asynchronous handler are available.

Continuation resume is analogous to Servlet 3.0 AsyncContext.dispatch().

Completing Request

As an alternative to completing a request, an asynchronous handler may write the response itself. After writing the response, the handler must indicate the request handling is complete by calling the complete

  void myAsyncCallback(Object results)  {    writeResults(continuation.getServletResponse(),results);    continuation.complete();  }

After complete is called, the container schedules the response to be committed and flushed.

Continuation resume is analogous to Servlet 3.0 AsyncContext.complete().

Continuation Listeners

An application may monitor the status of a continuation by using a ContinuationListener

  void doGet(HttpServletRequest request, HttpServletResponse response)  {    ...

    Continuation continuation = ContinuationSupport.getContinuation(request);    continuation.addContinuationListener(new ContinuationListener()    {      public void onTimeout(Continuation continuation) { ... }      public void onComplete(Continuation continuation) { ... }    });

    continuation.suspend();    ...  }

Continuation listeners are analogous to Servlet 3.0 AsyncListeners.


Continuation Patterns

Suspend Resume Pattern

The suspend/resume style is used when a servlet and/or filter is used to generate the response after a asynchronous wait that is terminated by an asynchronous handler. Typically a request attribute is used to pass results and to indicate if the request has already been suspended. 

  void doGet(HttpServletRequest request, HttpServletResponse response)  {     // if we need to get asynchronous results     Object results = request.getAttribute("results);     if (results==null)     {       final Continuation continuation = ContinuationSupport.getContinuation(request);

       // if this is not a timeout       if (continuation.isExpired())       {         sendMyTimeoutResponse(response);         return;       }

       // suspend the request       continuation.suspend(); // always suspend before registration

       // register with async service.  The code here will depend on the       // the service used (see Jetty HttpClient for example)       myAsyncHandler.register(new MyHandler()       {          public void onMyEvent(Object result)          {            continuation.setAttribute("results",results);            continuation.resume();          }       });       return; // or continuation.undispatch();     }

     // Send the results     sendMyResultResponse(response,results);   }

This style is very good when the response needs the facilities of the servlet container (eg it uses a web framework) or if the one event may resume many requests so the containers thread pool can be used to handle each of them.

Suspend Continue Pattern

The suspend/complete style is used when an asynchronous handler is used to generate the response: 

  void doGet(HttpServletRequest request, HttpServletResponse response)  {     final Continuation continuation = ContinuationSupport.getContinuation(request);

     // if this is not a timeout     if (continuation.isExpired())     {       sendMyTimeoutResponse(request,response);       return;     }

     // suspend the request     continuation.suspend(response); // response may be wrapped.

     // register with async service.  The code here will depend on the     // the service used (see Jetty HttpClient for example)     myAsyncHandler.register(new MyHandler()     {       public void onMyEvent(Object result)       {         sendMyResultResponse(continuation.getServletResponse(),results);         continuation.complete();       }     });   }

This style is very good when the response does not needs the facilities of the servlet container (eg it does not use a web framework) and if an event will resume only one continuation. If many responses are to be sent (eg a chat room), then writing one response may block and cause a DOS on the other responses.


Continuation Examples

Chat Servlet

The ChatServlet example shows how the suspend/resume style can be used to directly code a chat room. The same principles are applied to frameworks like which provide an richer environment for such applications, based on Continuations.

Quality of Service Filter

The QoSFilter(javadoc), uses suspend/resume style to limit the number of requests simultaneously within the filter. This can be used to protect a JDBC connection pool or other limited resource from too many simultaneous requests.

If too many requests are received, the extra requests wait for a short time on a semaphore, before being suspended. As requests within the filter return, they use a priority queue to resume the suspended requests. This allows your authenticated or priority users to get a better share of your servers resources when the machine is under load.

Denial of Service Filter

The DosFilter(javadoc) is similar to the QoSFilter, but protects a web application from a denial of service attack (as best you can from within a web application). If too many requests are detected coming from one source, then those requests are suspended and a warning generated. This works on the assumption that the attacker may be written in simple blocking style, so by suspending you are hopefully consuming their resources. True protection from DOS can only be achieved by network devices (or eugenics :).

Proxy Servlet

The ProxyServlet uses the suspend/complete style and the jetty asynchronous client to implement a scalable Proxy server (or transparent proxy).

Gzip Filter

The jetty GzipFilter is a filter that implements dynamic compression by wrapping the response objects. This filter has been enhanced to understand continuations, so that if a request is suspended in suspend/complete style and the wrapped response is passed to the asynchronous handler, then a ContinuationListener is used to finish the wrapped response. This allows the GzipFilter to work with the asynchronous ProxyServlet and to compress the proxied responses.


Where do you get it?

You can read about it, or download it with jetty or include it in your maven project like this pom.xml.


Continuations to Continue

One thought on “Continuations to Continue

Comments are closed.