Less is More? Evolving the Servlet API!

With the release of the Servlet API 5.0 as part of Eclipse Jakarta EE 9.0 the standardization process has completed its move from the now-defunct Java Community Process (JCP) to being fully open source at the Eclipse Foundation, including the new Eclipse EE Specification Process (JESP) and the transition of the APIs from the javax.* to the jakarta.* namespace.  The move represents a huge amount of work from many parties, but ultimately it was all meta work, in that Servlet 5.0 API is identical to the 4.0 API in all regards but name, licenses, and process, i.e. nothing functional has changed.

But now with the transition behind us, the Servlet API project is now free to develop the standard into a 5.1 or 6.0 release.  So in this blog, I will put forward my ideas for how we should evolve the Servlet specification, specifically that I think that before we add new features to the API, it is time to remove some.

Backward Compatibility

Version 1.0  was created in 1997 and it is amazing that over 2 decades later, a Servlet written against that version should still run in the very latest EE container.  So why with such a great backward compatible record should we even contemplate introducing breaking changes to future Servlet API specification?  Let’s consider some of the reasons that a developer might choose to use EE Servlets over other available technologies:

Performance

Not all web applications need high performance and when they do, it is seldom the Servlet container itself that is the bottleneck.   Yet pure performance remains a key selection criteria for containers as developers either wish to have the future possibility of high request rates or need every spare cycle available to help their application meet an acceptable quality of service. Also there is the environmental impact of the carbon foot print of unnecessary cycles wasted in the trillion upon trillions of HTTP requests executed.   Thus application containers always compete on performance, but unfortunately many of the features added over the years have had detrimental affects to over-all performance as they often break the “No Taxation without Representation” principle: that there should not be a cost for all requests for a feature only used by <1%.

Features

Developers seek to have the current best practice features available in their container.   This may be as simple as changing from byte[] to ByteBuffers or Collections, or it may be more fundamental integration of things such as dependency injection, coding by convention, asynchronous, reactive, etc.  The specification has done a reasonable job supporting such features over the years, but mistakes have been made and some features now clash, causing ambiguity and complexity. Ultimately feature integration can be an N² problem, so reducing or simplifying existing features can greatly reduce the complexity of introducing new features.

Portability

The availability of multiple implementations of the Servlet specification is a key selling point.  However the very same issues of poor integration of many features has resulted in too many dark corners of the specification where the expected behavior of a container is simply not defined, so portability is by no means guaranteed.   Too often we find ourselves needing to be bug-for-bug compatible with other implementations rather than following the actual specification.

Familiarity

Any radical departure from the core Servlet API will force developers away from what  they know and to evaluate alternatives.  But there are many non core features in the API and this blog will make the case that there are some features which can can be removed and/or simplified without hardly being noticed by the bulk of applications.  My aim with this blog is that your typical Servlet developer will think: “why is he making such a big fuss about something I didn’t know was there”, whilst your typical Servlet container implementer will think “Exactly! that feature is such a PITA!!!”.

If the Servlet API is to continue to be relevant, then it needs to be able to compete with start-of-the-art HTTP servers that do not support decades of EE legacy.  Legacy can be both a strength and a weakness, and I believe now is the time to focus on the former.  The namespace break from java.* to jakarta.* has already introduced a discontinuity in backward compatibility.   Keeping 5.0 identical in all but name to 4.0 was the right thing to do to support automatic porting of applications.  However, it has also given developers a reason to consider alternatives, so now is the time to act to ensure that Servlet 6.0 a good basis for the future of EE Servlets.

Getting Cross about Cross-Context Dispatch

Let’s just all agree upfront, without going into the details, that cross-context dispatch is a bad thing. For the purposes of the rest of this blog, I’m ignoring the many issues of cross-context dispatch.  I’ll just say that every issue I will discuss below becomes even more complex when cross-context dispatch is considered, as it introduces: additional class loaders; different session values in the same session ID space; different authentication realms; authorization bypass. Don’t even get me started on the needless mind-bending complexities of a context that forwards to another then forwards back to the original…

Modern web applications are now often broken up into many microservices, so the concept of one webapp invoking another is not in itself bad, but the idea of those services being co-located in the same container instance is not very general nor flexible assumption. By all means, the Servlet API should support a mechanism to forward or include other resources, but ideally, this should be done in a way that works equally for co-resident, co-located, and remote resources.

So let’s just assume cross-context dispatch is already dead.

Exclude Include

The concept of including another resource in a response should be straight forward, but the specification of RequestDispatcher.include(...) is just bizarre!

@WebServlet(urlPatterns = {"/servletA/*"})
public static class ServletA extends HttpServlet
{
    @Override protected void doGet(HttpServletRequest request,
                                   HttpServletResponse response) throws IOException
    {
        request.getRequestDispatcher("/servletB/infoB").include(request, response);
    }
}

The ServletA above includes ServletB in its response.  However, whilst within ServletB any calls to getServletPath() or getPathInfo(),will still return the original values used to call ServletA, rather than the “/servletB” or “/infoB”  values for the target Servlet (as is done for a call to  forward(...)).  Instead the container must set an ever-growing list of Request attributes to describe the target of the include and any non trivial Servlet that acts on the actual URI path must do something like:

public boolean doGet(HttpServletRequest request, HttpServletResponse response)
    throws ServletException, IOException
{
    String servletPath;
    String pathInfo;
    if (request.getAttribute(RequestDispatcher.INCLUDE_REQUEST_URI) != null)
    {
        servletPath = (String)
            request.getAttribute(RequestDispatcher.INCLUDE_SERVLET_PATH);
        pathInfo = (String)
            request.getAttribute(RequestDispatcher.INCLUDE_PATH_INFO);
    }
    else
    {
        servletPath = request.getServletPath();
        pathInfo = request.getPathInfo();
    }
    String pathInContext = URIUtil.addPaths(servletPath, pathInfo);
    // ...
}

Most Servlets do not do this, so they are unable to be correctly be the target of an include.  For the Servlets that do correctly check, they are more often than not wasting CPU cycles needlessly for the vast majority of requests that are not included.

Meanwhile,  the container itself must set (and then reset) at least 5 attributes, just in case the target resource might lookup one of them. Furthermore, the container must disable most of the APIs on the response object during an include, to prevent the included resource from setting the headers. So the included Servlet must be trusted to know that it is being included in order to serve the correct resource, but is then not trusted to not call APIs that are inconsistent with that knowledge. Servlets should not need to know the details of how they were invoked in order to generate a response. They should just use the paths and parameters of the request passed to them to generate a response, regardless of how that response will be used.

Ultimately, there is no need for an include API given that the specification already has a reasonable forward mechanism that supports wrapping. The ability to include one resource in the response of another can be provided with a basic wrapper around the response:

@WebServlet(urlPatterns = {"/servletA/*"})
public static class ServletA extends HttpServlet
{
    @Override
    protected void doGet(HttpServletRequest request,
                         HttpServletResponse response) throws IOException
    {
        request.getRequestDispatcher("/servletB/infoB")
            .forward(request, new IncludeResponseWrapper(response));
    }
}

Such a response wrapper could also do useful things like ensuring the included content-type is correct and better dealing with error conditions rather than ignoring an attempt to send a 500 status. To assist with porting, the include can be deprecated it’s implementation replaced with a request wrapper that reinstates the deprecated request attributes:

@Deprecated
default void include(ServletRequest request, ServletResponse response)
    throws ServletException, IOException
{
    forward(new Servlet5IncludeAttributesRequestWrapper(request),
            new IncludeResponseWrapper(response));
}

Dispatch the DispatcherType

The inclusion of the method Request.getDispatcherType()in the Servlet API is almost an admission of defeat that the specification got it wrong in so many ways that required a Servlet to know how and/or why it is being invoked in order to function correctly. Why must a Servlet know its DispatcherType? Probably so it knows it has to check the attributes for the corresponding values? But what if an error page is generated asynchronously by including a resource that forwards to another? In such a pathological case, the request will contain attributes for ERROR, ASYNC, and FORWARD, yet the type will just be FORWARD.

The concept of DispatcherType should be deprecated and it should always return REQUEST.  Backward compatibility can be supported by optionally applying a wrapper that determines the deprecated DispatcherType only if the method is called.

Unravelling Wrappers

A key feature that really needs to be revised is 6.2.2 Wrapping Requests and Responses, introduced in Servlet 2.3. The core concept of wrappers is sound, but the requirement of Wrapper Object Identity (see Object Identity Crisis below) has significant impacts. But first let’s look at a simple example of a request wrapper:

public static class ForcedUserRequest extends HttpServletRequestWrapper
{
    private final Principal forcedUser;
    public ForcedUserRequest(HttpServletRequest request, Principal forcedUser)
    {
        super(request);
        this.forcedUser = forcedUser;
    }
    @Override
    public Principal getUserPrincipal()
    {
        return forcedUser;
    }
    @Override
    public boolean isUserInRole(String role)
    {
        return forcedUser.getName().equals(role);
    }
}

This request wrapper overrides the existing getUserPrincipal() and isUserInRole(String)methods to forced user identity.  This wrapper can be applied in a filter or in a Servlet as follows:

@WebServlet(urlPatterns = {"/servletA/*"})
public static class ServletA extends HttpServlet
{
    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException
    {
        request.getServletContext()
            .getRequestDispatcher("/servletB" + req.getPathInfo())
            .forward(new ForcedUserRequest(req, new UserPrincipal("admin")),
                     response);
    }
}

Such wrapping is an established pattern in many APIs and is mostly without significant problems. For Servlets there are some issues: it should be better documented if  the wrapped user identity is propagated if ServletB makes any EE calls (I think no?);  some APIs have become too complex to sensibly wrap (e.g HttpInputStream with non-blocking IO). But even with these issues, there are good safe usages for this wrapping to override existing methods.

Object Identity Crisis!

The Servlet specification allows for wrappers to do more than just override existing methods! In 6.2.2, the specification says that:

“… the developer not only has the ability to override existing methods on the request and response objects, but to provide new API… “

So the example above could introduce new API to access the original user principal:

public static class ForcedUserRequest extends HttpServletRequestWrapper
{
    // ... getUserPrincipal & isUserInRole as above
    public Principal getOriginalUserPrincipal()
    {
        return super.getUserPrincipal();
    }
    public boolean isOriginalUserInRole(String role)
    {
        return super.isUserInRole(role);
    }
}

In order for targets to be able to use these new APIs then they must be able to downcast the passed request/response to the known wrapper type:

@WebServlet(urlPatterns = {"/servletB/*"})
public static class ServletB extends HttpServlet
{
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp)
        throws ServletException, IOException
    {
        MyWrappedRequest myr = (MyWrappedRequest)req;
        resp.getWriter().printf("user=%s orig=%s wasAdmin=%b%n",
            req.getUserPrincipal(),
            myr.getOriginalUserPrincipal(),
            myr.isOriginalUserInRole("admin"));
    }
}

This downcast will only work if the wrapped object is passed through the container without any further wrapping, thus the specification requires “wrapper object identity”:

This “wrapper object identity” requirement means that the container is unable to itself wrap requests and responses as they are passed to filters and servlets. This restriction has, directly and indirectly, a huge impact on the complexity, efficiency, and correctness of Servlet container implementations, all for very dubious and redundant benefits:

Bad Software Components

In the example of ServletB above, it is a very bad software component as it cannot be invoked simply by respecting the signature of its methods. The caller must have a priori knowledge that the passed request will be downcast and any other caller will be met with a ClassCastException. This defeats the whole point of an API specification like Servlets, which is to define good software components that can be variously assembled according to their API contracts.

No Multiple Concerns

It is not possible for multiple concerns to wrap request/responses. If another filter applies its own wrappers, then the downcast will fail. The requirement for “wrapper object identity” requires the application developer to have total control over all aspects of the application, which can be difficult with discovered web fragments and ServletContainerInitializers.

Mutable Requests

By far the biggest impact of “wrapper object identity” is that it forces requests to be mutable! Since the container is not allowed to do its own wrapping within RequestDispatcher.forward(...) then the container must make the original request object mutable so that it changes the value returned from getServletPath() to reflect the target of the dispatch.  It is this impact that has significant impacts on complexity, efficiency, and correctness:

• Mutating the underlying request makes the example implementation of isOriginalUserInRole(String) incorrect because it calls super.isUserInRole(String) whose result can be mutated if the target Servlet has a run-as configuration.  Thus this method will inadvertently return the target rather than the original role.
• There is the occasional need for a target Servlet to know details of the original request (often for debugging), but the original request can mutate so it cannot be used. Instead, an ever-growing list of Request attributes that must be set and then cleared on the original request attributes, just in case of the small chance that the target will need one of them.  A trivial forward of a request can thus require at least 12 Map operations just to make available the original state, even though it is very seldom required. Also, some aspects of the event history of a request are not recoverable from the attributes: the isUserInRolemethod; the original target of an include that does another include.
• Mutable requests cannot be safely passed to asynchronous processes, because there will be a race between the other thread call to a request method and any mutations required as the request propagates through the Servlet container (see the “Off to the Races” example below).  As a result, asynchronous applications SHOULD copy all the values from the request that they MIGHT later need…. or more often than not they don’t, and many work by good luck, but may fail if timing on the server changes.
• Using immutable objects can have significant benefits by allowing the JVM optimizer and GC to have knowledge that field values will not change.   By forcing the containers to use mutable request implementations, the specification removes the opportunity to access these benefits. Worse still, the complexity of the resulting request object makes them rather heavy weight and thus they are often recycled in object pools to save on the cost of creation. Such pooled objects used in asynchronous environments can be a recipe for disaster as asynchronous processes may reference a request object after it has been recycled into another request.

Unnecessary

New APIs can be passed on objects set as request attribute values that will pass through multiple other wrappers, coexist with other new APIs in attributes and do not require the core request methods to have mutable returns.

The “wrapper object identity” requirement has little utility yet significant impacts on the correctness and performance of implementations. It significantly impairs the implementation of the container for a feature that can be rendered unusable by a wrapper applied by another filter.  It should be removed from Servlet 6.0 and requests passed in by the container should be immutable.

Asynchronous Life Cycle

A bit of history

Jetty continuations were a non-standard feature introduced in Jetty-6 (around 2005) to support thread-less waiting for asynchronous events (e.g. typically another HTTP request in a chat room). Because the Servlet API had not been designed for thread-safe access from asynchronous processes, the continuations feature did not attempt to let arbitrary threads call the Servlet API.  Instead, it has a suspend/resume model that once the asynchronous wait was over, the request was re-dispatched back into the Servlet container to generate a response, using the normal blocking Servlet API from a well-defined context.

When the continuation feature was standardized in the Servlet 3.0 specification, the Jetty suspend/resume model was supported with the APIs ServletRequest.startAsync() and AsyncContext.dispatch() methods.  However (against our strongly given advice), a second asynchronous model was also enabled, as represented by ServletRequest.startAsync() followed by AsyncContext.complete().  With the start/complete model, instead of generating a response by dispatching a container-managed thread, serialized on the request, to the Servlet container, arbitrary asynchronous threads could generate the response by directly accessing the request/response objects and then call the AsyncContext.complete() method when the response had been fully generated to end the cycle.   The result is that the entire API, designed not to be thread safe, was now exposed to concurrent calls. Unfortunately there was (and is) very little in the specification to help resolve the many races and ambiguities that resulted.

Off to the Races

The primary race introduced by start/complete is that described above caused by mutable requests that are forced by “wrapper object identity”. Consider the following asynchronous Servlet:

@WebServlet(urlPatterns = {"/async/*"}, asyncSupported = true)
@RunAs("special")
public static class AsyncServlet extends HttpServlet
{
    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException
    {
        AsyncContext async = request.startAsync();
        PrintWriter out = response.getWriter();
        async.start( () ->
        {
            response.setStatus(HttpServletResponse.SC_OK);
            out.printf("path=%s special=%b%n",
                       request.getServletPath(),
                       request.isUserInRole("special"));
            async.complete();
        });
    }
}

If invoked via a RequestDispatcher.forward(...), then the result produced by this Servlet is a race: will the thread dispatched to execute the lambda execute before or after the thread returns from the `doGet` method (and any applied filters) and the pre-forward values for the path and role are restored? Not only could the path and role be reported either for the target or caller, but the race could even split them so they are reported inconsistently.  To avoid this race, asynchronous Servlets must copy any value that they may use from the request before starting the asynchronous thread, which is needless complexity and expense. Many Servlets do not actually do this and just rely on happenstance to work correctly.

This problem is the result of  the start/complete lifecycle of asynchronous Servlets permitting/encouraging arbitrary threads to call the existing APIs that were not designed to be thread-safe.  This issue is avoided if the request object passed to doGet is immutable and if it is the target of a forward, it will always act as that target. However, there are other issues of the asynchronous lifecycle that cannot be resolved just with immutability.

Out of Time

The example below is a very typical race that exists in many applications between a timeout and asynchronous processing:

@Override
protected void doGet(HttpServletRequest request,
                     HttpServletResponse response) throws IOException
{
    AsyncContext async = request.startAsync();
    PrintWriter out = response.getWriter();
    async.addListener(new AsyncListener()
    {
        @Override
        public void onTimeout(AsyncEvent asyncEvent) throws IOException
        {
            response.setStatus(HttpServletResponse.SC_BAD_GATEWAY);
            out.printf("Request %s timed out!%n", request.getServletPath());
            out.printf("timeout=%dms%n ", async.getTimeout());
            async.complete();
        }
    });
    CompletableFuture<String> logic = someBusinessLogic();
    logic.thenAccept(answer ->
    {
        response.setStatus(HttpServletResponse.SC_OK);
        out.printf("Request %s handled OK%n", request.getServletPath());
        out.printf("The answer is %s%n", answer);
        async.complete();
    });
}

Because the handling of the result of the business logic may be executed by a non-container-managed thread, it may run concurrently with the timeout callback. The result can be an incorrect status code and/or the response content being interleaved. Even if both lambdas grab a lock to mutually exclude each other, the results are sub-optimal, as both will eventually execute and one will ultimately throw an IllegalStateException, causing extra processing and a spurious exception that may confuse developers/deployers.

The current specification of the asynchronous life cycle is the worst of both worlds for the implementation of the container. On one hand, they must implement the complexity of request-serialized events, so that for a given request there can only be a single container-managed thread in service(...), doFilter(...), onWritePossible(), onDataAvailable(), onAllDataRead()and onError(), yet on the other hand an arbitrary application thread is permitted to concurrently call the API, thus requiring additional thread-safety complexity. All the benefits of request-serialized threads are lost by the ability of arbitrary other threads to call the Servlet APIs.

Request Serialized Threads

The fix is twofold: firstly make more Servlet APIs immutable (as discussed above) so they are safe to call from other threads;  secondly and most importantly, any API that does mutate state should only be able to be called from request-serialized threads!   The latter might seem a bit draconian as it will make the lambda passed to thenAccept in the example above throw an IllegalStateException when it tries to setStatus(int) or call complete(), however, there are huge benefits in complexity and correctness and only some simple changes are needed to rework existing code.

Any code running within a call to service(...), doFilter(...), onWritePossible(), onDataAvailable(), onAllDataRead()and onError() will already be in a request-serialized thread, and thus will require no change. It is only code executed by threads managed by other asynchronous components (e.g. the lambda passed to thenAccept() above) that need to be scoped. There is already the method AsyncContext.start(Runnable) that allows a non-container thread to access the context (i.e. classloader) associated with the request. An additional similar method AsyncContext.dispatch(Runnable) can be provided that not only scopes the execution but mutually excludes it and serializes it against any call to the methods listed above and any other dispatched Runnable. The Runnables passed may be executed within the scope of the dispatch call if possible (making the thread momentarily managed by the container and request serialized) or scheduled for later execution.  Thus calls to mutate the state of a request can only be made from threads that are serialized.

To make accessing the dispatch(Runnable) method more convenient, an executor can be provided with AsyncContext.getExecutor() which provides the same semantic.  The example above can now be simply updated:

@Override
protected void doGet(HttpServletRequest request,
                     HttpServletResponse response) throws IOException
{
    AsyncContext async = request.startAsync();
    PrintWriter out = response.getWriter();
    async.addListener(new AsyncListener()
    {
        @Override
        public void onTimeout(AsyncEvent asyncEvent) throws IOException
        {
            response.setStatus(HttpServletResponse.SC_BAD_GATEWAY);
            out.printf("Request timed out after %dms%n ", async.getTimeout());
            async.complete();
        }
    });
    CompletableFuture<String> logic = someBusinessLogic();
    logic.thenAcceptAsync(answer ->
    {
        response.setStatus(HttpServletResponse.SC_OK);
        out.printf("The answer is %s%n", answer);
        async.complete();
    }, async.getExecutor());
}

Because the AsyncContext.getExecutor() is used to invoke the business logic consumer, then the timeout and business logic response methods are mutually excluded. Moreover, because they are serialized by the container, the request state can be checked between each, so that if the business logic has completed the request, then the timeout callback will never be called, even if the underlying timer expires while the response is being generated. Conversely, if the business logic result is generated after the timeout, then the lambda to generate the response will never be called.  Because both of the tasks in this example call complete, then only one of them will ever be executed.

And Now You’re Complete

In the example below, a non-blocking read listener has been set on the request input stream, thus a callback to onDataAvailable() has been scheduled to occur at some time in the future.  In parallel, an asynchronous business process has been initiated that will complete the response:

@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException
{
    AsyncContext async = request.startAsync();
    request.getInputStream().setReadListener(new MyReadListener());
    CompletableFuture<String> logicB = someBusinessLogicB();
    PrintWriter out = response.getWriter();
    logicB.thenAcceptAsync(b ->
    {
        out.printf("The answer for %s is B=%s%n", request.getServletPath(), b);
        async.complete();
    }, async.getExecutor());
}

The example uses the proposed APIs above so that any call to complete is mutually excluded and serialized with the call to doGet and onDataAvailable(...). Even so, the current spec is unclear if the complete should prevent any future callback to onDataAvailable(...) or if the effect of complete() should be delayed until the callback is made (or times out). Given that the actions can now be request-serialized, the spec should require that once a request serialized thread that has called complete returns, then the request cycle is complete and there will be no other callbacks other than onComplete(...), thus cancelling any non-blocking IO callbacks.

To Be Removed

Before extending the Servlet specification, I believe the following existing features should be removed or deprecated:

• Cross context dispatch deprecated and existing methods return null.  Once a request is matched to a context, then it will only ever be associated with that context and the getServletContext() method will return the same value no matter what state the request is in.
• The “Wrapper Object Identity” requirement is removed and the request object will be required to be immutable in regards to the methods affected by a dispatch and may be referenced by asynchronous threads.
• The RequestDispatcher.include(...) is deprecated and replaced with utility response wrappers.  The existing API can be deprecated and its implementation changed to use a request wrapper to simulate the existing attributes.
• The special attributes for FORWARD, INCLUDE, ASYNC are removed from the normal dispatches.  Utility wrappers will be provided that can simulate these attributes if needed for backward compatibility.
• The getDispatcherType() method is deprecated and returns REQUEST, unless a utility wrapper is used to replicate the old behavior.
• Servlet API methods that mutate state will only be callable from request-serialized container-managed threads and will otherwise throw IllegalStateException. New AsyncContext.dispatch(Runnable) and AsyncContext.getExecutor() methods will provide access to request-serialization for arbitrary threads/lambdas/Runnables

With these changes, I believe that many web applications will not be affected and most of the remainder could be updated with minimal effort. Furthermore, utility filters can be provided that apply wrappers to obtain almost all deprecated behaviors other than Wrapper Object Identity. In return for the slight break in backward compatibility, the benefit of these changes would be significant simplifications and efficiencies of the Servlet container implementations. I believe that only with such simplifications can we have a stable base on which to build new features into the Servlet specification. If we can’t take out the cruft now, then when?

The plan is to follow this blog up with another proposing some more rationalisation of features (I’m looking at you sessions and authentication), before another blog proposing some new features an future directions.