Servlet Performance Report – The Jetty Response.

Christopher Merrill of Web Performance Inc has release a report of Servlet Performance, which included Jetty.

Firstly it is good to see somebody again tackling the difficult task of benchmarking servlet containers. It is a difficult task and will always draw criticism and the results will always be disputed.

The basic methodology of the test is to gradually increase load on the servers tested and chart their performance over time. The results in a nutshell were that all servers did pretty much the same up to a point. One by one, the servers reached a limit and their performance degraded.

Unfortunately, Jetty was one of the first servers to break ranks and display degraded performance. This however, is not as disturbing as might seem at first. The test used the default configuration for all the servers, which greatly differs. For example, Jetty by default is configured with 50 threads maximum, while tomcat has 150 by default. This is a significant flaw in the study and makes it all but useless for comparing the maximum performances of the different containers. In fact it is amazing that Jetty kept pace with the others for so long, considering it had limited resources.

The bad news for Jetty, is that even if you ignore the other containers in the study, the shape of the curve is not nice. When a server hits a configured limit it should gracefully degrade it performance in the face of more load. Unfortunately when presented with the load in this study, Jetty’s degradation is a bit less than graceful. We have not seen this ugly curve in our own testing and reports from the field are mostly about grace under fire. So hopefully this is an artifact from the artificial load used in the test. However, this test does show that there is at least one load profile that causes Jetty grief once resources are exhausted. So we have some work to do.

In summary, the study is interesting in that it at least shows that most servlet containers are approaching terminal velocity for less than pathological loads. But if you want to maximise throughput for your web application, then don’t use the default configuration. Instead read the Jetty Optimisation Guide.