Comet Performance

In response to the recent discussion of push v pull Ajax performance, I decided to do some performance testing of the Jetty implementation of bayeux for the cometd project. This was also a great way to test the  asynchronous http client  that is now included with Jetty.
The test scenarios was simple: 1000,2000,5000 & 10000 users simultaneously connect to the cometd server and subscribe to a chat room. We publish a fixed number of messages to the rooms and vary the number of users per room while measuring message throughput and latency. The software was written using dojo-0.9 for the cometd client and Jetty 6.1.5 for the server. A long polling transport was used, which means that each client has an outstanding request parked in the server waiting for an event so that a response may be sent to the client.
The server  machine  was a Intel 1.83GHz core duo minimac with 1GB ram running OSX. The client machine was an Intel 2GHz centrino duo thinkpad  with 512M ram running ubuntu linux.  Suns Java 1.5.11  was used on both.  These are pretty small machines to be testing 10000 simultaneous users, but they managed well enough to draw some conclusions. The crucial configuration for the server was:

<Set name="ThreadPool">  <New class="org.mortbay.thread.BoundedThreadPool">    <Set name="minThreads">10</Set>    <Set name="maxThreads">250</Set>    <Set name="lowThreads">25</Set>  </New></Set><Call name="addConnector">  <Arg>    <New class="org.mortbay.jetty.nio.SelectChannelConnector">      <Set name="port">8080</Set>      <Set name="maxIdleTime">240000</Set>      <Set name="Acceptors">2</Set>      <Set name="acceptQueueSize">1000</Set>      <Set name="lowResourcesConnections">11000</Set>      <Set name="lowResourcesMaxIdleTime">1000</Set>    </New>  </Arg></Call>

The jetty asynchronous http client is based on the same NIO technology as the server. But instead or parsing requests and generating responses, it generates requests and parses responses.   Thus the latency due to NIO scheduling will measured twice in this test, once for the server and once for the client. In reality the 10000 clients would be running on 10000 machines each using blocking IO for the 1 or 2 TCP/IP connections they maintain to the server.  So the actual results for Bayeux can be expected to be better (perhaps significantly) than the numbers here:

full size
The results show large numbers of simultaneous users can indeed be handled with low latency. It must be remember that each connected user has 1 or 2 TCP/IP connections and will have at least 1 outstanding HTTP request.  With a non NIO or non Continuation based  server, this would require around 11,000 threads to handle 10,000 simultaneous users.  Jetty handles this number of connections with only 250 threads.
Below 1000 message deliveries per second, the average latency is small and almost constant for 1000, 2000 and 5000 users, but for 10,000 users the latency starts creeping up to a few hundred milliseconds, which still highly interactive and sufficient for a chat, collaborative editing and many games.
Above 1000 messages per second, the latency starts to suffer and at 3000 messages per second, reaches 1.5 seconds for 1000 users and 7 seconds for 5000 users. For less than 10,000 users, this degradation can be described as graceful and is a reflection of the bayeux protocols ability to batch more messages when under duress in a classic latency vs throughput tradeoff.
Above all, it must be recognized that all results in this test have superior latency to the pull solutions referred to in the link above. To achieve 1 second average latency, pull solutions would need to poll every 2 seconds, generating 5000 requests per second for a server with 10000 idle users!
The conclusion from these tests is that Bayeux + Jetty + Continuations does indeed provide scalable low latency communications for Ajax applications.