This Jetty Case Study takes a look at an intriguing
software infrastructure product called 1060 NetKernelTM.

Announcing the recent release of version 2 of the product, 1060 Research describes NetKernel as “an advanced service oriented microkernel”. Complex applications are produced by creating simple services and then aggregating or pipelining them together. NetKernel services interact with the external environment via pluggable transports, such as SMTP, SOAP and – importantly – HTTP.

NetKernel can be used standalone, for example in place of a J2EE application server, or alternatively embedded within a J2EE app server, or in fact embedded in any Java application.

We spoke via email with Peter Rodgers – founder and CEO of 1060 Research and one of the product’s architects – to find out more about NetKernel, and how Jetty contributes to it:

  1. You describe NetKernel as a “REST” based microkernel. Firstly, can
    you explain what is “REST”?

    REST is an acronym for REpresentation State Transfer. It was coined by
    Roy Fielding in his PhD dissertation which retrospectively presents a
    formalism of the Web architecture.

    What it means in practise is that resources are addressed by URI and
    instead of a resource being generated by hidden interactions it is generated
    by the transfer of state to a service – often though not exclusively
    expressed in the URI.

    Whilst this seems like a new pattern when applied in the context of the
    Web, it is really an old pattern that is basically one of the
    foundational principles of Unix. In Unix, software applications are
    modular services which may be configured by switches to process a
    resource. Higher order applications are created by orchestrating
    pipelines of lower-level software services – today this design pattern
    is frequently called “loose coupling”.

    The NetKernel microkernel allows software applications to be flexibly
    composed like a Unix system, but employs URI addressing and a URI address
    space abstraction to present a uniform application context.

  2. Now can you describe NetKernel for us?

    Fundamentally NetKernel is a virtual operating system abstraction which
    you could describe as “Unix meets the Web”. Although, more practically,
    the microkernel provides the basis for a general purpose Application
    Server which in-particular supports rich XML processes and services.

    Software applications on NetKernel consist of fine-grained URI
    addressable services which may be composed into higher-order services or
    abstracted behind higher-level URI interfaces. Basically the Unix model
    of loose coupling in a URI address space.

    NetKernel’s URI address space is an *internal* abstraction – composite
    services may be exposed to the world by mapping their URI address onto a

  3. You mention request shaping and request scheduling. Does NetKernel
    also support load balancing of requests or failover of services?

    We perform request shaping between transports and the microkernel
    scheduler – this ensures that we get close to ideal throughput for any
    given load[1].

    We can do this since the microkernel has a re-entrant asynchronous
    scheduler – in effect every transport-generated-request initiates the
    execution of an asynchronous application on NetKernel.

    This is different to a free-running multi-threaded model typical in
    unmanaged application servers. Basically once there are more than one
    or two threads per native CPU adding more threads does not mean more
    processing, it just increases the native OS context switching overhead.
    A well managed system will linearly increase it’s throughput with load
    until all CPUs are fully occupied and then operate with constant
    throughput as load increases further – the NetKernel throttle ensures
    that the system operates in this regime.

    The NetKernel scheduler actually allows concurrent execution of
    applications on a single Java thread if necessary.

    In terms of system-wide load-balancing – so far we’ve concentrated on
    the architectural fundamentals. NetKernel can be embedded within J2EE
    application servers. So, many Enterprise deployments steal the
    load-balancing infrastructure of their existing application servers!

    We will very soon have native JMS support which will allow NetKernel to
    be used as highly-scaled general purpose message-oriented middleware.

  4. What about hot deployment of services? Graceful service upgrades?

    As a microkernel architecture NetKernel is completely modular – it
    supports hot installation and updates of all services.

    It also supports version enforcement on modules which means that you can
    concurrently execute multiple generations of the same application in
    isolation on the same system. Good for development and it allows legacy
    systems to keep running irrespective of future additions.

  5. Are requests asynchronous, synchronous or either?

    NetKernel is an asynchronous architecture. However the microkernel will
    attempt to optimally reuse threads synchronously such that context
    switching is minimized.

    However, asynchronous applications are generally not easy to build and
    maintain, so we provide the NetKernel Foundation API which is written so
    that most applications can be developed using synchronous patterns – even
    though under the hood they’ll execute asynchronously.

    At the application level, just like on Unix, applications can fork
    asynchronous sub-processes – they may also explicitly join a forked
    sub-process in-order to retrieve a result or handle exceptions.

  6. Requests are received on transports. What type of transports does
    NetKernel support?

    A transport on NetKernel is a little like a device-driver. It issues
    NetKernel URI requests based upon some application or application
    protocol specific event. It is pretty easy to add new transports.

    Out of the box we have HTTP, SMTP, POP, IMAP, Telnet, In-tray (directory
    based), SOAP 1.1/1.2

    The NetKernel administration applications and services run as
    Web-applications over HTTP on port 1060 – not very subtle subliminal

  7. Which brings us to Jetty – what role/s does Jetty play within NetKernel?

    We use Jetty as the backbone of our HTTP transport module. HTTP is a
    very important application protocol since Web-applications are the
    dominant class of Enterprise application today.

    It’s interesting, we didn’t design NetKernel as a Web-application server
    – we came from wanting a general resource processing model – but it
    turns out that it is very simple to expose NetKernel services as
    Web-applications over the Jetty HTTP transport.

    This is quite different from a Servlet which provides a hard-boundary
    between Web and non-Web. With a Web-application on NetKernel the
    Web-boundary becomes a continuum – you can also do some cool things like
    aspect-oriented layering over the web-address space, but that’s probably
    an advanced topic!

    Jetty is an outstanding HTTP server.

  8. Why did you select Jetty?

    We needed a clean, simple HTTP server. To boot strap our system we
    started off writing our own – it worked, but you wouldn’t have wanted to
    rely on it! So we looked around and discovered Jetty – it had all the
    features we were looking for…

    • A pure HTTP server – we had no need for a Servlet engine.
    • Small footprint.
    • Great compliance with HTTP 1.0 and 1.1 standards.
    • Highly scalable.
    • Easy XML-based configurability.
    • Widespread adoption.
    • Extensible with a custom Request Handler chain.
    • An open-source implementation that overlapped with our business model.

    Jetty has proved to be an incredibly dependable infrastructure, to the
    point where we now just don’t really think about it!

  9. Have you written any custom extensions to Jetty for NetKernel, and if
    so, what were your experiences?

    The NetKernel HTTP transport implements a NetKernel ITransport interface
    which is managed by the microkernel. When the transport is started it
    fires up the Jetty container and uses an XML configuration document to
    declaratively configure Jetty – this includes things like request and
    thread limits and SSL.

    Very early on we developed a custom Request Handler which is hooked into
    the Jetty request handler stack. This handler hooks all HTTP requests
    and wraps them as URI requests against the NetKernel URI address space.
    So, with this handler, Jetty is a thin-bridge from the HTTP application
    protocol to the NetKernel virtual address space.

    That’s sort of where Jetty ends and NetKernel begins – though not quite.
    We’ve generalized the idea of the Servlet from being a static interface
    which tightly binds the HTTP protocol. Instead we offer a service
    called the HTTPBridge – this is a configurable HTTP Request
    Filter service which can be transparently layered over the
    URI address space.

    The HTTPBridge pre-processes the low-level HTTPRequest. It can be dynamically
    configured to XML’ize URI parameters or POST data, pre-process file uploads,
    process HTTP headers, process Cookies etc etc. It is a general-purpose
    service which can slice and dice the low level HTTPRequest in many
    ways – including for example performing SOAP HTTP bindings.

    The HTTPBridge re-issues the pre-processed request into the NetKernel URI
    space. Ultimately the Bridge receives a response for the
    request, for which it then performs any final HTTP specific processing –
    such as setting HTTP response codes, cookies, etc and of course
    serializing any generated resource into the HTTPResponse stream.

    So the NetKernel HTTP transport is decomposed into clean, easily
    reconfigurable layers and of course the HTTPRequest/Response objects
    come from Jetty.

  10. Are there any features you would like to see in Jetty and why? For
    example, you mention NIO in relation to the Http transport …

    Jetty is an excellent solution as it is. We have a philosophy of always
    trying to keep everything as simple, as lean and as minimal as possible.
    So we’re always looking for more from less. As mentioned above we do
    this in the kernel by always operating with an efficiently small number
    of threads. We’re also considering a NetKernel Micro-Edition for J2ME
    embedded applications.

    At the moment, actually based on your empirical advice on the Jetty
    site, we have not used NIO. We’d be very interested to understand
    better if NIO would offer any advantage in terms of HTTP server
    footprint etc. Though obviously the performance trade-off would need to
    be understood.

  11. It looks like you guys have had fun with NetKernel – I’m referring
    in particular to a home monitoring system that you’ve put together
    ( . .

    Yes, though NetKernel is a serious software infrastructure, it is
    completely general purpose. This application was put together by Tony
    Butterfield as an example of something a little more entertaining – it
    also means he can talk about how much rain we get from anywhere in the

    I reckon a good criteria for evaluating software is ‘Is this cool?’.
    I put Jetty into that category straight-away – our hope is that
    people will have the same reaction the first time they boot up NetKernel.

  12. What licensing options are there for NetKernel?

    We have a dual license business model – basically, NetKernel is on the open-source
    commons, to use it we request you to OSI license your code. If you are unable
    to or prefer the additional benefits of a commercial relationship then we
    offer flexible commercial licensing.

  13. You’ve just released NetKernel 2.0, what’s next on the horizon?

    Immediately we have a 2.0.1 update due in the next few weeks. This
    ships some trivial patches but more importantly will provide a new JMS
    transport which didn’t make the release cut for the 2.0 product.

    Our short-term plans are to keep explaining what NetKernel is! We’re
    finding that once people understand, they really like it – but when
    anything is fundamentally different it can take a while to get used to.

    Next steps for NetKernel are a general Unix-like security infrastructure.


[1] Whitepaper: NetKernel Scalability