The Jetty Overlay Deployer allows multiple WAR files to be overlayed so that a web application can be customised, configured and deployed without the need to unpack, modify and repack the WAR file. This has the benefits of:

  • The WAR file may be kept immutable, even signed, so that it is clear which version has been deployed.
  • All modifications made to customise/configure the web application are kept in a separate wars and thus are easily identifiable for review and migration to new versions.
  • A parameterised template overlay can be created that contains common customisations and configuration that apply to many instances of the web application (eg for multi-tenant deployment).
  • Because the layered deployment clearly identifies the common and instance specific components, then Jetty is able to share classloaders and static resource caches for the template, greatly reducing the memory footprint of multiple instances

This blog is a tutorial of how to configure Jetty to use the Overlay deployer, and how to deploy multiple instances of the JTrac web application.

Overview

The customisation, configuration and deployment a web application bundled as a WAR file frequently includes some or all of:

  • Editing the WEB-INF/web.xml file to set init parameters, add filters/servlets or to configure JNDI resources.
  • Editing other application specific configuration files in WEB-INF/*
  • Editing container specific configuration files in WEB-INF/* (eg jetty-web.xml or jboss-web.xml)
  • Adding/modifying static content such as images and css to style/theme the webapplication
  • Adding jars to the container classpath for Datasource and other resources
  • Modifying the container configuration to provide JNDI resources

The result is that the customisations and configurations are blended into both the container and the WAR file. If either the container or the base WAR file are upgraded to a new version, it can be a very difficult and error prone task to identify all the changes that have been made and to reapply them to a new version.

Overlays

To solve the problems highlighted above, jetty 7.4 introduces WAR overlays (which are a concept borrowed from the maven war plugin). An overlay is basically just another WAR files, whose contents are merged on top of the original WAR so that files may be added or replaced.

However,  jetty overlays also allow mixin fragments of web.xml, so the configuration can be modified without being replaced

Jtrac Overlay Example

The jtrac issue tracking webapplication is a good example of a typical web application, as it uses the usual suspects of libs: spring, hibernate, dom4j, commons-*, wicket, etc. So I’ve used it as the basis of this example.

The files for this demonstration are available in overlays-demo.tar.gz. This could be expanded on top of the jetty distribution, but for this tutorial we will expand it to /tmp and install the components step by step:

Configuring Jetty for Overlays

Overlays support is included in jetty distributions from 7.4.1-SNAPSHOT onwards, so you can download a distribution from oss.sonatype.org or maven central(once 7.4.1 is released) and unpack into a directory.    The start.ini file then needs to be edited so that it includes the overlay option and configuration file.   The resulting file should look like:

The smarts of this are in etc/jetty-deploy.xml files, which installs the OverlayedAppProvider into the DeploymentManager. Jetty can then be started normally:

Jetty will now be listening on port 8080, but with no webapp deployed.   The rest of the tutorial should be conducted in another window with the JETTY_HOME environment set to the jetty distribution directory.

Installing the WebApp

The WAR file for this demo can be downloaded and deployed using the following commands, which essentially downloads and extracts the WAR file to the $JETTY_HOME/overlays/webapps directory

When you have run these commands (or equivalent), you will see in the jetty server window a message saying that the OverlayedAppProvider has extracted and loaded the war file:

Unlike the normal webapps dir, loading a war file from the overlays/webapp dir does not deploy the webapplication.  It simply makes it available to be used as the basis for templates and overlays.

Installing a Template Overlay

A template overlay is a WAR structured directory/archive that contains just the files that have been added or modified to customize/configure the webapplication for all instances that will be deployed.

The demo template can be installed from the downloaded files with the command:

In the Jetty server window, you should see the template loaded with a message like:

The contents of the loaded template is as follows:

The name of the template directory (or it could be a war) uses the ‘=’ character in jtracTemplate=jtrac-2.1.0 to separates the name of the template from the name of the WAR file in webapps that it applies to.  If  = is a problem, then -- may also be used.

WEB-INF/classes/jtrac-init.properties – replaces the jtrac properties file with an empty file, as the properties contained within it are configured elsewhere

WEB-INF/log4j.properties – configures the logging for all instances of the template.

WEB-INF/overlay.xml – a Jetty XML formatted IoC file that is used to inject/configure the ContextHandler for each instances. In this case it just sets up the context path:

WEB-INF/template.xml – a Jetty XML formatted IoC file that is used to inject/configure the resource cache and classloader that is shared by all instances of the template. It is run only once per load of the template:

WEB-INF/web-overlay.xml – a web.xml fragment that is overlayed on top of the web.xml from the base WAR file, that can set init parameters and add/modify filters and servlets. In this it sets the application home and springs rootKey:

Note the use of parameterisation of values such as ${overlays.instance.classifier}, as this allows the configuration to be made in the template and not customised for each instance.

Without the overlayed deployer, all the configurations above would still need to have been made, but rather that being in a single clear structure they would have been either in the servers common directory, the servers webdefaults.xml (aka server.xml), or baked into the WAR file of each application instance using copied/modified files from the original. The overlayed deployer allows us to make all these changes in one structure, more over it allows some of the configuration to be parameterised to facilitate easy multi-tenant deployment.

Installing an Instance Overlay

Now that we have installed a template, we can install one or more instance overlays, which deploy the actual web applications:

As each instance is moved into place, you will see the jetty server window react and deploy that instance. Within each instance, there is the structure:

WEB-INF/overlay.xml – a Jetty XML format IoC file that injects/configures the context for the instance. In this case it sets up a virtual host for the instance:

favicon.ico – replaces the icon in the base war with one themed for the instance colour.

resources/jtrac.css – replaces the style sheet from the base war with one themed for the instance colour

The deployed instances can now be viewed by pointing your browser at http://127.0.0.1:8080, http://127.0.0.2:8080 and http://127.0.0.3:8080. The default username/password for jtrac is admin/admin.

Things to know and notice

  • Each instance is themed with images and styles sheets from the instance overlay.
  • Each instance is running with it’s own application directory (eg. /tmp/jtrac-red), that is set templates web-overlay.xml.
  • The instances are distinguished by virtual host that is set in the instance overlay.xml
  • The static content from the base war and template are shared between all instances. Specifically there is a shared ResourceCache so only a single instance of each static content is loaded into memory.
  • The classloader at the base war and template level is shared between all instances, so that only a single instance of common classes is loaded into memory. Classes with non shared statics can be configured to load in the instances classloader.
  • All overlays are hot deployed and dependencies tracked. If an XML is touched in an instance, it is redeployed. If an XML is touched in a template, then all instances using it are redeployed. If a WAR file is touched, then all templates and all instances dependant on it are redeployed.
  • New versions can easily be deployed. Eg when jtrac-2.2.0.war becomes available, it can just be dropped into overlays/webapps and then rename jtracTemplate=jtrac-2.1.0 to jtracTemplate=jtrac-2.2.0
  • There is a fuller version of this demo in overlays-demo-jndi.tar.gz, that uses JNDI (needs options=jndi,annotations and jetty-plus.xml in start.ini) and shows how extra jars can be added in the overlays.
Jetty Overlayed WebApp Deployer

9 thoughts on “Jetty Overlayed WebApp Deployer

  • May 13, 2011 at 3:54 am
    Permalink

    Any chance this makes 8.0.0-M3? Any idea when we’ll see M3?

    • May 13, 2011 at 4:00 am
      Permalink

      I’m working on 7.4.1 final, and as soon as that is done there will be a 8M3, so next week I hope at the latest.

      • May 13, 2011 at 1:45 pm
        Permalink

        As always… you’re one step ahead of me. We’re currently using Maven in a large enterprise system to overlay about 30 different webapps on top of our “shared” webapp. We’re in the very early stages of this so it’s not much of an issue today… But overlaying at runtime is a lot less complex to update everything globally then doing it in the build process.

        Thanks for the great work you guys are doing…

  • June 7, 2011 at 9:56 pm
    Permalink

    If you are using spring MVC in your webapp, typically you will need to set the webAppRootKey context attribute in each instance. The can be done in the WEB-INF/web.xml and can be parameterized with ${overlay.instance.classifier} if done in the template.

  • July 4, 2011 at 3:22 pm
    Permalink

    Hi Greg,
    I have followed your example and got that working fine. However I then tried to apply this to my application (which uses Jersey and an intercepting filter) I encountered problems.
    Whilst my application runs fine when deployed in the standard way without the overlay, when I try to use the overlay approach I invariably get the following error:

    2011-07-04 15:52:12.272:WARN::Unable to reach node goal: started
    java.lang.ClassCastException: org.eclipse.jetty.webapp.WebAppContext cannot be c
    ast to org.eclipse.jetty.server.handler.ContextHandler
    at org.eclipse.jetty.overlays.OverlayedAppProvider.createContextHandler(
    OverlayedAppProvider.java:412)
    at org.eclipse.jetty.deploy.App.getContextHandler(App.java:97)
    at org.eclipse.jetty.deploy.bindings.StandardDeployer.processBinding(Sta
    ndardDeployer.java:33)
    at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:1
    80)
    at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentM
    anager.java:482)
    at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.j
    ava:135)
    at org.eclipse.jetty.overlays.OverlayedAppProvider.redeploy(OverlayedApp
    Provider.java:1085)
    at org.eclipse.jetty.overlays.OverlayedAppProvider.updateLayers(Overlaye
    dAppProvider.java:922)
    at org.eclipse.jetty.overlays.OverlayedAppProvider$1.filesChanged(Overla
    yedAppProvider.java:283)
    at org.eclipse.jetty.util.Scanner.reportBulkChanges(Scanner.java:677)
    at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:542)
    at org.eclipse.jetty.util.Scanner.scan(Scanner.java:394)
    at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:329)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
    Cycle.java:58)
    at org.eclipse.jetty.overlays.OverlayedAppProvider.doStart(OverlayedAppP
    rovider.java:779)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
    Cycle.java:58)
    at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(Deploymen
    tManager.java:543)
    at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.
    java:218)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
    Cycle.java:58)
    at org.eclipse.jetty.util.component.AggregateLifeCycle.doStart(Aggregate
    LifeCycle.java:41)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHand
    ler.java:50)
    at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrappe
    r.java:90)
    at org.eclipse.jetty.server.Server.doStart(Server.java:258)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
    Cycle.java:58)
    at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:11
    97)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:112
    0)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.eclipse.jetty.start.Main.invokeMain(Main.java:469)
    at org.eclipse.jetty.start.Main.start(Main.java:612)
    at org.eclipse.jetty.start.Main.parseCommandLine(Main.java:265)
    at org.eclipse.jetty.start.Main.main(Main.java:79)

    I have tried with a few different versions of Jetty (7.4.2, 7.4.3 and 7.4.3-hightide) but no luck.
    Can you shed any light on the problem?

    Thank you very much.

    Andrew

    • July 27, 2011 at 11:50 pm
      Permalink

      Andrew,

      sorry for slow response.

      Can you raise this in the jetty mailing lists and we can have a look there. But is sounds like you have some jetty jars in a WEB-INF/lib somewhere, so that you have more than one copy of ContextHandler class

  • July 26, 2011 at 11:42 am
    Permalink

    With wget I see that all it’s ok (with firefox it’s a problem). Congratulation, nice article.

  • July 26, 2011 at 1:46 pm
    Permalink

    For http://127.0.0.x:8080/I have an Error 404. In console I see a strange WARN:OverlayedAppProvider:No webapp found for template: jtracTemplate=jtrac-2.1.0
    I checked and in overlays/webbapps is jtrac.war. Below are all jetty console messages. Any advice is welcome.

    2011-07-26 16:37:17.771:INFO::jetty-7.4.5.v20110725
    2011-07-26 16:37:17.800:INFO:OverlayedAppProvider:Node=nonameNode Scan= /tmp/jetty-7.4.5/overlays
    2011-07-26 16:37:17.835:INFO:OverlayedAppProvider:loaded jtracTemplate=jtrac-2.1.0@1311687437833
    2011-07-26 16:37:17.841:INFO:OverlayedAppProvider:Extract jar:file:/tmp/jetty-7.4.5/overlays/webapps/jtrac.war!/ to /tmp/jtrac_6211496015575616479.extract
    2011-07-26 16:37:18.448:INFO:OverlayedAppProvider:loaded jtrac@1311687437833
    2011-07-26 16:37:18.449:INFO:OverlayedAppProvider:loaded jtracTemplate=blue@1311687437833
    2011-07-26 16:37:18.449:INFO:OverlayedAppProvider:loaded jtracTemplate=green@1311687437833
    2011-07-26 16:37:18.449:INFO:OverlayedAppProvider:loaded jtracTemplate=red@1311687437833
    2011-07-26 16:37:18.450:WARN:OverlayedAppProvider:No webapp found for template: jtracTemplate=jtrac-2.1.0

Comments are closed.