Jetty Overlayed WebApp Deployer

The Jetty Overlay Deployer allows multiple WAR files to be overlayed so that a web application can be customised, configured and deployed without the need to unpack, modify and repack the WAR file. This has the benefits of:

• The WAR file may be kept immutable, even signed, so that it is clear which version has been deployed.
• All modifications made to customise/configure the web application are kept in a separate wars and thus are easily identifiable for review and migration to new versions.
• A parameterised template overlay can be created that contains common customisations and configuration that apply to many instances of the web application (eg for multi-tenant deployment).
• Because the layered deployment clearly identifies the common and instance specific components, then Jetty is able to share classloaders and static resource caches for the template, greatly reducing the memory footprint of multiple instances

This blog is a tutorial of how to configure Jetty to use the Overlay deployer, and how to deploy multiple instances of the JTrac web application.

Overview

The customisation, configuration and deployment a web application bundled as a WAR file frequently includes some or all of:

• Editing the WEB-INF/web.xml file to set init parameters, add filters/servlets or to configure JNDI resources.
• Editing other application specific configuration files in WEB-INF/*
• Editing container specific configuration files in WEB-INF/* (eg jetty-web.xml or jboss-web.xml)
• Adding/modifying static content such as images and css to style/theme the webapplication
• Adding jars to the container classpath for Datasource and other resources
• Modifying the container configuration to provide JNDI resources

The result is that the customisations and configurations are blended into both the container and the WAR file. If either the container or the base WAR file are upgraded to a new version, it can be a very difficult and error prone task to identify all the changes that have been made and to reapply them to a new version.

Overlays

To solve the problems highlighted above, jetty 7.4 introduces WAR overlays (which are a concept borrowed from the maven war plugin). An overlay is basically just another WAR files, whose contents are merged on top of the original WAR so that files may be added or replaced.

However,  jetty overlays also allow mixin fragments of web.xml, so the configuration can be modified without being replaced

Jtrac Overlay Example

The jtrac issue tracking webapplication is a good example of a typical web application, as it uses the usual suspects of libs: spring, hibernate, dom4j, commons-*, wicket, etc. So I’ve used it as the basis of this example.

The files for this demonstration are available in overlays-demo.tar.gz. This could be expanded on top of the jetty distribution, but for this tutorial we will expand it to /tmp and install the components step by step:

cd /tmp
wget http://webtide.intalio.com/wp-content/uploads/2011/05/overlays-demo.tar.gz
tar xfvz overlays-demo.tar.gz
export OVERLAYS=/tmp/overlays

Configuring Jetty for Overlays

Overlays support is included in jetty distributions from 7.4.1-SNAPSHOT onwards, so you can download a distribution from oss.sonatype.org or maven central(once 7.4.1 is released) and unpack into a directory.    The start.ini file then needs to be edited so that it includes the overlay option and configuration file.   The resulting file should look like:

OPTIONS=Server,jsp,jmx,resources,websocket,ext,overlay
etc/jetty.xml
etc/jetty-deploy.xml
etc/jetty-overlay.xml

The smarts of this are in etc/jetty-deploy.xml files, which installs the OverlayedAppProvider into the DeploymentManager. Jetty can then be started normally:

java -jar start.jar

Jetty will now be listening on port 8080, but with no webapp deployed.   The rest of the tutorial should be conducted in another window with the JETTY_HOME environment set to the jetty distribution directory.

Installing the WebApp

The WAR file for this demo can be downloaded and deployed using the following commands, which essentially downloads and extracts the WAR file to the $JETTY_HOME/overlays/webapps directory

cd /tmp
wget -O jtrac.zip http://sourceforge.net/projects/j-trac/files/jtrac/2.1.0/jtrac-2.1.0.zip/download
jar xfv jtrac.zip jtrac/jtrac.war
mv jtrac/jtrac.war $JETTY_HOME/overlays/webapps

When you have run these commands (or equivalent), you will see in the jetty server window a message saying that the OverlayedAppProvider has extracted and loaded the war file:

2011-05-06 10:31:54.678:INFO:OverlayedAppProvider:Extract jar:file:/tmp/jetty-distribution-7.4.1-SNAPSHOT/overlays/webapps/jtrac-2.1.0.war!/ to /tmp/jtrac-2.1.0_236811420856825222.extract
2011-05-06 10:31:55.235:INFO:OverlayedAppProvider:loaded jtrac-2.1.0@1304641914666

Unlike the normal webapps dir, loading a war file from the overlays/webapp dir does not deploy the webapplication.  It simply makes it available to be used as the basis for templates and overlays.

Installing a Template Overlay

A template overlay is a WAR structured directory/archive that contains just the files that have been added or modified to customize/configure the webapplication for all instances that will be deployed.

The demo template can be installed from the downloaded files with the command:

mv $OVERLAYS/jtracTemplate=jtrac-2.1.0 $JETTY_HOME/overlays/templates/

In the Jetty server window, you should see the template loaded with a message like:

2011-05-06 11:00:08.716:INFO:OverlayedAppProvider:loaded jtracTemplate=jtrac-2.1.0@1304643608715

The contents of the loaded template is as follows:

templates/jtracTemplate=jtrac-2.1.0
└── WEB-INF
    ├── classes
    │   └── jtrac-init.properties
    ├── log4j.properties
    ├── overlay.xml
    ├── template.xml
    └── web-overlay.xml

The name of the template directory (or it could be a war) uses the ‘=’ character in jtracTemplate=jtrac-2.1.0 to separates the name of the template from the name of the WAR file in webapps that it applies to.  If  = is a problem, then -- may also be used.

WEB-INF/classes/jtrac-init.properties – replaces the jtrac properties file with an empty file, as the properties contained within it are configured elsewhere

WEB-INF/log4j.properties – configures the logging for all instances of the template.

WEB-INF/overlay.xml – a Jetty XML formatted IoC file that is used to inject/configure the ContextHandler for each instances. In this case it just sets up the context path:

  /

WEB-INF/template.xml – a Jetty XML formatted IoC file that is used to inject/configure the resource cache and classloader that is shared by all instances of the template. It is run only once per load of the template:

  
    true
    10000000
    1000
    64000000
  

WEB-INF/web-overlay.xml – a web.xml fragment that is overlayed on top of the web.xml from the base WAR file, that can set init parameters and add/modify filters and servlets. In this it sets the application home and springs rootKey:

  
    jtrac.home
    /tmp/jtrac-${overlay.instance.classifier}
  
  
    webAppRootKey
    jtrac-${overlay.instance.classifier}
  
  

Note the use of parameterisation of values such as ${overlays.instance.classifier}, as this allows the configuration to be made in the template and not customised for each instance.

Without the overlayed deployer, all the configurations above would still need to have been made, but rather that being in a single clear structure they would have been either in the servers common directory, the servers webdefaults.xml (aka server.xml), or baked into the WAR file of each application instance using copied/modified files from the original. The overlayed deployer allows us to make all these changes in one structure, more over it allows some of the configuration to be parameterised to facilitate easy multi-tenant deployment.

Installing an Instance Overlay

Now that we have installed a template, we can install one or more instance overlays, which deploy the actual web applications:

mv /tmp/overlays/instances/jtracTemplate=blue $JETTY_HOME/overlays/instances/
mv /tmp/overlays/instances/jtracTemplate=red $JETTY_HOME/overlays/instances/
mv /tmp/overlays/instances/jtracTemplate=blue $JETTY_HOME/overlays/instances/

As each instance is moved into place, you will see the jetty server window react and deploy that instance. Within each instance, there is the structure:

instances/jtracTemplate=red/
├── WEB-INF
│   └── overlay.xml
├── favicon.ico
└── resources
    └── jtrac.css

WEB-INF/overlay.xml – a Jetty XML format IoC file that injects/configures the context for the instance. In this case it sets up a virtual host for the instance:

  
    
      127.0.0.2
      red.myVirtualDomain.com
    
  

favicon.ico – replaces the icon in the base war with one themed for the instance colour.

resources/jtrac.css – replaces the style sheet from the base war with one themed for the instance colour

The deployed instances can now be viewed by pointing your browser at http://127.0.0.1:8080, http://127.0.0.2:8080 and http://127.0.0.3:8080. The default username/password for jtrac is admin/admin.

Things to know and notice

• Each instance is themed with images and styles sheets from the instance overlay.
• Each instance is running with it’s own application directory (eg. /tmp/jtrac-red), that is set templates web-overlay.xml.
• The instances are distinguished by virtual host that is set in the instance overlay.xml
• The static content from the base war and template are shared between all instances. Specifically there is a shared ResourceCache so only a single instance of each static content is loaded into memory.
• The classloader at the base war and template level is shared between all instances, so that only a single instance of common classes is loaded into memory. Classes with non shared statics can be configured to load in the instances classloader.
• All overlays are hot deployed and dependencies tracked. If an XML is touched in an instance, it is redeployed. If an XML is touched in a template, then all instances using it are redeployed. If a WAR file is touched, then all templates and all instances dependant on it are redeployed.
• New versions can easily be deployed. Eg when jtrac-2.2.0.war becomes available, it can just be dropped into overlays/webapps and then rename jtracTemplate=jtrac-2.1.0 to jtracTemplate=jtrac-2.2.0
• There is a fuller version of this demo in overlays-demo-jndi.tar.gz, that uses JNDI (needs options=jndi,annotations and jetty-plus.xml in start.ini) and shows how extra jars can be added in the overlays.