Sifting Logs in Jetty with Logback

Ever wanted to create log files at the server level that are named based on some sort of arbitrary context?It is possible to do with Slf4j + Logback + Jetty Webapp Logging in the mix.
Example projects for this can be found at github
https://github.com/jetty-project/jetty-and-logback-example
Modules:

/jetty-distro-with-logback-basic/

This configures the jetty distribution with logback enabled at the server level with     an example logback configuration.

/jetty-distro-with-logback-sifting/

This configures the jetty distribution with logback, centralized webapp logging,     a MDC handler, and a sample logback configuration that performs sifting based  on the incoming Host header on the requests.

/jetty-slf4j-mdc-handler/

This provides the Slf4J MDC key/value pairs that are needed to perform the     sample sifting with.

/jetty-slf4j-test-webapp/

This is a sample webapp+servlet that accepts arbitrary values on a form POST     and logs them via Slf4J, so that we can see the results of this example.

Basic Logback Configuration for Jetty

See the /jetty-distro-with-logback-basic/ for a maven project that builds this configuration.Note: the output directory /jetty-distro-with-logback-basic/target/jetty-distro/ is where this configuration will be built by maven.
What is being done:

1. Unpack your Jetty 7.x Distribution Zip of choice
   The example uses the latest stable release
   (7.4.5.v20110725 at the time of writing this)
2. Install the slf4j and logback jars into ${jetty.home}/lib/logging/
   • slf4j-api-1.6.1.jar
   • logback-classic-0.9.29.jar
   • logback-core-0.9.29.jar
3. Configure ${jetty.home}/start.ini to add the lib/logging directory into the server classpath

   #===========================================================
   # Start classpath OPTIONS.
   # These control what classes are on the classpath
   # for a full listing do
   #   java -jar start.jar --list-options
   #-----------------------------------------------------------
   OPTIONS=Server,resources,logging,websocket,ext
   #-----------------------------------------------------------
   #===========================================================
   # Configuration files.
   # For a full list of available configuration files do
   #   java -jar start.jar --help
   #-----------------------------------------------------------
   etc/jetty.xml
   # etc/jetty-requestlog.xml
   etc/jetty-deploy.xml
   etc/jetty-webapps.xml
   etc/jetty-contexts.xml
   etc/jetty-testrealm.xml
   #===========================================================

4. Create a ${jetty.home}/resources/logback.xml file with the configuration you want.

   <?xml version="1.0" encoding="UTF-8"?>
   <!--
     Example LOGBACK Configuration File
     http://logback.qos.ch/manual/configuration.html
     -->
   <configuration>
     <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
       <!-- encoders are assigned the type
            ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
       <encoder>
         <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
       </encoder>
     </appender>
     <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
       <file>${jetty.home}/logs/jetty.log</file>
       <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
         <!-- daily rollover -->
         <fileNamePattern>jetty_%d{yyyy-MM-dd}.log</fileNamePattern>
         <!-- keep 30 days' worth of history -->
         <maxHistory>30</maxHistory>
       </rollingPolicy>
       <encoder>
         <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
       </encoder>
     </appender>
     <root level="info">
       <appender-ref ref="STDOUT" />
       <appender-ref ref="FILE" />
     </root>
   </configuration>

That’s it, now you have (in the following order)

1. Jetty configured to use slf4j
   (via the existance slf4j-api.jar in the classpath on Jetty startup)
2. slf4j configured to use logback
   (via the existance of logback-core.jar in the classpath at Jetty startup)
3. logback configured to produce output to:
   • ${jetty.home}/logs/jetty.log (with daily rolling)
   • and STDOUT console

Pretty easy huh?
Go ahead and start Jetty.

$ java -jar start.jar

You’ll notice that the log events being produced by Jetty are being handled by Slf4j and Logback is doing the writing of those events to the STDOUT console and logs/jetty.log file
Now lets try something a bit more complex.

Sifting Logs produced by webapps via Hostname using Logback in Jetty

Lets say we have several virtual hosts, or a variety of DNS hostnames for the Jetty instance that is running.And you want to have the logging events being produced by the webapps captured into uniquely named log files by the hostname that the request came in on.
This too is possible with logback, albeit with a little help from slf4j and jettty WebappContextClassloader configuration.
See the /jetty-distro-with-logback-sifting/ project example from the github project above for a build-able configuration of the following instructions:

1. Unpack your Jetty 7.x Distribution Zip of choice.
   The example uses the latest stable release.
   (7.4.5.v20110725 at the time of writing this)
2. Install the slf4j and logback jars into ${jetty.home}/lib/logging/
   • slf4j-api-1.6.1.jar
   • logback-classic-0.9.29.jar
   • logback-core-0.9.29.jar
   • jetty-webapp-logging.jar (be sure you match your jetty version here)
   • jetty-slf4j-mdc-handler.jar (found in the examples project)
3. Configure ${jetty.home}/start.ini to add the lib/logging directory into the server classpath

   #===========================================================
   # Start classpath OPTIONS.
   # These control what classes are on the classpath
   # for a full listing do
   #   java -jar start.jar --list-options
   #-----------------------------------------------------------
   OPTIONS=Server,resources,logging,websocket,ext
   #-----------------------------------------------------------
   #===========================================================
   # Configuration files.
   # For a full list of available configuration files do
   #   java -jar start.jar --help
   #-----------------------------------------------------------
   etc/jetty.xml
   # etc/jetty-requestlog.xml
   etc/jetty-mdc-handler.xml
   etc/jetty-deploy.xml
   etc/jetty-webapps.xml
   etc/jetty-contexts.xml
   etc/jetty-webapp-logging.xml
   etc/jetty-testrealm.xml
   #===========================================================

   The key entries here are the addition of the “logging” OPTION to load the classes in ${jetty.home}/lib/logging into the jetty server classpath, and the 2 new configuration files:

   etc/jetty-mdc-handler.xml

   This adds wraps the MDCHandler found in jetty-slf4j-mdc-handler around all of the handlers in Jetty Server.

   etc/jetty-webapp-logging.xml

   This adds a DeploymentManager lifecycle handler that configures the created Webapp’s Classloaders to deny      acccess to any webapp (war) file contained logger implementations in favor of using the ones that exist      on the server classpath.      This is a concept known as Centralized Webapp Logging.

4. Create a ${jetty.home}/resources/logback.xml file with the configuration you want.

   <?xml version="1.0" encoding="UTF-8"?>
   <!--
     Example LOGBACK Configuration File
     http://logback.qos.ch/manual/configuration.html
     -->
   <configuration>
     <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
       <!-- encoders are assigned the type
            ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
       <encoder>
         <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
       </encoder>
     </appender>
     <appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
       <!-- in the absence of the class attribute, it is assumed that the
            desired discriminator type is
            ch.qos.logback.classic.sift.MDCBasedDiscriminator -->
       <discriminator>
         <key>host</key>
         <defaultValue>unknown</defaultValue>
       </discriminator>
       <sift>
         <appender name="FILE-${host}" class="ch.qos.logback.core.rolling.RollingFileAppender">
           <file>${jetty.home}/logs/jetty-${host}.log</file>
           <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
             <!-- daily rollover -->
             <fileNamePattern>jetty-${host}_%d{yyyy-MM-dd}.log</fileNamePattern>
             <!-- keep 30 days' worth of history -->
             <maxHistory>30</maxHistory>
           </rollingPolicy>
           <encoder>
             <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
           </encoder>
         </appender>
       </sift>
     </appender>
     <root level="INFO">
       <appender-ref ref="STDOUT" />
       <appender-ref ref="SIFT" />
     </root>
   </configuration>

That’s it, now you have (in the following order):

1. Jetty configured to use slf4j
   (via the existence slf4j-api.jar in the classpath on Jetty startup)
2. Jetty is configured to modify incoming Webapp’s classloaders to favor server logging classes   over the webapp’s own logging classes.
   (a.k.a. Centralized Webapp Logging)
3. slf4j configured to use logback
   (via the existence of logback-core.jar in the classpath at Jetty startup)
4. logback configured to produce output to:
   • ${jetty.home}/logs/jetty-${host}.log (with daily rolling)  and using “unknown” for log events that don’t originate from a request.
   • and STDOUT console

Not too bad huh?
Go ahead and start Jetty.

$ java -jar start.jar

If you have started the distribution produced by the example configuration, you can use the provided /slf4j-tests/ context to experiment with this.
Go ahead and use the default URL of http://localhost:8080/slf4j-tests/

Now try a few more URLs that are for the same Jetty instance.

• http://127.0.0.1:8080/slf4j-tests/
• http://127.0.1.1:8080/slf4j-tests/
• http://lapetus:8080/slf4j-tests/
• http://lapetus.local:8080/slf4j-tests/

Note: “lapetus” is the name of my development machine.
You should now have a few different log files in your ${jetty.home}/logs/ directory.