Combining Javascript and CSS content

At the Rich Web Experience conference this week, there were many talks about performance and tools like ySlow. Everybody acknowledges that downloading lots of javascript and css with multiple script or link tags is a key reason that Ajax sites are slow to start, due to the latency of multiple round trips over the network. Yet almost every example I saw at the show had multiple script tags and css links.

More over, none of the frameworks that I know provide mechanisms for easily switching from development mode (with individual scripts and links) to production mode with combined scripts that are stripped and compressed and it is an error prone process to edit markup in the transition from development to production.

So I have written ConcatServlet that allows multiple scripts tags like:

to be replaced with the single tag:

The same servlet can also be used for css links.

The servlet uses RequestDispatcher.include, so it can be used with dynamic scripts, such as those generated for DWR. The down side of this approach is that it can’t get the last modified dates of the individual resources, so the browser will not well cache the combination. So the servlet may be run in development mode, where the content is always combined and served, or in production mode where the start time of the servlet is used as the last modified time. This is production, the servlet needs to be restarted before any changed scripts will be served in response to If-Modified-Since requests.

If this style of script and link tag is well received, then in future, the servlet could be modified to do a better job with detecting modifications, caching combinations and perhaps stripping comments and whitespace. It would also be possible to cache combinations in direct buffers so they can be served efficiently by Jetty’s NIO layer.

This servlet will be in the utilities jar of the 6.1.6 Jetty release.

Categories: GeneralUncategorized

9 Comments

Tuomas Kiviaho · 09/09/2007 at 13:30

With suitable HTML preprocessor the concat servlet could be hidden completely from development cycles by concatenating script tags based on their mime type and placement. Although this generates overhead it might be desirable to be able to handle files individually when debugging.

Brian Smith · 09/09/2007 at 17:39

Shouldn’t the concatenation be done in the build script? For example, use the Apache <concat> task to generate a file that is then deployed instead of the individual source files.

This could be combined with build-time XSLT transforms on templates to embed the scripts or stylesheets directly into key page templates. For example, a weblog would be better off embedding its CSS into each entry’s page instead of using a link, in order to survive slashdot/digg/reddit effects.

The servlet approach seems like it could easily suffer from DOS attacks. For example, an attacker could use it to repeatedly concat one large file (e.g. an image or movie) multiple times, causing out-of-memory conditions or even out-of-disk-space conditions.

(Your comment editor disables Firefox’s spellchecker’s ability to suggest corrections via the context menu. It also forces the text to be almost unreadably small on my small-screen, high-DPI laptop screen. Otherwise, it seems quite nice.)

Alexandre Rafalovitch · 09/09/2007 at 20:31

I think DOJO 0.9 introduced concept of layers that did do exactly what you were talking about. Did you have a chance to look at that?
Not that I am trying to discount your idea, as it is framework agnostic.

Greg Wilkins · 10/09/2007 at 05:36

Theoretically, doing the concat in the build cycle is the right thing to do, but
this is still a pain in development mode.

However, the main thing that I am advocating is script and link tags that list multiple content. For production, it may be best to turn off concatenation and only server pre-concatenated bundles that are named
with the ?foo.js&bar.js style.

The key thing is to have markup that will not need to be changed between development and testing.

Panagiotis Astithas · 10/09/2007 at 08:25

I believe GWT does exactly what you want. They concatenate and compress the generated Javascript and image files and serve them with unique names (via an MD5 on their contents) so they can be cached forever. This way they minimize the HTTP requests on startup, even on the first visit to the page.

Mike Wilson · 17/09/2007 at 13:52

JBoss Seam has had a similar way of including resources (component interfaces) for a while, see end of chapter 10.1 in http://docs.jboss.com/seam/1.1GA/reference/en/html/remoting.html

Jelan · 11/10/2007 at 10:08

Greg, have a look at this very usefull java lib which handles precisely what you are starting to do and beyond:

http://www.galan.de/projects/packtag

It might give you some cool ideas.

Regards,
Chris

cbh · 28/02/2008 at 03:37

Has anyone actually tried this with DWR? I wrote essentially the same mechanism — what I’ve found is that dispatch.include fails silently and returns an empty response. On the other hand, dispatch.forward works. It isn’t clear from the servlet-api docs or source code why this would be the case.

ConcatServlet behaves the same way as my code: including static js files works fine, including dwr js files does not. The problem does not seem to be another filter or servlet.

Greg Wilkins · 02/03/2008 at 23:38

No have not done this with DWR, and it will be a little more difficult as much of the content from DWR is dynamically generated.

However, the includes should work and if they are failing then I would suspect a buglet with DWR. Worth raising on the DWR list, as I know Joe is also concerned with the number of loads required and is likely taking his own actions to improve the loading.