Implementing JAR file support in ABCL

News

February 7, 2010

Committed a (slight revised) changeset to ABCL trunk as [svn 12422]

February 5, 2010

Second release candidate patch that both loads from network locations and replaces the functionality of ASDF:LOAD-JUST-FASLS-OP into ASDF:LOAD-op. This small update to the patchset released earlier today fixes the problem with errors relating to SYS:PATHNAME-JAR-P not being an external symbol.

January 28, 2010

Third patchset that is able to load ASDF from jars via ADSF:LOAD-JUST-FASLS-OP.

January 26, 2010

Updated patchset that passes all tests under Windows

January 24, 2010

First public availability.

Overview

This work has now been merged to the ABCL trunk.

.Notable features besides fixing the buggy implemnations are the ability to load FASLs from network locations and the ability to load ASDF systems from JAR files via the standard ASDF:LOAD-OP verb. ABCL's implementation of ASDF now does not attempt to compile ASDF systems if the the output location for the FASLs would be within a JAR file.

To load ASDF from JAR files, one first sets ASDF:*CENTRAL-REGISTRY* to contain the JAR location. For the 'test.jar' located in the GoogleCode reporsitory containing a top-level entry for 'test.asd', the appropiate command would be

CL-USER> (pushnew "jar:http://abcl-dynamic-install.googlecode.com/files/test.jar!/" asdf:*central-registry*)

Then one just uses the standard ADSF verb

CL-USER> (asdf:operate 'asdf:just-load-op :test)

And Voila! Off to the races as you can then execute the loaded defun

CL-USER> (foo) 42

The implications of using this with pathnames starting with "jar:URI" where URI is a network location that references loadable locations seem twofold. On the one hand, with suitable cleverness in a server handling URI requests with a tiny bit of additional HTTP headers to indicating the requesting ABCL version, we could have standarized network runnable ASDF systems for ABCL. And with suitable control of DNS, could fairly easily make a distributed system that would scale. On the other hand, we have no security mechanism in place for verifying code origins or integrity, so loading malicious code would be quite possible.

My >notes on the design have been folded into the patchset itself, under the 'doc/pathnames' directory. The implementation deviates a little from my original posting of a couple weeks ago. The main change from that design is that the PATHNAME DEVICE is now possibly a proper list of jars so that

#p"jar:jar:file:foo.jar!/bar.abcl!/bar._"

constructs a Pathname that has the layout

Pathname: { device: ( Pathname { name: "foo"; type: "jar"}; Pathname { name: "bar"; type: "abcl"}; ); name: "bar"; type: "_"; }

This patch implements support for specifying ABCL FASLs from JAR files to the documented level for JarURLConnection for the LOAD, TRUENAME, and PROBE-FILE commands.

It is not currently possible to use JAR Pathnames with OPEN, which is currently quite securely tied in the ABCL internals to the notion of java.io.File. In the interests of getting this patchset stable, I decided to forgo that work until another time.

The LOAD mechanism now searches the JAR in the standard method as on the filesystem, so the command

CL-USER> (load #p"jar:file:invoke.jar!/a/b/invoke")

will search invoke.jar for entries named "a/b/invoke.abcl", and "a/b/invoke.lisp". If both exists, it will compare timestamps. If neither exists it will attempt to load "a/b/invoke._".

Internally within ABCL this patch reworks most of the loading routines to use Pathnames instead of strings, and reducing a lot of the complexity in looking for the correct file. All the resolution methods have been collected in the single Load.findLoadableFile() for easier maintenance.

There are both Java based and Lisp based tests included. The Lisp based tests now run under both UNIX and Windows as I have implemented the necessary SYS:UNZIP function (as well as augmenting SYS:ZIP to allow creation of hierarchial entries.)

Patches