Saturday 8 August 2009

The state of Maven in Debian

Hello World! First blog post, I'm joining the blogging world at last.

I have been working a lot on enabling Maven as first class citizen in Debian and as a tool to use when building Debian packages which follow the strict Debian policies (everything is built from source or from other Debian packages, no connection to Internet ispossible during the build - which means Maven cannot download dependencies)

First, Maven as a first class Debian citizen:
- Maven 2.2.0 is available in Debian experimental (soon, there are some NEW packages that need clearance).
- It is now built using most of the bootstrapping procedure, in particular Maven Shade plugin is used to build the final jar, so that means that the common libraries used by Maven are shadded (their Java packages have been renamed and prefixed with hidden. to avoid version clashes with plugins that may use a different version of the same libraries). In short, what's available in Debian is identical to the official Maven binary release.
- A cool bash tab completion script has been added. It should help you to find most available options, build phases and plugins very easily.
- Due to the contraints of Debian, Maven and its mirriad of dependencies are limited to one version. One version of plexus-utils, once version of modello and so on. No more 'Maven downloads the internet' effect!
- We are building a Maven repository for Debian. It is located at /usr/share/maven-repo and contains already a few hundred libraries with clean Maven POM descriptors.

At this point, I have to mention that this Maven repository is mostly compatible with what's available on Central, but there are important differences as I'm trying to keep both Debian and Maven happy. The major feature of this repository is that is usually contains 2 versions for each artifact. The first version is the normal version as you may find it in Central, for example junit:junit:3.8.2 . The second version is the Debian-managed version, for example 3.x. The jar for junit-3.x.jar is a symlink to junit-3.8.2.jar. So 3.x and 3.8.2 are the same jars. But there is a difference: 3.x is a series of (hopefully) compabible versions managed by Debian. In Debian, we try hard to limit the number of versions for the same library. So when junit 3.8.3 is released and packaged in Debian, 3.8.2 will be REMOVED from the repository.

Of course this means that any other library depending on junit 3.8.2 will have a problem. The second trick in this repository is that all dependencies refer to the Debian version of the library. So if another project uses Junit, then it will declare this dependency:

<dependencies><dependency>
<groupid>junit</groupid>
<artifactid>junit</artifactid>
<version>3.x</version>
</dependency>
</dependencies>


This Maven repository
- keeps only the most recent version of each library in a series
- maintains synthetic Debian versions of the libraries (usually named debian or 1.x,2.x,3.x)
- associates the Debian version of the artifact with the most recent version.

Now we have a repository that plays nicely with Maven and Debian. To fill it with lots of useful software is another matter. If you needed to edit all POM files by hand and replace all dependency versions with their Debian versions, that would be a lot of work. Fortunately, we have some tools to help us here:

maven-repo-helper: This package maintains the Maven repository and performs all cleanup and version transformations for you, starting with a POM file coming from a project or from the Maven Central repository. It provides a few tools (mh_lspoms, mh_cleanpom, mh_installpoms, mh_installjar) which can be used when you have a project that provides some libraries that you want to add to the Maven repository, but the build doesn't use Maven.

maven-ant-helper: This package can be used to build simple libraries which were built with Maven but where you need to use Ant for building the Debian package. It is used a lot for build Maven and Plexus libraries because it avoid a circular build dependency on Maven.

maven-debian-helper: This package allows you to use Maven itself for building your package in Debian. There is a CDBS rule that provides support for Maven. To satisfy the Debian policy, Maven is run in offline mode, so all build dependencies must have been packaged for Debian before you can build your package. Fortunately, most Maven plugins have already been packaged, and there is quite a good amount of commons Java libraries available as Debian packages with support for Maven.

maven-debian-helper provides a very handy tool to build the Debian packaging files from an existing Maven project: run mh_make on the root of your project, and if you have all project dependencies already packaged and installed, you should get 90% of the Debian packaging done for you. It takes me less than 20 minutes to package a new library for Debian now.

Torsten Werner from Debian has posted his status report as well, have a look for a more Debian oriented point of view.
If you are interesting in helping Debian to package more Java software, consider joining the pkg-java team and read more about building Debian packages with Maven here. To use the latest Maven and packaging utilities on Ubuntu, add my PPA to your sources.list

I'm also planning to add tighter integration between Maven and Debian, in particular the local Maven repository in ~/.m2/repository should be able to use jars already present in /usr/share/maven-repo

Next, I will post a detailed how to for packaging a Java project with Maven.

4 comments:

  1. I can understand maintaining certain debian versions of native libraries, but what's the benefit to having a "Debian approved" version of java jars? Especially in the example given of junit. If I intend to write tests that use junit 4.0 then you simply can't build my code and run tests because to do it on debian requires people to use some other arbitrary version seems quite odd. After all, we're talking about Java here not C.

    ReplyDelete
  2. Brian Fox: it aint a problem because we have both junit 3 and junit 4 in Debian.

    ReplyDelete
  3. That was just an example...it still doesn't explain the need to recompile jars that have no native linking...

    ReplyDelete
  4. Very cool!!!
    Do I have to wait for these libs to be packaged for debian before I can use it ?
    Dependency is not packaged in the Maven repository for Debian: junit:junit:4.4
    Dependency is not packaged in the Maven repository for Debian: org.codehaus.mojo:appassembler-maven-plugin:null
    Dependency is not packaged in the Maven repository for Debian: org.codehaus.mojo:exec-maven-plugin:null
    Dependency is not packaged in the Maven repository for Debian: org.codehaus.mojo:tomcat-maven-plugin:null
    Dependency is not packaged in the Maven repository for Debian: org.apache.cxf:cxf-bundle-jaxrs:2.3.0-SNAPSHOT
    Dependency is not packaged in the Maven repository for Debian: org.codehaus.mojo:tomcat-maven-plugin:null

    ReplyDelete