Packaging System

1 Introduction

XSLT, XQuery and XProc are amazing programming languages. But they lack a large choice of libraries, and when such libraries do exist, this is a challenge to install. There is no automatic install process, the rules are different for each processor, library authors do not follow the same rules regarding the info they provide, the cataloging, the way they reference third-party libraries, etc.

All those problems (well, most of them) can be addressed by a packaging system that would be broadly adopted by processor vendors and library authors. The cornerstone of such a system is the packaging format: a description of the information to be provided by the library authors and how to provide and structure them.

2 Concepts

<package>
   module
</package>

<module name = NCName
        version = string>
   title,
   (xslt
   |xquery
   |xproc
   |xsd
   |rng
   |rnc
   |...)+
</module>

3 Package descriptor

<package name = uri>
   dependency*,
   module
</package>

<dependency name = uri>
   empty
</dependency>

<module name = NCName
        version = string>
   title,
   (xslt
   |xquery
   |xproc
   |xsd
   |rng
   |rnc
   |...)+
</module>

<xslt>
   import-uri,
   file
</xslt>

<xquery>
   (namespace
   |import-uri),
   file
</xquery>

<xquery>
   namespace,
   file
</xquery>

<xproc>
   import-uri,
   file
</xproc>

<xsd>
   namespace,
   file
</xsd>

<rng>
   import-uri,
   file
</rng>

<rnc>
   import-uri,
   file
</rnc>

<schematron>
   import-uri,
   file
</schematron>

<nvdl>
   import-uri,
   file
</nvdl>

4 Processor behaviour

A processor is any program that use packaged components. For instance, an XSLT processor uses XSLT stylesheets (as well as XML schemas for an XSLT 2.0 SA processor), an XML database can use XQuery modules, XSLT stylesheets or any other kind of component, etc. But processors include also IDEs, editors, and any program that could want to use packaged components and that supports this specification.

The installed package list is implementation-defined. Each implementation (for a specific processor) can define its own way to install and remove packages, as long as it properly documents it. A processor should use, when appropriate, the on-disk repository layout as defined below.

When a reference to a file of a specific kind is done via an absolute URI, a processor must look up for this URI in the corresponding URI space in the repository. How the repository is set to the processor is implementation-defined (a processor can also use a list of repositories, and enable or disable some libraries in any implementation-defined way.)

The URI space to use is defined by the nature of the reference. An XSLT href attribute on xsl:import will use the xslt URI space, while it will use the xsd space for xsl:import-schema.

<p:xslt>
   <p:input port="stylesheet">
      <p:document href="..." pkg:kind="xslt"/>
   </p:input>
   ...
</p:xslt>

5 On-disk repository layout

This section defines a standard structure for on-disk repositories. An implementation can choose to not support this kind of repository and to define its own one (or even to not define it publicly, just to provide the ability to install and remove packages, in a clearly documented way.) However, there are several advantages to support this structure, the most obvious one is to be able to benefit from existing tools to manage such repositories as well as existing libraries to access those repositories.

The resolving machinery is based on OASIS XML Catalogs [Catalogs]. The repository is a simple directory, each subdirectory of which is an installed package (aka a package dir.) The only exception to this is the subdirectory .expath-pkg/ which is dedicated to store working information about the installed packages, among which the catalogs (aka the admin dir.)

[repository-dir]/
   .expath-pkg/
      xquery-catalog.xml
      xslt-catalog.xml
      lib1/
         xquery-catalog.xml
         xslt-catalog.xml
      lib2/
         ...
   lib1/
      query.xq
      style.xsl
   lib2/
      ...

The package dirs are really simple: they are simply an unzipped version of the XAR file. The name of the directory is simply the same as the name of the module in the package. The admin dir contains a catalog for each URI space (the catalog for one specific URI space can not be there if there is no one file in that URI space in the whole repository.) The name of such a catalog is [space]-catalog.xml where [space] is either xslt, xquery, rnc, etc. Those catalogs are called repository catalogs. It also contains a subdirectory for each installed package, with the same naming convention. In turn, those directories contain catalog files, containing the mappings defined in the corresponding package descriptors (pointing to the actual files installed in the package dirs.) Those are called the package catalogs. They follow the same naming convention than the repository catalogs (divided by URI spaces.) The repository catalogs just include the several package catalogs for the same URI space.

[ ... TODO ... ]

8 Editorial notes

• Should the package system define a set of XPath functions? Instead of just defining the package format and letting everything else as implementation-defined, should it in addition define a module of functions to install, delete, and more generally manage packages from within a processor?

  Drawback: potential problems if the processor requires to be stopped?

  Advantages: enables writing tools on top of the system (one single graphical package manager for one system, simply using the XPath functions, as well as easy integration within IDEs; or even other systems could be more easily be built on top of it, like a packaging system for XRX applications for instance.)

• Should we add a generic "xml" URI space, for any XML document?

Add a restriction (actual add a definition of such URIs at the beginning) on public URIs: require them to be absolute, and maybe only either HTTP or URN (or at least prohibit the FILE scheme.)

• Interesting use case with XProc and NVDL: How to configure the XSLT processor in Calabash with the NVDL URI space for an NVDL implementation for XProc written using XSLT as a plain library step?

About the standard directory layout: should we still use XML Catalogs, or just a simple format to map public URIs to files? (should really this repository layout be defined here, anyway?)

Add in the package descriptor an element for a description, or at very least a URL to the home of the project.