PhoenixmlDb

XQuery Library Modules

Importing XQuery library modules from files and http(s) URLs, the module settings and limits, resolution rules, fn:load-xquery-module, and the errors each host reports

#XQuery Library Modules

A library module is an XQuery file that begins module namespace p = "uri";. It is a file on disk or a document at an http(s) URL. A query brings it in with

import module namespace p = "uri" at "location";
            

or with fn:load-xquery-module("uri", map { "location-hints": "location" }).

Library modules work in the embedded engine, the gRPC server and the REST server. For the language itself, see Functions and Modules.

#Enabling modules

Nothing outside the stored documents is readable by default. An operator lists the directories and http(s) origins that modules may be read from. The same lists govern every other read; see Resource Access.

#Embedded engine

using PhoenixmlDb.Storage;
using PhoenixmlDb.Storage.Security;
var options = new ResourceAccessOptions();
options.AllowedFileRoots.Add("/srv/xquery-modules");            // absolute path of an existing directory
options.AllowedHttpOrigins.Add("https://modules.example.com");  // scheme://host[:port], no path
using var db = new DocumentDatabase("/var/lib/mydb");
db.ResourceAccessPolicy = ResourceAccessPolicy.Create(options);
            

• The default is ResourceAccessPolicy.DenyAll.

• ResourceAccessPolicy.Create throws ArgumentException listing each bad value. ResourceAccessOptions.Validate() returns the same list.

• Under ResourceAccessPolicy.Unrestricted, the XQuery library resolves and reads locations itself, and the limits and error texts on this page do not apply.

#Servers

The gRPC server (PhoenixmlDb.Server) and the REST server bind the same section, PhoenixmlDb:ResourceAccess:

{
  "PhoenixmlDb": {
    "ResourceAccess": {
      "AllowedFileRoots": [ "/srv/xquery-modules" ],
      "AllowedHttpOrigins": [ "https://modules.example.com" ]
    }
  }
}
            

For the gRPC server, as environment variables:

PhoenixmlDb__ResourceAccess__AllowedFileRoots__0=/srv/xquery-modules
PhoenixmlDb__ResourceAccess__AllowedHttpOrigins__0=https://modules.example.com
            

A bad value stops startup with a message that names the key, for example:

PhoenixmlDb:ResourceAccess:MaxModules must be between 1 and 1024 (was 0).
            

The gRPC server logs the policy at startup:

file roots: …; HTTP origins: …; module limits: depth 32, 256 modules, 4194304 bytes each, 33554432 bytes in total; text resources: 67108864 bytes each, 268435456 bytes in total
            

#Settings

All settings are under PhoenixmlDb:ResourceAccess. Each is also a property of the same name on ResourceAccessOptions.

Bounds that are not settings:

• An import chain longer than 64 modules is XQST0059, whatever MaxModuleDepth is. It applies to the longest chain in the import graph.

• One http(s) fetch may take 30 seconds, headers and body together.

• Redirects are not followed.

PhoenixmlDb:ResourceAccess:AllowStylesheetFileAccess is not a setting. A configuration that carries the key, with any value, fails validation at startup:

PhoenixmlDb:ResourceAccess:AllowStylesheetFileAccess is not a setting: what a stylesheet may read follows PhoenixmlDb:ResourceAccess:AllowedFileRoots and PhoenixmlDb:ResourceAccess:AllowedHttpOrigins. Remove it.
            

#Worked example

Five documents in the container, b1.xml … b5.xml:

<book isbn="isbn-1"><title>t1</title><price>10</price></book>
<book isbn="isbn-2"><title>t2</title><price>20</price></book>
…
<book isbn="isbn-5"><title>t5</title><price>50</price></book>
            

Module file /srv/xquery-modules/shop.xqm, with /srv/xquery-modules in AllowedFileRoots:

module namespace shop = "urn:example:shop";
declare variable $shop:vat as xs:decimal := 0.2;
declare variable $shop:factor as xs:decimal := 1 + $shop:vat;
declare %private variable $shop:currency := "EUR";
declare %private function shop:format($amount as xs:decimal) as xs:string {
  $shop:currency || " " || string($amount)
};
declare function shop:gross($book as element(book)) as xs:string {
  shop:format(xs:decimal($book/price) * $shop:factor)
};
declare function shop:titles() as xs:string* {
  collection()/book/title/string()
};
declare function shop:cheaper-than($limit as xs:decimal) as element(book)* {
  collection()/book[xs:decimal(price) lt $limit]
};
declare function shop:by-isbn($isbn as xs:string) as element(book)* {
  collection()/book[@isbn = $isbn]
};
declare function shop:label($book as element(book)) as xs:string {
  "book " || $book/title
};
            

Query:

import module namespace shop = "urn:example:shop" at "/srv/xquery-modules/shop.xqm";
shop:gross(/book)
            

Result: EUR 12, EUR 24, EUR 36, EUR 48, EUR 60.

More queries with the same import:

The same module through fn:load-xquery-module:

load-xquery-module("urn:example:shop", map { "location-hints": "/srv/xquery-modules/shop.xqm" })
  ?functions(QName("urn:example:shop", "gross"))(1)(/book)
            

Result: EUR 12, EUR 24, EUR 36, EUR 48, EUR 60.

#How a module function reaches the stored documents

• A function body has no context item. Pass the node in (shop:gross(/book)) or read collection(). A function that uses / with no argument is XPDY0002.

• A query whose answer depends on one document at a time runs once per document.

• A query that reaches collection(), in its own text or inside an imported module function it calls, runs once over the whole container.

• A module that is only loaded at run time with fn:load-xquery-module is not looked into when the engine decides how to run the query. count(collection()) inside a loaded function answers once per document. Read the collection in the query text to get one answer.

#Resolution rules

• Every location is checked against the allowlist, at every depth.

• at "a.xqm", "b.xqm" loads both as one module. Every listed location must exist, hold a library module, and declare the imported namespace.

• Two modules that import each other compile and call each other.

• A location is allowed when the file it resolves to is inside an allowed directory. A symbolic link inside an allowed directory that leads out of it is refused.

• Only regular files are read.

• A module is read once per query, however many documents the query runs over. Nothing is kept between queries: the next query reads and compiles the module again.

• If the query and a module write the same relative hint and the two resolve to different files, the query is refused (XQST0059 … name different files).

• import schema written inside a library module is XQST0009. Import the schema in the query.

• Cancelling the query stops the reading of modules.

• A query that includes a module read from an http(s) location reads no files; see Modules read over HTTP.

# fn:load-xquery-module

• location-hints is required. Without it: FOQM0002 ("no location-hints were given, and modules are loaded from location hints only").

• The result is map { "functions": map(QName → map(arity → function)), "variables": map(QName → value) } with the public declarations only.

• The variables option (a map with xs:QName keys) binds the module's external variables. context-item supplies its context item.

• Several hints load one module.

• The limits are the query's. A loaded module continues from the depth of the code that loads it and counts in the same totals.

• The same load made again by the query itself in one evaluation (same module URI and locations, no variables, no context-item) returns the module already loaded. Every other load is a new module. One evaluation loads at most MaxModules.

• A loaded module runs under the database's query limits, including the regular-expression match time limit.

#Errors

How each host reports a query error:

On the REST server, the compile-only endpoints read the imported modules too: POST /api/query/validate reports each static error with its own code, and POST /api/query/compile answers 400 with xqueryErrorCode.

#Known limits

• Modules are not stored in the database. A module is a file or a URL.

• There is no compiled-module cache. Every query reads and compiles its modules again, and fetches an http(s) module again.

• A predicate inside a module function is not answered from a value index. /book[@isbn = 'isbn-7'] ! shop:label(.) uses the index; shop:by-isbn('isbn-7') returns the right book from a scan.

• A module loaded only at run time does not change how the query is run (see How a module function reaches the stored documents).

• fn:unparsed-text and fn:json-doc in queries read files only, not http(s).

• XSLT cannot import XQuery modules.

#Next Steps

• Resource Access: everything else a query or stylesheet may read

• Server Configuration