Tools
XQuery and XSLT modules
XQuery 3.1 and XSLT 3.0 library modules for JSON, XML, CSV and YAML
#XQuery and XSLT modules
Library modules make common JSON, XML, CSV and YAML tasks short: load files, select values, group,
count, compare, and convert between formats. They are plain XQuery 3.1 and XSLT 3.0 source, so
they run on the xquery and xslt tools, the PhoenixmlDb engines
and other XQuery 3.1 and XSLT 3.0 processors.
|
Module |
Language |
Namespace |
File |
|---|---|---|---|
|
JSON ( |
XQuery |
|
|
|
XML ( |
XQuery |
|
|
|
CSV ( |
XQuery |
|
|
|
Report ( |
XQuery |
|
|
|
JSON to XML ( |
XSLT |
|
|
|
YAML ( |
XSLT |
|
The source, tests and releases are in the xquery-modules repository. The license is Apache 2.0.
#Import a module
The at URL of an XQuery import, or the href of an xsl:import, tells the processor where to
read the module. The processor reads the file when it compiles the query or stylesheet; nothing is
installed.
import module namespace pj = "https://phoenixml.dev/modules/json"
at "https://phoenixml.dev/modules/1.1.0/json.xqm";
pj:select(pj:load("orders.json", static-base-uri()), "items.*.name")
<xsl:import href="https://phoenixml.dev/modules/1.1.0/yaml.xsl"/>
The namespace is the module's name only; the processor does not read anything from it.
Use a version in the URL, such as 1.1.0. The file at a version URL does not change; a new
release gets a new version URL. The same files are in the GitHub repository, for example at
https://raw.githubusercontent.com/phoenixmldb/xquery-modules/v1.1.0/modules/json.xqm.
Or use a local copy. Download the file, put it next to your query, and import it by its path. A local copy does not need the network, and you control which code your queries run:
import module namespace pj = "https://phoenixml.dev/modules/json" at "json.xqm";
Resource policy. A host that runs queries under a ResourcePolicy must allow the module URL
for import, and the data files for reading. The import rules, settings and limits are in
XQuery library modules.
#Relative paths
A relative path in a library module resolves against the module's own location, not against your
query. For a module read from phoenixml.dev, "orders.json" would point to phoenixml.dev. The
load functions therefore take your query's base URI as a second argument:
pj:load(("a.json", "b.json"), static-base-uri())
With one argument, load takes absolute URIs only, and a relative path raises the error
pj:relative-uri (or px:relative-uri, pc:relative-uri).
#JSON functions (pj)
|
Function |
Result |
|---|---|
|
|
Each JSON file, parsed. The URIs must be absolute. |
|
|
Each JSON file, parsed. Paths resolve against |
|
|
Values at a dotted path. A step is a key, |
|
|
All values of field |
|
|
A map from each dotted path ( |
|
|
All array members, with nested arrays flattened. |
|
|
Each record with only the fields in |
|
|
The records in order of a field, smallest first; |
|
|
A map from each value of field |
|
|
CSV text with a header row. Values with a comma, quote or line break are quoted; maps and arrays are written as JSON. |
|
|
The W3C XML representation ( |
(: Items over 3, from two files, as CSV :)
let $items := pj:records(pj:load(("data/a.json", "data/b.json"), static-base-uri())?items)
return pj:to-csv($items[?price > 3], ("name", "price"))
#XML functions (px)
|
Function |
Result |
|---|---|
|
|
Each XML document. The URIs must be absolute. |
|
|
Each XML document. Paths resolve against |
|
|
The element structure: one line for each element path, with counts and attribute names. Use it to explore a file you do not know. |
|
|
A JSON-ready value: |
|
|
CSV text with a header row. A field is a child element name or |
|
|
A copy with local names only. |
|
|
A readable path, for example |
|
|
The paths where two nodes differ (names, attributes, text, children by position). Whitespace-only text is ignored. |
|
|
A map from each key to its count. |
(: What is in this file? :)
px:outline(px:load("catalog.xml", static-base-uri()))
catalog (1)
product (3) @category @id
name (3)
price (3)
tag (2)
x:note (1)
b (1)
#CSV functions (pc)
The parser follows RFC 4180: a quoted field can hold the separator, line breaks and quotes ("").
Blank lines are skipped. All values are strings; cast them where you need numbers.
|
Function |
Result |
|---|---|
|
|
Each CSV file with a header row, as records (maps from header name to value). |
|
|
Records from CSV text with a header row. A missing value is |
|
|
Each row as an array of strings, with no header handling. |
|
|
The header names, in column order. |
|
|
|
let $orders := pc:load("orders.csv", static-base-uri())
return sum($orders ! xs:decimal(?total))
#Report functions (pr)
|
Function |
Result |
|---|---|
|
|
|
|
|
A text table with aligned columns, for terminal output. A record is a map or an element. |
pr:table(pj:load("a.json", static-base-uri())?items?*, ("name", "price"))
name price
---- -----
pen 2.5
ink 7
#XSLT: JSON and YAML to XML
Both give the same readable XML form:
-
an object member gives an element named by its key; a key that is not an XML name gives
<entry key="…">; -
an array gives one
<item>child for each member; -
a string, number or boolean gives its text; null gives an empty element;
-
member order is kept.
|
Function |
Result |
|---|---|
|
|
JSON text as readable XML, with a root element named |
|
|
The result of |
|
|
YAML text as readable XML. |
|
|
YAML text as JSON text. |
|
|
YAML text as maps and arrays, as |
|
|
YAML text as the W3C XML representation of JSON. |
<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:yx="https://phoenixml.dev/modules/xslt/yaml">
<xsl:import href="https://phoenixml.dev/modules/1.1.0/yaml.xsl"/>
<xsl:template name="xsl:initial-template">
<xsl:sequence select="yx:to-xml(unparsed-text('config.yaml'), 'config')"/>
</xsl:template>
</xsl:stylesheet>
From the command line, json-to-xml.xsl and yaml-to-xml.xsl convert one file:
xslt -it main https://phoenixml.dev/modules/1.1.0/yaml-to-xml.xsl -p href="file://$PWD/config.yaml" -p root=config
href is a URI. A relative href, or a path without file://, resolves against the stylesheet's
location, so give a file:// URI (on Windows, file:///C:/data/config.yaml).
YAML support. One document with block mappings and sequences, plain and quoted scalars, flow
collections ([a, b], {a: 1}), literal (|) and folded (>) block scalars, and comments.
Plain scalars resolve as in the YAML 1.2 core schema (null, booleans, integers including 0x and
0o, floats). Anchors, aliases, tags, complex keys, several documents, and scalars or flow
collections over more than one line raise yx:unsupported. Wrong indentation raises yx:syntax.
Each error message gives the line number.
Run a query file with the CLI:
xquery -f report.xq