node-exist

Mostly a shallow wrapper for eXist's XML-RPC API and eXist's REST API. Attempts to translate terminologies into node world. Uses promises.

• Install
• Use
• Connection Options
• Components
• Command Line Scripts
• Test
• Roadmap
• Compatibility
• Disclaimer

Install

npm install @existdb/node-exist

NOTE: If you are looking for a command line client have a look at xst

Use

In addition to eXist-db's XML-RPC API you do now also have the option to leverage the potential of its REST-API. This allows to choose the best tool for any particular task. Both APIs are used in combination in the upload example.

NOTE: This package is now ESM only.

REST

Status: unstable

NOTE: eXist-db's REST-API has its on methodology and available options may differ between instances. Especially, the ability to download the source code of XQuery files is prohibited by default (_source=yes is only availabe if enabled in descriptor.xml). For details of available options for each method please see the REST-API documentation

First, we need an instance of the restClient.

import { getRestClient } from '@existdb/node-exist'
const rc = getRestClient()

For more information see also Connection Options.

The returned HTTP client has 4 methods

• post(query, path[, options]): execute query in the context of path. The query is expected to be a XQuery main module and will be wrapped in the XML-fragment that exist expects.

  const { bodyText } = await rc.post('count(//p)', '/db')
  console.log(bodyText)
  

• put(data, path): create resources in the database. If sub-collections in path are missing, they will be created for you. The server will respond with StatusCode 400, Bad Request, for not-well- formed XML resources. In this case, the body contains a detailed description of the validation error.

  await rc.put('<root />', '/db/rest-test/test.xml')
  

• get(path [, options][, writableStream]): read data from the database. The response body will contain the contents of the resource or a file listing, if the provided path is a collection. If a writableStream is passed in, the response body will be streamed into it.

  const { bodyText } = await rc.get('/db/rest-test/test.xml')
  console.log(bodyText)
  

• del(path): remove resources and collections from an existdb instance

  await rc.del('/db/rest-test/test.xml')
  

Have a look at the rest-client example. The REST-client uses the Got library and works with streams and generators. Look at the rest tests to see examples.

XML-RPC

Creating, reading and removing a collection:

import { getXmlRpcClient } from '@existdb/node-exist'
const db = getXmlRpcClient()

db.collections.create('/db/apps/test')
  .then(result => db.collections.describe('/db/apps/test'))
  .then(result => console.log('collection description:', result))
  .catch(e => console.error('fail', e))

Uploading an XML file into the database

import { getXmlRpcClient } from '@existdb/node-exist'
const db = getXmlRpcClient()

db.documents.upload(Buffer.from('<root/>'))
  .then(fileHandle => db.documents.parseLocal(fileHandle, '/db/apps/test/file.xml'))
  .then(result => db.documents.read('/db/apps/test/file.xml'))
  .then(result => console.log('test file contents', result))
  .catch(error => console.error('fail', error))

Since all interactions with the database are promises you can also use async functions

import { getXmlRpcClient } from '@existdb/node-exist'
const db = getXmlRpcClient()

async function uploadAndParse (filePath, contents) {
  const fileHandle = await db.documents.upload(contents)
  await db.documents.parseLocal(fileHandle, filePath, {})
  return filePath
}

try {
  const filePath = await uploadAndParse('/db/apps/test-file.xml', Buffer.from('<root/>'))
  console.log("uploaded", filePath)
} catch (error) {
  console.error(error)
}

You can also have a look at the examples for more use-cases.

Connection Options

In the previous section you learned that there are two APIs you can use to interact with an exist-db instance.

Both client constructor functions do accept an option argument of type NodeExistConnectionOptions. Calling them without arguments, as in the examples above will use default options.

{
  basic_auth: {
    user: 'guest',
    pass: 'guest'
  },
  protocol: 'https:',
  host: 'localhost',
  port: '8443',
  path: '/exist/rest'|'/exist/xmlrpc'
}

NOTE: The path property, the endpoint to reach an API, is different for REST ('/exist/rest') and XML-RPC ('/exist/xmlrpc'). You most likely do not need to change it. However, if you need to you can override those.

RESTClient with defaults

import { getRestClient } from '@existdb/node-exist'
const rest = getRestClient({
  basic_auth: {
    user: 'guest',
    pass: 'guest'
  },
  protocol: 'https:',
  host: 'localhost',
  port: '8443',
  path: '/exist/rest'
})

XmlRpcClient with defaults

import {getXmlRpcClient} from '@existdb/node-exist'
const db = getXmlRpcClient({
  basic_auth: {
    user: 'guest',
    pass: 'guest'
  },
  protocol: 'https:',
  host: 'localhost',
  port: '8443',
  path: '/exist/xmlrpc'
})

Examples

• Connect as someone else than guest

  {
    basic_auth: {
      user: 'me',
      pass: '1 troubadour artisanal #compost'
    }
  }
  

• Connect to a local development server using HTTP

  {
    protocol: 'http:',
    port: 8080
  }
  

• Connect to a server with an invalid or expired certificate.

  {
    host: 'here.be.dragons',
    rejectUnauthorized: false
  }
  

  NOTE: For remote hosts this is considered bad practice as it does only offer a false sense of security. For hosts considered local - localhost, 127.0.0.1 and [::1] - this is set automatically, because it is impossible to have trusted certificates for local hosts.

• Give up on requests that take longer than ten minutes

  {
    timeout: 600000
  }
  

  timeout is in milliseconds and applies both to waiting for the response headers and to the pause between two chunks of the response body. The default, 0, waits indefinitely, so long running queries are not cut off by the client. Queries and REST post accept a timeout in their options that overrides it for a single call. Package operations (db.app.*) never time out.

Read options from environment

readOptionsFromEnv offers a comfortable way to read the connection options from a set of environment variables

NOTE: In order to connect to an instance as a user other than guest both EXISTDB_USER and EXISTDB_PASS have to be set!

import {getXmlRpcClient, getRestClient, readOptionsFromEnv} from '@existdb/node-exist'
const db = getXmlRpcClient(readOptionsFromEnv())
const rest = getRestClient(readOptionsFromEnv())

For more details you can have a look how it is used in the connection script that is used for testing and in all example scripts.

Components

The XML-RPC commands are grouped into components by what they operate on. Every method returns a promise.

Queries

Status: working

execute, read and readAll accept a timeout in options, in milliseconds (0 waits indefinitely). It overrides the connection timeout for this query only and is not sent to the database.

db.queries.readAll(query, { variables, timeout: 0 })

execute

db.queries.execute(query, options)

read

db.queries.read(query, options)

readAll

This convenience function calls queries.count then retrieves all result pages and returns them in an array.

db.queries.readAll(query, options)

Example:

const query = `xquery version "3.1";
xmldb:get-child-collections($collection)
  => string-join(",\n")
`
const options = { variables: collection: "/db/apps" }

db.queries.readAll(query, options)
  .then(result => {
    const response = Buffer.concat(result.pages).toString() 
    console.log(response)
  })
  .catch(error => console.error(error))

count

db.queries.count(resultHandle)

retrieve

db.queries.retrieveResult(resultHandle, page)

retrieveAll

db.queries.retrieveAll(resultHandle)

releaseResult

free result on server

db.queries.releaseResult(resultHandle)

Documents

A document can be seen as a file. It might be indexed if it's type is not binary.

upload

Resolves into a file handle which can then be used by db.documents.parseLocal.

db.documents.upload(Buffer.from('test'))

parseLocal

db.documents.parseLocal(fileHandle, 'foo/test.txt')

read

Reads resources stored as XML (XMLResource). You can control how they are serialized by setting serialization options in the options parameter.

Use default serialization options.

db.documents.read('foo.xml')

Force XML declaration to be returned.

db.documents.read('foo.xml', { "omit-xml-declaration": "no" })

Force the file to end in a blank line (available since eXist-db v6.0.1).

db.documents.read('foo.xml', { "insert-final-newline": "yes" })

readBinary

Reads resources stored as binary (BinaryResource) inside existdb such as XQuery, textfiles, PDFs, CSS, images and the like.

db.documents.readBinary('foo.xml')

remove

db.documents.remove('foo.xml')

Resources

Status: working

A resource is identified by its path in the database. Documents and collections are resources.

describe

db.resources.describe(resourcePath)

setPermissions

db.resources.setPermissions(resourcePath, 400)

getPermissions

db.resources.getPermissions(resourcePath)

Collections

Status: working

create

db.collections.create(collectionPath)

remove

db.collections.remove(collectionPath)

describe

db.collections.describe(collectionPath)

read

db.collections.read(collectionPath)

exists

This function checks if the collection exists.

• returns true if the collection exists and the current user can open it
• returns false if the collection does not exist
• throws an exception if the collection exists but the current user cannot access it

db.collections.exists(collectionPath)

existsAndCanOpen

This function checks if the collection exists and if it does, if the current user can access it.

• returns true if the collection exists and the current user can open it
• returns false if the collection does not exist
• throws an exception if the collection exists but the current user cannot access it

db.collections.existsAndCanOpen(collectionPath)

App

Status: working

upload

After uploading a XAR you can install it

db.app.upload(xarBuffer, xarName)

Example:

const xarContents = fs.readFileSync('spec/files/test-app.xar')

db.app.upload(xarContents, 'test-app.xar')
  .then(result => console.log(result))
  .catch(error => console.error(error))

install

Install an uploaded XAR (this will call repo:install-and-deploy-from-db). For extra safety a previously installed version will be removed before installing the new version.

Dependencies will be resolved from http://exist-db.org/exist/apps/public-repo by default. If you want to use a different repository provide the optional customPackageRepoUrl.

db.app.install(xarName[, customPackageRepoUrl])

Example:

db.app.install('test-app.xar')
  .then(result => console.log(result))
  .catch(error => console.error(error))

Returns

{
  "success": true,
  "result": {
    "update": false, // true if a previous version was found
    "target": "/db/apps/test-app"
  }
}

Error

{
  success: false,
  error: Error
}

remove

Uninstall and remove the application identified by its namespace URL. If no app with packageUri could be found then this counts as success.

db.app.remove(packageUri)

Example:

db.app.remove('http://exist-db.org/apps/test-app')
  .then(result => console.log(result))
  .catch(error => console.error(error))

Returns

{ success: true }

Error

{
  success: false,
  error: Object | Error
}

packageCollection

The path to the collection where node-exist will upload packages to (/db/pkgtmp). Useful for cleanup after succesful installation.

Example:

db.documents.remove(`${db.app.packageCollection}/test-app.xar`)

Indices

Status: TODO

Users

Status: working

getUserInfo

Will return the information about the given user.

db.users.getUserInfo(username)

Example:

db.users.getUserInfo('admin')

Returns:

{
    uid: 1048574,
    'default-group-id': 1048575,
    umask: 18,
    metadata: {
      'http://exist-db.org/security/description': 'System Administrator',
      'http://axschema.org/namePerson': 'admin'
    },
    'default-group-name': 'dba',
    'default-group-realmId': 'exist',
    name: 'admin',
    groups: [ 'dba' ],
    enabled: 'true'
}

list

db.users.list()

Returns an array of user info objects (see getUserInfo()).

server

Status: working

version

Query the eXist-db version running on the server. Returns the SemVer version as a string (e.g. 5.4.1 or 6.1.0-SNAPSHOT).

db.server.version()

syncToDisk

db.server.syncToDisk()

shutdown

db.server.shutdown()

Note: There is no way to bring it up again.

Command Line Scripts

You can use this library to build a command line interface that interacts with existdb instances. A few basic examples how to do this are included in this repository.

Example:

spec/examples/exist-ls /db/apps

NOTE: Have a look at xst for a CLI client built with node-exist.

Test

All tests are in spec/tests and written for tape

npm test

NOTE: You can override connection settings with environment variables. See examples for more information.

To execute a single run using a different server you can also just define the variable directly:

EXISTDB_SERVER=http://localhost:8888 npm test

Roadmap

• switch to use eXist-db's REST-API (available through rest-client)
• refactor to ES6 modules
• better type hints

Compatibility

node-exist is tested to be compatible with eXist-db 4, 5 and 6. It should be compatible with version 3, except for the XAR installation.

Disclaimer

Use at your own risk.

This software is safe for development. It may be used to work with a production instance, but think twice before your data is lost.