Shell quoting DB

A collection of JSON-encoded configurations for escaping arguments on invoking shells.

License

MIT

About

This package has been created to document and consider different requirements of existing shells for properly escaping arguments to prevent unintended side effects e.g. due to passing functional characters.

All configurations in this package result from testing different ways of escaping every special character in ASCII code range with and without wrapping quotes. The resulting records can be read directly from a JSON file or via some convenience helper method.

To be clear: Quoting arguments in your code does not require to run any tests first. All tests mentioned here have been performed in advance to generate configuration files. Your code is going to simply use those configuration files to properly decide when to use what way of quoting and escaping characters based on an optionally (!) selected shell.

Testing procedure

Tests for assessing a shell are based on two scripts with one of the scripts repeatedly invoking the other one passing some data to be displayed by the latter script. The resulting output is then assessed according to whether data has been received by the second script as intended or not.

The tests cover special characters and whitespace in ASCII code page. They try passing them without any escaping as well as with different common ways of escaping. In addition, they check how quoting an argument is affecting the result and how either tested character works in combination with regular letters.

Several Linux shells have been tested in a Debian-based Docker container. Windows' cmd.exe and powershell.exe have been manually tested on a Windows 11-based device.

Configurations

For every tested shell there is a JSON file describing test results prepared for quoting and escaping arguments when using either shell. Every file's data is divided into separate configurations per character optionally used to wrap an argument. This character is either the empty string for using no enclosing quotes or " or ' for using that character to wrap the argument. A configuration for a quoting character may be missing in case the tested shell has rendered incapable of handling accordingly wrapped arguments during testing.

Either configuration per quoting character consists of these properties:

• mapPattern is a string containing a regular expression matching any character that needs to be escaped.
• map is mapping every character requiring to be escaped into either one's escape sequence.
• rejectPattern is a string with another regular expression matching characters that can't be escaped at all. Arguments matching this pattern should be rejected unless there is a configuration for a different quoting character supporting them.
• invalid is mapping characters that can't be escaped into true for simplified lookups.
• prevent is map primarily for information purposes. It marks characters that should not be escaped with a preceding backslash and/or caret as those combinations will be visible to the invoked script. When using patterns as described before, it is safe to ignore this map, though.

In addition, there is a Javascript module providing a pre-compiled collection of all configurations for simplified integration e.g. with synchronously working code. This collection is limited to information necessary for the quoting process, thus lacking basic meta information per shell and properties invalid and prevent in configurations per quoting character as described above.

Usage

All configurations are provided as dedicated JSON files each referring to one tested shell. You can use them for whatever you intend to do.

const ashConfiguration = await import( "@cepharum/quoting-db/db/ash.json" );

This example requires support for importing JSON modules. Another approach to fetching a shell's configuration is provided by a helper function:

const { getShellConfiguration } = require( "@cepharum/quoting-db" );

const ashConfiguration = await getShellConfiguration( "ash" );

The configuration can be used for quoting arguments on invoking a sub-process via some shell as illustrated in this example:

import { spawn } from "node:child_process";
import { getShellConfiguration, quote } from "@cepharum/quoting-db";

const configuration = await getShellConfiguration();
const args = [ "--opt", "some argument with space", "--filter", "!$%><" ];
const quoted = args.map( arg => quote( arg, configuration ) );

const child = spawn( "some-tool", quoted, {
  shell: true,
} );

Here, helper methods getShellConfiguration() and quote() are used.

• getShellConfiguration() promises the configuration of current system's default shell. You can provide a different shell's name as argument. The resulting configuration can then be used with quote() function.

• quote( rawArgument, configuration ) returns a sanitized version of a provided raw argument based on a shell's given configuration. It may throw in case the provided argument contains characters that can't be escaped at all.

Synchronous processing

If your code requires to work synchronously, fetching a full configuration from dedicated JSON files may not be an option. The library provides a pre-compiled collection of those JSON files as Javascript module covering all information necessary for quoting.

An accordingly modified version of previous example would look like this:

import { spawn } from "node:child_process";
import { getShellConfigurationSync, quote } from "@cepharum/quoting-db";

const configuration = getShellConfigurationSync();

const args = [ "--opt", "some argument with space", "--filter", "!$%><" ];
const quoted = args.map( arg => quote( arg, configuration ) );

const child = spawn( "some-tool", quoted, {
  shell: true,
} );

• getShellConfigurationSync() is the synchronous counterpart to getShellConfiguration(). It returns the configuration of current system's default shell. You can provide a different shell's name as argument. The resulting configuration can be used as second argument to the quote() function, too.

Additional API elements

• getShellName() is a helper function used internally by getShellConfiguration() and getShellConfigurationSync(). It delivers the normalized name of current system's default shell. You can also provide a shell's pathname as used with shell option of Node's spawn() method. In this case, the normalized basename of given shell is returned.

• The compiled collection of reduced configurations per shell is exposed as configurationPerShell.

  import { configurationPerShell, quote } from "@cepharum/quoting-db";
  
  const quoted = quote( "$ome argument <to> e%scape", configurationPerShell.ash );