AzCopy Node

This package allows you to invoke AzCopy v10 from NodeJS.

Basic Usage

const { AzCopyClient } = require("@azure-tools/azcopy-node");
let client = new AzCopyClient();

// AzCopy is used to move things from one location to another.
let src = /* a `AzCopyLocation` */;
let dst = /* a `AzCopyLocation` */;

// All functions that start AzCopy jobs return a job ID.
let jobId = await client.copy(src, dst);

let status;
// You use this job ID to check on the progress of your job, and know if it has finished.
while (!status || status.StatusType !== "EndOfJob") {
    let jobInfo = await copyClient.getJobInfo(jobId)
    status = jobInfo.latestStatus;
    await new Promise((resolve, reject) => setTimeout(resolve, 1000));
}

Where to Find an AzCopy Executable

AzCopy executables are available on NPM for Windows (32 and 64 bit), macOS, and Linux. This package already declares each of these as optional dependencies. The exact packages are:

• @azure-tools/azcopy-win32
• @azure-tools/azcopy-win64
• @azure-tools/azcopy-win-arm64
• @azure-tools/azcopy-linux
• @azure-tools/azcopy-darwin
• @azure-tools/azcopy-darwin-arm64

They are optional dependencies because each package declares itself to only be valid for the OS it is intended for. This means that when you run npm install that only the AzCopy executable for the OS you are currently on will be installed.

If you wish to install all versions you can use the --force flag with npm install.

Specifying What AzCopy Executable to Use

By default, AzCopyClient will use the @azure-tools AzCopy executable package which best matches your OS and architecture. There are two possibilities for using a different executable:

1. Using a different architecture of @azure-tools AzCopy executable package (bit-ness) if available
2. Using a custom AzCopy executable

For the first scenario, if you want to force run either the 32bit for 64bit executable you can specify a bitness in the options object for the AzCopyClient constructor. Since Windows is the only OS to offer both 32 and 64 bit executables, this is really only something to consider for Windows.

For the second scenario you can use the exe field in the options object for the AzCopyClient constructor to specify the path to a custom AzCopy executable. If you do this there are two things to be aware of:

1. This package is verified against a specific version of AzCopy. It is not recommended to use a different version than that. If you decide to do so, make sure to test and verify appropriately.
2. When using a IRemoteAuthLocation, you cannot use the non-NPM versions of AzCopy on Linux. The AzCopy in @azure-tools/azcopy-linux uses a special build of AzCopy.

Azure AD Scenarios

When you use an IRemoteAuthLocation, azcopy-node must share the authentication/refresh token included in that location with the running AzCopy executable. This is done via the OS credential store. It is up to you to implement an ICredentialStore and include it in the options for your AzCopyClient if you wish for this to happen.

new AzCopyClient({
    credentialStore: {
        setEntry: async (service: string, account: string, value: string, exePath?: string, description?: string): Promise<void> => {
            // you implement this
        },
        getEntry: async (service: string, account: string): Promise<string | null> => {
            // you implement this
        },
        deleteEntry: async (service: string, account: string): Promise<boolean> => {
            // you implement this
        }
    }
});

More information can be found in the JSDoc comments for ICredentialStore and TokenRefresher.

chmod

On macOS and Linux you may need to chmod the executables in order for them to run.

Change Log

3.4.15

• Validated against AzCopy 10.32.2.

3.4.14

• Validated against AzCopy 10.32.0.

3.4.13

• Validated against AzCopy 10.31.1.

3.4.12

• Validated against AzCopy 10.31.0.
• Added preserveSymlinks to ICopyOptions for NFS transfers.

3.4.11

• Validated against AzCopy 10.30.1.

3.4.10

• Breaking Change! Renamed preserveSmbInfo to preserveInfo and preserveSmbPermissions to preservePermissions in ICopyOptions to reflect the generalized AzCopy flags (--preserve-info, --preserve-permissions).
• Added "File" to AzCopyTarget for use as a generic file share target alongside "FileSMB" and "FileNFS".
• getFromToOption() now returns equivalent valid options for invalid combinations (e.g., getFromToOption("Blob", "FileSMB") returns "BlobFile").

3.4.9

• Validated against AzCopy 10.30.0.
• Added NFS file share support with new "FileNFS" and "FileSMB" targets in AzCopyTarget.
• Changed FromToOption from a hardcoded union to a computed type derived from AzCopyTarget combinations.
• Added getFromToOption() helper function to compute valid --from-to options from source/destination targets.

3.4.8

• When running on Node.js Windows ARM64, AzCopyClient will now use/prefer the @azure-tools/azcopy-win-arm64 exe over the @azure-tools/azcopy-win64 exe.

3.4.7

• Validated against AzCopy 10.29.1.
• Moved to using SHA256 for sharing tokens with AzCopy.

3.4.2

• Validated against AzCopy 10.26.0.

3.4.1

• Validated against AzCopy 10.25.1.

3.3.0

• Validated against AzCopy 10.24.0.
• Fixed a bug where AzCopy client fail to start a job when HTTPS_PROXY or NO_PROXY environment variables are set.
• When running on Node.js Windows ARM64, AzCopyClient will now use/prefer the @azure-tools/azcopy-win64 exe over the @azure-tools/azcopy-win32 exe.

3.2.0

• Validated against AzCopy 10.23.0.

3.1.0

• Validated against AzCopy 10.22.2.
• Added getCliCommand method on AzCopyClient.

3.0.0

• Added macOS ARM64 AzCopy as a dependency.
• Added trailingDot to ICommonOptions.
• Removed Bitness from IAzCopyClientOptions.
• Updated functions for getting available AzCopy executables.

2.12.0

• Validated against AzCopy 10.18.1
• Add "Cold" as supported block blob tier in ICopyOptions.

2.11.0

• Validated against AzCopy 10.17.0
• Improved comments for listOfFiles and listOfVersions in ICommonOptions.

2.10.1

• Added "FileBlob" and "BlobFile" to FromToOption

2.10.0

• Added preserveBlobTags to ICopyOptions

2.9.0

• Added jobsRemove method on AzCopyClient

2.8.0

• Validated against AzCopy 10.16.2
• Added "FileFile to FromToOption
• Added includeRegex to ICopyOptions
• Added asSubDir to ICopyOptions
• Added several new types for convenience:
  • OverwriteExistingOption for use in ICopyOptions
  • AccessTier for use in ICopyOptions
  • RemoteAzCopyLocation union of all remote locations

2.7.0

• Validated against AzCopy 10.16.0
• Added a deleteJob method to AzCopyClient. Use this function to save memory once a job's info/status is no longer needed.
• Added support for changing bandwidth dynamically. Support includes:
  • Addition of nextPerformanceAdjustmentTime to IJobInfo
  • Addition of adjustCapMbps method on AzCopyClient

2.6.0

• Validated against AzCopy 10.14.1.

2.5.0

• Validated against AzCopy 10.13.0.

2.4.0

• Validated against AzCopy 10.12.2
• Added includeDirectoryStub to ICopyOptions
• Fixed --trusted-microsoft-suffixes being added to commands when trustedDomainSuffixes is an empty array.

2.3.0

• Validated against AzCopy 10.11.0
• Options common to both ICopyOptions and IDeleteOptions were moved a new interface, ICommonOptions, with ICopyOptions and IDeleteOptions each inheriting that interface.
• Added includePath to ICopyOptions and IDeleteOptions.
• Added logLevel to ICopyOptions and IDeleteOptions.
• Added disableAutoDecoding to ICopyOptions.
• Added cacheControl to ICopyOptions.

2.2.0

• Validated against AzCopy 10.10.0.
• Added blockSizeMb to ICopyOptions.
• Added trustedDomainSuffixes to ICopyOptions and IDeleteOptions.

2.1.0

• Validated against AzCopy 10.9.0.

2.0.0

• Validated against AzCopy 10.8.0.
• Breaking Change! This package no longer directly depends on keytar or macos-keychain. If you want to use an IRemoteAuthLocation you must now implement an ICredentialStore and include it in the options for your AzCopyClient. See Azure AD Scenarios and the JSDoc comments in ICredentialStore and TokenRefresher for more information.
• All remote locations now include an optional versionId. This can be used to point at a blob version.
• Added listOfVersions to ICopyOptions and IDeleteOptions. Works similarly to listOfFiles. See the JSDoc comments in either of the options files for more details.
• Added "Archive" as a possible value of the accessTier copy option.
• Added forceIfReadOnly to IDeleteOptions.
• Added tags to ICopyOptions.

1.0.0

• First version of this package!
• Validated against AzCopy 10.5.0.