@gmod/bgzf-filehandle
Reads block-gzipped (BGZF) files using uncompressed coordinates. @gmod/bam, @gmod/tabix and @gmod/indexedfasta build on it.
- In: a bgzip file and its
.gziindex, or a chunk from a BAI/TBI index - Out: the decompressed bytes
- Inflates with libdeflate compiled to
WebAssembly and inlined as base64, so there is no
.wasmfile to serve. On this repo's fixtures it runs 2-3x faster than pako (benchmarks)
Install
npm install @gmod/bgzf-filehandle
BgzfFilehandle
import { BgzfFilehandle } from '@gmod/bgzf-filehandle'
import { LocalFile } from 'generic-filehandle2'
const f = new BgzfFilehandle({
filehandle: new LocalFile('my_file.gz'),
gziFilehandle: new LocalFile('my_file.gz.gzi'),
})
const data = await f.read(300, 0) // length, position
Create the index with bgzip -i my_file, or bgzip -r my_file.gz for an
existing file. For a remote file, wrap it in
@gmod/range-cache-filehandle.
unzip
Decompresses a BGZF or plain gzip buffer:
import { unzip } from '@gmod/bgzf-filehandle'
const decompressed = await unzip(compressedData)
unzipChunkSlice
Decompresses a chunk {minv, maxv} from a BAI or TBI index, where each end is a
virtual offset {blockPosition, dataPosition}:
import { MAX_BGZF_BLOCK_SIZE, unzipChunkSlice } from '@gmod/bgzf-filehandle'
// read one max-size block past maxv, since nothing records that block's length
const compressedData = await filehandle.read(
maxv.blockPosition + MAX_BGZF_BLOCK_SIZE - minv.blockPosition,
minv.blockPosition,
)
const { buffer, cpositions, dpositions } = await unzipChunkSlice(
compressedData,
{ minv, maxv },
)
cpositions and dpositions hold block boundaries, for building feature IDs
that stay stable across chunks.
A third argument spreads the blocks across Web Workers, which makes a multi-megabyte chunk 1.1-2.0x faster (docs/worker-pool.md):
import { getSharedWorkerPool } from '@gmod/bgzf-filehandle'
const pool = await getSharedWorkerPool() // undefined without Worker support
const result = await unzipChunkSlice(compressedData, { minv, maxv }, pool)
unzipChunkSegments takes the same arguments and skips the final copy into one
buffer, for readers that can parse across segment boundaries
(docs/worker-pool.md).
Docs
- docs/optimizations.md: terms, benchmarks, and
why not
DecompressionStream - docs/worker-pool.md: sizing, lifecycle and measurements
Academic use
Written with NHGRI funding as part of JBrowse. If you use this in a publication, please cite the most recent JBrowse paper at jbrowse.org.
License
MIT © Robert Buels, Colin Diesh