@chonkiejs/core
Core chunking library for Chonkie - lightweight and efficient text chunking with zero dependencies.
Features
✨ Simple & Clean API - Easy to use OOP design
⚡ Zero Dependencies - Minimal, lightweight, fast
🔤 Character-based - Simple tokenization (1 char = 1 token)
🎯 Recursive Chunking - Smart hierarchical text splitting
📦 TypeScript First - Full type safety with TypeScript
Installation
Install with npm:
npm i @chonkiejs/core
Install with pnpm:
pnpm add @chonkiejs/core
Install with yarn:
yarn add @chonkiejs/core
Install with bun:
bun add @chonkiejs/core
Quick Start
import { RecursiveChunker } from '@chonkiejs/core';
// Create a chunker
const chunker = await RecursiveChunker.create({
chunkSize: 512,
minCharactersPerChunk: 24
});
// Chunk your text
const chunks = await chunker.chunk('Your text here...');
// Use the chunks
for (const chunk of chunks) {
console.log(chunk.text);
console.log(`Tokens: ${chunk.tokenCount}`);
}
Available Chunkers
| Name | Description |
|---|---|
RecursiveChunker | Recursively splits text using hierarchical rules (paragraphs → sentences → punctuation → words → characters). Each level only activates if chunks exceed the configured size. |
TokenChunker | Splits text into fixed-size token chunks with optional overlap. Uses character-based tokenization by default, or HuggingFace models with @chonkiejs/token. |
TableChunker | Splits Markdown or HTML tables into smaller table chunks while repeating the original header in each chunk. Supports row-based and token-based modes. |
FastChunker | Uses byte-based boundary detection through @chonkiejs/chunk for very fast chunking with delimiter or pattern controls. |
For detailed API documentation, configuration options, and advanced usage, see DOCS.md.
Contributing
Want to help grow Chonkie? Check out CONTRIBUTING.md to get started! Whether you're fixing bugs, adding features, improving docs, or simply leaving a ⭐️ on the repo, every contribution helps make Chonkie a better CHONK for everyone.
Remember: No contribution is too small for this tiny hippo!
Acknowledgements
Chonkie would like to CHONK its way through a special thanks to all the users and contributors who have helped make this library what it is today! Your feedback, issue reports, and improvements have helped make Chonkie the CHONKIEST it can be.
And of course, special thanks to Moto Moto for endorsing Chonkie with his famous quote:
"I like them big, I like them chonkie in TypeScript" ~ Moto Moto... definitely did not say this
Citation
If you use Chonkie in your research, please cite it as follows:
@software{chonkie2025,
author = {Bhavnick Minhas and Shreyash Nigam},
title = {Chonkie: A no-nonsense fast, lightweight, and efficient text chunking library},
year = {2025},
publisher = {GitHub},
howpublished = {\url{https://github.com/chonkie-inc}},
}