@banou/media-player

A React video player for files the browser cannot open on its own. It takes a read(offset, size) and a byte length, remuxes into fragmented MP4 as it plays through libav-wasm, and renders ASS/SSA subtitles with jassub. Nothing is downloaded up front, so it plays a 4 GB MKV over HTTP range requests, out of a torrent, off a local disk, or out of anything else that can answer for a byte range.

Playback state runs on video.js v10. None of its skin is used: the chrome here is its own.

Usage

import MediaPlayer from '@banou/media-player'

<MediaPlayer
  read={(offset, size) => Promise<ArrayBuffer>}
  size={fileByteLength}
  publicPath="/"
  libavWorkerUrl="/libav-worker.js"
  jassubWorkerUrl={jassubWorkerUrl}
  jassubWasmUrl={jassubWasmUrl}
  jassubLegacyWasmUrl={jassubLegacyWasmUrl}
  defaultFontUrl={defaultFontUrl}
  cjkFontUrl={cjkFontUrl}
  title="episode.mkv"
  autoplay
/>

read and size travel together: pass both or neither. With neither, the player renders its chrome over a black frame and waits, which is the empty state.

inputToRemuxerInput builds the pair from a Blob/File, a URL (probed for its length over a range request), or your own reader:

import { inputToRemuxerInput } from '@banou/media-player'

const source = await inputToRemuxerInput({ blob: file })
const source = await inputToRemuxerInput({ url: 'https://example.com/episode.mkv' })
const source = await inputToRemuxerInput({ length, read })

A URL's source comes back with readsDownload: true, which the player takes beside read and size: each read of it is a download, so the volume normalizer then analyses the audio playback fetched rather than fetching the file again.

usePlayer reads and drives playback state from anywhere inside a MediaPlayer. It is the only hook the chrome uses: the built-in video.js state and this player's own source state (tracks, thumbnails, indexes, readiness) live on one store, so there is no second context to reach for.

import { usePlayer } from '@banou/media-player'

const paused = usePlayer((state) => state.paused)          // subscribes to that field
const player = usePlayer()                                  // no selector: the store, no subscription
player.play()

useSeekThumbnails and usePictureInPicture are exported for reuse outside the bundled chrome.

useSeekThumbnails returns { thumbnails, requestThumbnail }. Previews are generated from the start of the file to the end, which is right until somebody points at the seekbar: requestThumbnail(time) moves the preview covering time to the front of that queue, and the walk carries on behind it from wherever it had got to. Pass undefined when the pointer leaves. The bundled seekbar calls it for you, so this is only for a chrome you draw yourself. It cannot interrupt a decode already in flight, so the wait is the tail of that one rather than the whole backlog.

downloadedRanges paints byte spans you already hold onto the seekbar, mapped through the keyframe index rather than by percentage, because a file's download progress is not its playback progress: containers carry headers, fonts and attachments that occupy no time at all.

The assets your app has to serve

The workers and the wasm are fetched at runtime from urls you provide, never from anything this package resolves for itself. libav's are plain files to copy out of node_modules, which is what the copy-assets script does. jassub's cannot be copied and have to be built by your bundler; see below. src/asset-urls.ts is the worked example for both.

publicPath is the directory libav's two wasm files are served from, and both have to be there:

libav-wasm picks between them at runtime on typeof WebAssembly.Suspending === 'function', so serving only one does not fail everywhere: it fails on exactly the browsers that pick the missing file, which reads as a browser bug rather than a missing asset. Serve both.

libavWorkerUrl (libav-wasm/build/worker.js) is named individually and still copies fine.

Copy all three from the libav-wasm this package depends on. A worker from before libav-wasm 0.9.0 writes another timeline and reports no timestampOffset, so playback fails at load.

cjkFontUrl is this package's own noto-sans-jp.otf (import cjkFontUrl from '@banou/media-player/noto-sans-jp.otf?no-inline&url'), Noto Sans JP cut to the JIS X 0208 kanji. jassub's libass has no system font fallback, so without it Japanese in a style whose font has no Japanese, which is how releases set song lyrics, draws as empty boxes. It is fetched only once the selected track shows CJK text, after defaultFontUrl, and draws only the glyphs that face lacks.

normalizerWorkletUrl is this package's own dist/volume-normalizer-worklet.js, one plain module with no imports that copies fine or imports as a url; see the volume normalizer.

jassub's assets are imported, not copied

jassub 2's worker is an ES module that imports abslink and lfa-ponyfill by bare specifier plus five relative siblings, and it spawns a nested worker of its own. Copying one file out of node_modules produces something no browser can load, so the bundler has to build that graph. With vite that is one query each, in your own source:

import jassubWorkerUrl from 'jassub/dist/worker/worker.js?worker&url'
import jassubWasmUrl from 'jassub/dist/wasm/jassub-worker-modern.wasm?no-inline&url'
import jassubLegacyWasmUrl from 'jassub/dist/wasm/jassub-worker.wasm?no-inline&url'
import defaultFontUrl from 'jassub/dist/default.woff2?no-inline&url'

?no-inline is load bearing for anything built with build.lib set: vite inlines every asset as a base64 data url in library mode and ignores assetsInlineLimit while doing it, so without it the two wasm builds and the font land in your entry chunk as about 5.9 MB of base64. src/asset-urls.ts in this repo is the same four lines and is meant to be copied.

If you build with build.lib set, build these four in a separate pass with it UNSET. ?no-inline covers the assets you name, and it cannot cover the ones jassub names for itself: its emscripten glue resolves the wasm with new URL('./wasm/...', import.meta.url) inside the worker, and lib mode inlines those too. Measured on this version, one entry importing exactly the four lines above:

assetsInlineLimit: 0 and the function form both make no difference, because lib mode decides to inline before it consults either. A second vite config with a single JS entry, no lib, and the same outDir produces the 110 KB worker, the 30 KB pthread glue and both wasm files as real files.

?no-inline&url needs a type declaration, because vite's own ambient *?url only matches a specifier ENDING in ?url. See src/vite-env.d.ts.

The two wasm builds are picked at runtime:

The test is RELAXED simd, not baseline simd: jassub validates a module using i8x16.relaxed_swizzle. That is a much later feature than plain SIMD, so the second file is not a museum piece for ancient Safari the way it was under jassub 1. Serve both.

Serve the wasm as application/wasm: jassub instantiates it with instantiateStreaming, which refuses any other content type.

No cross-origin isolation is needed. jassub allocates a shared WebAssembly.Memory unconditionally, which constructs on a plain origin in both engines (measured on Chrome 152 and Firefox 154), and its own thread count is gated on crossOriginIsolated, so without COOP and COEP it simply runs single threaded.

The chrome brings its own scale and needs nothing from the host page's root font. Everything is sized against --mp-unit, which defaults to 10px on the player element; set it there to rescale the whole chrome at once.

What it does

Play and pause, seek with a preview thumbnail and a keyframe-accurate scrub, chapters on the seekbar, an offer to skip an opening or ending, volume on a log curve, mute, playback speed, audio track selection, subtitle track selection, picture in picture, fullscreen, and keyboard shortcuts. Nothing is persisted: volume and speed start at their defaults every load, and tracks at the preferred languages you pass, or the file's defaults.

The shortcuts (Space or k, f, m, the arrow keys, and < and > to pause and step one frame back or forward) listen on the whole page, and leave Ctrl, Cmd and Alt combinations to the browser. The frame steps need a media with requestVideoFrameCallback, which a media playing in another document does not have; its host can pass stepFrame to make the step inside that document instead. Pass shortcuts="player" for a player among other content: a key then acts only from inside the player (a click on the picture puts the focus there) or while it is fullscreen, and Space and the arrows keep scrolling the page everywhere else.

Chapters

Chapters come from the container, and the seekbar draws them as segments with a break at each boundary. Hovering one lifts it above its neighbours and names it beside the time in the preview.

Pass chapters to supply them yourself, on either arm, and they win over whatever the file declared:

<MediaPlayer {...source} chapters={[{ start: 0, end: 88, title: 'Opening' }]} />

They are { start, end, title } in seconds, ordered and non-overlapping, and they need not cover the whole duration. A local file gets them from libav automatically, so this is for a source that knows chapters the container does not.

Skip Opening and Skip Ending

When a chapter looks like an opening or an ending, a button offers to jump past it, appearing a second before the chapter and staying six seconds. It never skips on its own, so a wrong guess costs a button nobody presses rather than a jump out of the episode.

The guess is made from the chapter title, against the names releases actually use (OP, Opening, ED, Ending, Credits, and the same names carrying a song), and it defers across the whole file: Intro is the opening where nothing else claims to be, and the cold open where an OP follows it. A file whose chapters carry no usable name at all falls back to shape, where a chapter of about ninety seconds in each half of the runtime is the opening and the ending. Nothing shorter than fifteen seconds is ever offered, and a disc that is mostly themes offers nothing, because there the themes are what is being watched.

The same guess is published on its own, for a page that has to know what a chapter is without loading the player: classifyChapters(chapters) from @banou/media-player/chapters, which imports nothing else of the player, gives 'opening', 'ending' or undefined for each chapter.

Picture in picture keeps the subtitles

Subtitles are painted by jassub onto a canvas over the video, and picture in picture takes a video element and nothing else, so the browser has no way to composite the two: a plain requestPictureInPicture() puts the bare video in the window and leaves the subtitles on the page.

That canvas belongs to jassub from version 2 on: the constructor transfers it to a worker, which an element accepts exactly once, and destroy() removes it from the document. So the player renders a subtitle LAYER and the engine mounts one canvas inside it per pipeline build. Anything reaching for the surface has to look it up through the layer each time rather than hold it.

So the player composites them itself. Every presented frame is drawn to an offscreen canvas with the subtitle canvas on top, and captureStream() turns that into a MediaStream backing a hidden video element, which is the one that enters the window. The original element keeps playing and stays the only audio source, since a canvas stream carries no audio track.

The mirror element is never paused. A paused video stops rendering its MediaStream, so pausing it to reflect the real element froze the window: seeking while paused left the old scene on screen. The transport state is carried by the Media Session instead, which is what the window reads for its play/pause button.

The window outlives the player. A player unmounted while its window is open leaves it open for a second, and the next player mounted in the same document in that time plays in it, at the shape of its own video. So an app that mounts a new player for the next episode keeps the window without a click, which a new window would need. If no player takes the window in that second, it closes.

Where that path is unavailable the player offers no picture in picture control.

Picture in picture for a media you do not own

A browser opens picture in picture only for a click in the document that owns the <video>, so a remote media gets the control only when the host opts in to taking that click:

// `.frame` sets `pointer-events: none`, so clicks reach the chrome everywhere else
<MediaPlayer
  media={media}
  pictureInPicture={{ onArmedChange: (armed) => { frame.current!.style.pointerEvents = armed ? 'auto' : '' } }}
>
  <iframe ref={frame} className='frame' />
</MediaPlayer>

While the pointer is over the control, the chrome lets a click through at the control's box and the host is told it is armed, so its frame takes that click and enters picture in picture from inside. The control follows the media's enterpictureinpicture and leavepictureinpicture events, and in picture in picture a click on it calls the media's exitPictureInPicture.

A finger never rests on the control before it presses, so a tap cannot arm it. The control is offered only where the primary pointer hovers ((hover: hover) and (pointer: fine)), and a touch hides it until a pointer that hovers moves again.

Fullscreen kept by the host page

A page that fullscreens an element of its own around the player, so that fullscreen outlasts the player in it, passes fullscreen:

<div ref={stage}>
  <MediaPlayer
    {...source}
    fullscreen={{
      active: stageIsFullscreen,
      request: () => document.fullscreenElement ? document.exitFullscreen() : stage.current!.requestFullscreen(),
    }}
  />
</div>

The player then calls no requestFullscreen. The button, f and a double click each call request once, during the press's own event, so the page's requestFullscreen has the press's activation, and the button and shortcuts="player" follow active, which the page keeps up to date from its fullscreenchange. Without the prop the player fullscreens itself. A player in another document hands its fullscreen over through <MediaPlayer expose>, to a mirror that asks for it.

The volume normalizer

A "Normalize volume" switch in the settings menu keeps the file's own normal level and turns down what goes well over it, so a sudden blast comes out near the dialogue and the dialogue is left as it was. It is offered once you pass the worklet's url, and it is on unless the viewer turns it off or you pass normalizeVolume={false}:

import normalizerWorkletUrl from '@banou/media-player/volume-normalizer-worklet.js?no-inline&url'

<MediaPlayer
  {...source}
  normalizerWorkletUrl={normalizerWorkletUrl}
  normalizeVolume={saved ?? true}
  onNormalizeVolumeChange={save}
/>

The player saves nothing, so an app that wants the choice to stick saves what onNormalizeVolumeChange reports and passes it back.

It works in two stages. A worker reads the file a second time, through thumbnailRead when you pass one and only within downloadedRanges (for a source that readsDownload it reads nothing and takes the audio playback fetched, about 30 s ahead of the playhead), decodes its audio with WebCodecs, learns the file's normal level (the median 400 ms loudness) and computes a cut with unlimited lookahead, so a cut starts before a spike: every 10 ms of the output is held to 10 LU over the normal level and every 50 ms to 8 LU. It analyses what is near the playhead first and then the rest of the file, keeping only 1 ms loudness readings, never the audio. Wherever that analysis has not reached the playhead, and for a track this browser's WebCodecs cannot decode, an AudioWorklet plays a live stage: a 100 ms detector that cuts what goes 5 LU over the level heard so far, and the 10 ms loudness held to 10 LU over it, seen 10 ms ahead. Neither stage ever boosts. The sound plays 10 ms behind the picture, on or off, once routed.

The url names one file that is both the worklet and the worker's script, so it has to be a script your origin serves: a page whose CSP says script-src 'self' refuses a worklet or a worker from a blob: or data: url, and build.lib inlines assets as data: without ?no-inline. A worker must also be same-origin, so serve it from the page's own origin.

The first time it is on, the <video> is routed through Web Audio for the rest of its life, because an element cannot be handed back; off is a bypass. Routing waits until the AudioContext runs: one that autoplay rules hold back resumes on the page's next click, tap or key press, and the video plays as it always did until then. WebKit is refused (createMediaElementSource in @banou/ponyfill/web-audio), since a routed element loses its volume and mute there. The refusal comes when routing would, once the switch is on and the AudioContext runs: until then the switch reads as available, and from then on it reads Unavailable, with the element never routed and playing as it always did.

Preferred languages

Pass the languages to start a file in, and save what the viewer picks:

<MediaPlayer
  {...source}
  preferredAudioLanguage={saved.audio}
  preferredSubtitleLanguage={saved.subtitles}
  onPreferredAudioLanguageChange={(audio) => save({ ...saved, audio })}
  onPreferredSubtitleLanguageChange={(subtitles) => save({ ...saved, subtitles })}
/>

A preference is a tag or a name (ja, ja-JP, jpn, Japanese, English (US)), and a pick is reported the way the track names its language, its tag or else its title, or null for subtitles off. A file starts on its full audio track in the preferred language, else on its default: a commentary or an audio description in it does not count. Subtitles start on the full track in the language, else on a signs and songs one in it, else on the default. What the viewer picks wins for that file. A subtitle track that names no language is reported by its title, and only when it turns subtitles on, so the next file starts with them on, on the track of that title if it has one.

To start in the viewer's own language, pass it as both preferences with alwaysShowSubtitles={false}: a file with audio in it plays that audio with subtitles off, and any other plays its original audio with subtitles in it. Subtitles are left off only where the audio plays in their language, so a picked audio language other than theirs keeps them on. The original is the file's track in originalLanguage when the source knows it (ja for anime), else the file's default, which is its first track: on a dual audio file whose first track is a dub, only originalLanguage finds the original. onAlwaysShowSubtitlesChange reports true when the viewer turns on subtitles in the language the audio plays in, for the app to pass back as alwaysShowSubtitles. The rule is startingTracks(audio, subtitles, preferences), from @banou/media-player/language.

A media the player does not own reports picks the same way, by the option's label once the source's select resolves, and its host applies the preferences to its own tracks: preferredSelection(kind, selections, preferences), from @banou/media-player/language, names the option to select by the same rule, the audio selected standing for the source's default, and imports nothing else of the player. languageTag(pick) from the same module gives a pick's BCP 47 tag (jpn gives ja, Português (Brasil) gives pt-BR), or undefined for one naming no language, for code that reads tags.

Labels in the viewer's language

The controls, tooltips, menus, skip offers and failures are in English. Pass language, a BCP 47 tag such as the host page shows, for another:

<MediaPlayer {...source} language="fr" />

They are shown in the closest language the player has: Latin American Spanish (es-419, for every es-*), Brazilian Portuguese (pt-BR, for every pt-*), French, German, Italian, Russian, Japanese, Korean, Simplified and Traditional Chinese (zh-Hans, zh-Hant), Turkish, Polish, Arabic, Hebrew, Indonesian, Vietnamese, Thai, Hindi, Ukrainian, Dutch, Swedish, Czech, Greek, Romanian, Hungarian and Malay, else English. The numbers in them are written as the tag writes them (es-AR gives 0,5x, es-419 itself 0.5x), and track languages are named in it, even when the labels are English (en-DE gives 0,5x). A language's text is fetched the first time a player shows it, with FormatJS (@formatjs/intl), and the labels are English until it has arrived. A player given no language, or one it does not have, fetches neither. The bar and the seek bar stay left to right in every language: a host page written right to left puts dir="ltr" on the player's parent.

Each language is src/lib/locales/<tag>.json, keyed by the English of src/lib/labels.ts; a unit test holds every file to FormatJS's verify against that English.

Layout

One package. src/ is the demo app, src/lib is the library it publishes, and the app imports it by its published name so it stays an honest consumer.

• src/ the app: main.tsx, routes/home.tsx. Built by vite.config.ts into build/.

• src/lib/engine/ the pipeline, with no React in it: MediaSource feeding, remux, jassub, thumbnails. Published as @banou/media-player/engine.

• src/lib/remote is the player driven from ANOTHER document, over osra. Published as @banou/media-player/remote.

  In the document that renders the player, one prop serves it to whoever frames that document, and only to them, or to a named origin:

  <MediaPlayer expose read={read} size={size} publicPath="/" {...workers} />
  <MediaPlayer expose={{ origin: 'https://anime.fkn.app' }} media={media} />
  

  A document that serves more than one player gives each an id, and an embedder asks for the one it wants:

  <MediaPlayer expose={{ id: 'left' }} {...left} />
  <MediaPlayer expose={{ id: 'right' }} {...right} />
  

  const left = mediaPlayer(iframe, { id: 'left', origin })
  const right = mediaPlayer(iframe, { id: 'right', origin })
  

  Both sides default to one unnamed player, so a document with one never says it. Whatever the count, it is ONE connection per document in each direction: the ids multiplex over it, serving an id twice replaces that player (which is how a source switch is followed), and asking for an id nobody serves yet simply waits until somebody does.

  In the embedder, mediaPlayer hands back a media of its own, the same PlayerMedia shape the player drives, so it reads, moves, listens and can even be handed to a second <MediaPlayer>:

  import { mediaPlayer } from '@banou/media-player/remote'
  
  const player = mediaPlayer(iframe, { origin: 'https://torrent.fkn.app' })
  await player.ready
  await player.play()
  player.currentTime = 30
  player.addEventListener('seeked', () => console.log(player.currentTime))
  player.destroy()
  

  play() settles with the far element's own answer and rejects if the mirror is destroyed while it is in flight, the way an element's rejects on an interrupted play, so await it or catch it; pause() and load() swallow theirs.

  Reads are synchronous because the far side is mirrored: every event over there arrives with a snapshot, and a write moves the mirror at once before going out to be applied. play() settles with the far element's own answer, so an autoplay refusal there rejects here. ready stays pending while nobody answers and rejects on destroy() or an aborted signal; a MessagePort whose peer has gone raises no event, so nothing can tell a dead transport from a slow one and the wait has to be bounded rather than waited out. Use signal for the player's whole lifetime, or Promise.race([player.ready, timeout]) for the wait alone. A player that switches sources, or a player document that reloads, is followed: the mirror reports emptied and then the new state, the way an element would. Hand a remote player to a second <MediaPlayer media={player}> after ready. A document served with no frame around it serves nobody; exposePlayer(media) does the same without React.

  A <MediaPlayer expose> also carries its chapters, as player.chapters with a chapterschange event after each change (a bare element has none), and hands its fullscreen to a mirror made with fullscreen: true: the far player's button, f and double click then fire one fullscreenrequest on the mirror each, within the press's activation, and show player.fullscreen as this page sets it, until destroy() gives the fullscreen back.

  const player = mediaPlayer(iframe, { origin, fullscreen: true })
  player.addEventListener('fullscreenrequest', () => document.fullscreenElement ? document.exitFullscreen() : stage.requestFullscreen())
  document.addEventListener('fullscreenchange', () => { player.fullscreen = !!document.fullscreenElement })
  player.addEventListener('chapterschange', () => console.log(player.chapters))
  

  A page with something of its own in the corner of the far player's skip offer sets player.endingOffered = false to take the offer of the last ending away. The read is the far player's own word, from after its render, with a chapterschange event following each change: wait for the false before drawing over that corner, and a player that cannot take it (a bare element, an older version) stays true.

• src/lib/react/ the player component, its chrome, and the hooks.

• src/lib/labels.ts the chrome's text in English, and src/lib/locales/ its other languages.

Built by vite.lib.config.ts into dist/, which is what npm publishes.

Development

npm install
npm run dev        # the demo app on port 4560
npm run build      # the app, into build/
npm run build-lib  # the library, into dist/

The app opens on an empty player: black, with the chrome and nothing else. Drop a file anywhere, click to pick one, or paste a URL.