@animated-waffle/react

React hook and Avatar renderer for embedding a published Narya talking Agent. This package depends on and re-exports the whole of @animated-waffle/client, so React applications install only this package.

Install the current npm release without copying a mutable registry version:

pnpm add @animated-waffle/react@latest

The managed-session identity and sessionContext APIs described here were introduced in version 0.3.1. The workspace package.json is the source of truth for this repository's version; @latest resolves the current npm release.

The Agent owns its avatar, voice, and prompt. Media runtime, API routes, and avatar asset URLs are SDK implementation details. The renderer loads the Avatar bound to the Agent session; callers do not pass an avatarUrl. Production is the default API origin. Tests and non-production integrations may pass apiOrigin; the SDK still owns the Agent session route.

Use

Keep the permanent workspace API key on your server. Exchange it for a short-lived Agent session token and give the hook a callback that obtains one for each connection:

import {
  AvatarStage,
  WaffleDeveloperFeedback,
  WaffleDiagnosticsPanel,
  useWaffle,
} from '@animated-waffle/react'

const waffle = useWaffle({
  agentId,
  inputAudioEnabled: false, // Text-triggered, output-only application.
  sessionContext: {
    userName: ' Space é ',
    timeZone: 'Asia/Shanghai',
  },
  sessionToken: () => getSessionToken(agentId),
  // Optional creator-only dev BFF. It returns the public snake_case grant.
  developerFeedback: ({ agentId, sourceSessionId, signal }) =>
    fetch('/api/narya-developer-feedback-grant', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ agentId, sourceSessionId }),
      signal,
    }),
})

return (
  <>
    <AvatarStage {...waffle.avatar} style={{ width: 480, height: 480 }} />
    <WaffleDeveloperFeedback waffle={waffle} />
    <WaffleDiagnosticsPanel waffle={waffle} />
  </>
)

With the callback, the feedback component authorizes saves only after a connected Session exists. Without it, the server-authored Studio link remains available for continuing there. Each save appends to a private draft round and can distinguish Preserve/Change, whole Session/persisted Assistant reply, preferred reply, and Persona/behavior/Memory/Avatar/Voice/runtime dimensions. It never starts prompt generation or publication; the server-authored Dashboard link continues that workflow. The component keeps one item UUID across an ambiguous failed retry.

sessionContext?: { userName?: string; timeZone?: string } belongs only to the default sessionToken starter. Nonblank values are sent exactly as provided and blank values are omitted. It is mutually exclusive with the advanced startSession callback, so token-path consumers do not recreate the raw managed-session request.

The hook returns the app-facing client surface (status, session, transcript, transcriptProgress, agentSpeaking, decline, error, connect, disconnect, sendText, muted / setMuted, and the push-to-talk controls) plus the opaque avatar binding for AvatarStage and the avatarLoad projection. Transport audio tracks are not part of this default React API. Its action functions keep stable identity across renders, so effects and event subscriptions can depend on them directly.

error is a WaffleError with a typed stage. The hook is the single React error owner: connection failures retain their client stage, while Avatar model and renderer failures use avatar-load and avatar-renderer without changing the connection state. The hook also accepts the client's optional reportedSource; it defaults to SDK.

AvatarStage owns the renderer lifetime. Session disconnects, reconnects, and replacements only rebind performance signals; they do not remove the visible canvas. Model replacement reuses the stage, so the previous avatar stays visible until the replacement model is ready. The stage is destroyed only when its host component unmounts.

waffle.avatarLoad reports the Agent avatar's loading state:

const label = waffle.avatarLoad.fromCache
  ? 'Loading cached avatar…'
  : waffle.avatarLoad.progress === null
    ? 'Downloading avatar…'
    : `Downloading avatar… ${Math.round(waffle.avatarLoad.progress * 100)}%`

The first VRM download is streamed with byte progress. The SDK verifies the model against the content hash supplied by the Agent session and stores it in browser Cache Storage. Later sessions reuse that content even though their short-lived asset URL is different.

setMuted silences the Agent's voice while it keeps performing. Taking playback over entirely is a @animated-waffle/client option: this hook hands back no audio track, so there would be nothing for the application to play.

For an Agent with deliberate silence enabled, decline contains the latest { reason }. Use it to settle the current serialized reaction.

The Avatar's lip sync runs an audio worklet the package carries in its bundle and hands to the browser as an object URL, so no build step has to copy assets and no bundler can lose them. An application with a Content-Security-Policy needs blob: in script-src.

See the SDK documentation for the complete integration guide.

Development diagnostics

useWaffle automatically shows a collapsed Aniwaffle · Diagnostics button at the bottom right in development builds. Click it to expand; click again or press Escape inside it to collapse. Production builds hide it. Set diagnostics: false or true in hook options to override. The default follows the app bundler's process.env.NODE_ENV.

Read client.diagnostics and subscribe with client.on('diagnostics', render). React returns the same snapshot as waffle.diagnostics; render <WaffleDiagnosticsPanel waffle={waffle} /> from @animated-waffle/react for an inline view, with diagnostics: false to avoid also showing the overlay. Other browser apps use mountWaffleDiagnostics(host, client) from @animated-waffle/web. Mounting a panel never starts a connection or microphone.

The panel shows connection/media/Avatar state, connection phases, latest client input-end-to-playback estimate, safe errors and the current Session Studio link. Updated workers additionally report ASR, LLM, TTS and worker-output timings using Studio's reducer. Client and server rows have separate correlation IDs and clocks; do not sum the stages or treat a worker output as browser playback. Missing data remains unavailable. Voice input activity and RTC/paired output timing are estimates, not calibrated acoustic measurements. Snapshots exclude text, audio, tokens and raw error messages. The panel is optional; existing timing telemetry is independent of it.

Microphone meter and device selector

import { useWaffle, WaffleMicrophoneControls, WaffleInputMeter } from '@animated-waffle/react'

const waffle = useWaffle({ agentId, sessionToken })
// Includes the current input name, live meter, in-call device switch and input notices.
<WaffleMicrophoneControls waffle={waffle} />
// Or render your own controls around these primitives:
<WaffleInputMeter level={waffle.inputLevel} />
// waffle.microphone.devices, waffle.microphone.deviceId, waffle.setInputDevice(id)

These components do not request capture on mount. Connect first to obtain device labels and live energy. The meter reports microphone energy, not ASR or proof that the user is speaking. waffle.inputIssue contains nonfatal input feedback separately from connection errors. Device-switch errors stay beside the selector.

Region

Use useWaffle({ agentId, sessionToken, region: 'jp' }) to select the Japan API and Agent, or region: 'us' for the US. connect({ region: 'jp' }) overrides the default for a new connection. For DEV, also set apiOrigin: 'https://dev.aniwaffle.com'; custom API origins stay unchanged. See the region guide.