Indic Transliterate

Transliteration component for React with support for 21 Indic languages. Uses API from AI4Bharat IndicXlit.

Table of contents

• 1. About
• 2. Demo
• 3. Install
• 4. Usage
  • 4.1. Basic example
  • 4.2. With custom component
  • 4.3. With TypeScript
  • 4.4. With Material-UI
  • 4.5 With External API for transliteration
• 5. Get transliteration suggestions
  • 5.1. Standalone Example
  • 5.2. Custom trigger keys
  • 5.3. Props
• 6. Languages
  • 6.1. Get supported languages
  • 6.2. List of language codes
• 7. New Feature: Caching for Frequently Used Words
• 8. License

1. About

This is a frontend library to enable your users to type in many different languages of South Asia, and can be integrated into any React-based application. This library is a fork of react-transliterate, which uses Google Transliterate API which supports around 40 languages across the globe. In this module, our focus is to provide high-quality transliteration-suggestions for Indic languages, especially for low-resource languages like Kashmiri, Manipuri, etc. (which are not supported by Google). For more details about the AI system behind this, please check AI4Bhārat Indic-Xlit.

2. Demo

Click-here to try our demo!

Source of this demo is available in the example folder of the repo.

3. Install

npm install --save @ai4bharat/indic-transliterate

OR

yarn add @ai4bharat/indic-transliterate

4. Usage

4.1. Basic example

import React, { useState } from "react";

import { IndicTransliterate } from "@ai4bharat/indic-transliterate";
import "@ai4bharat/indic-transliterate/dist/index.css";

const App = () => {
  const [text, setText] = useState("");

  return (
    <IndicTransliterate
      value={text}
      onChangeText={(text) => {
        setText(text);
      }}
      lang="hi"
    />
  );
};

export default App;

4.2. With custom component

import React, { useState } from "react";

import { IndicTransliterate } from "@ai4bharat/indic-transliterate";
import "@ai4bharat/indic-transliterate/dist/index.css";

const App = () => {
  const [text, setText] = useState("");

  return (
    <IndicTransliterate
      renderComponent={(props) => <textarea {...props} />}
      value={text}
      onChangeText={(text) => {
        setText(text);
      }}
      lang="hi"
    />
  );
};

export default App;

4.3. With TypeScript

import React, { useState } from "react";

import { IndicTransliterate, Language } from "@ai4bharat/indic-transliterate";
import "@ai4bharat/indic-transliterate/dist/index.css";

const App = () => {
  const [text, setText] = useState("");
  const [lang, setLang] = useState<Language>("hi");

  return (
    <IndicTransliterate
      renderComponent={(props) => <textarea {...props} />}
      value={text}
      onChangeText={(text) => {
        setText(text);
      }}
      lang={lang}
    />
  );
};

export default App;

4.4. With Material-UI

import React, { useState } from "react";

import { IndicTransliterate, Language } from "@ai4bharat/indic-transliterate";
import "@ai4bharat/indic-transliterate/dist/index.css";

import Input from "@material-ui/core/Input";

const App = () => {
  const [text, setText] = useState("");
  const [lang, setLang] = useState<Language>("hi");

  return (
    <IndicTransliterate
      renderComponent={(props) => {
        const inputRef = props.ref;
        delete props["ref"];
        return <Input {...props} inputRef={inputRef} />;
      }}
      value={text}
      onChangeText={(text) => {
        setText(text);
      }}
      lang={lang}
    />
  );
};

export default App;

4.5. With External API for transliteration

import { IndicTransliterate } from "@ai4bharat/indic-transliterate";
import "@ai4bharat/indic-transliterate/dist/index.css";

const App = () => {
  const [text, setText] = useState("");

  return (
    <IndicTransliterate
      value={text}
      onChangeText={(text) => {
        setText(text);
      }}
      lang="hi"
      customApiURL="https://your-custom-transliteration-api/{language}/{text}"
    />
  );
};

export default App;

This allows users to integrate their own APIs instead of relying on the default transliteration API, as long as the external API is a GET API with the following format: https://your-custom-transliteration-api/{language}/{text} and returns a response in this format:

{
  "result": [
    "suggestion-1",
    "suggestion-2",
    "suggestion-3",
    "suggestion-4",
    "suggestion-5",
    ...
  ],
  ...
}

5. Get transliteration suggestions

5.1. Standalone example

import { getTransliterateSuggestions } from "@ai4bharat/indic-transliterate";

const data = await getTransliterateSuggestions(
  word, // word to fetch suggestions for
  {
    numOptions: 5, // number of suggestions to fetch
    showCurrentWordAsLastSuggestion: true, // add the word as the last suggestion
    lang: "hi", // target language
  },
);

5.2. Custom trigger keys

Keys which when pressed, input the current selection to the textbox

Indic Transliterate uses the event.keycode property to detect keys. Here are some predefined keys you can use. Or, you can enter the integer codes for any other key you'd like to use as the trigger

import React, { useState } from "react";

import { IndicTransliterate, TriggerKeys } from "@ai4bharat/indic-transliterate";
import "@ai4bharat/indic-transliterate/dist/index.css";

import Input from "@material-ui/core/Input";

const App = () => {
  const [text, setText] = useState("");

  return (
    <IndicTransliterate
      value={text}
      onChangeText={(text) => {
        setText(text);
      }}
      lang="hi"
      triggerKeys={[
        TriggerKeys.KEY_RETURN,
        TriggerKeys.KEY_ENTER,
        TriggerKeys.KEY_SPACE,
        TriggerKeys.KEY_TAB,
      ]}
    />
  );
};

export default App;

5.3. Props

6. Languages

6.1. Get supported languages

import { getTransliterationLanguages } from "@ai4bharat/indic-transliterate";

const data = await getTransliterationLanguages();

6.2. List of language codes

Currently supports the following 21 languages from the Indian subcontinent:

7. New Feature: Caching for Frequently Used Words

Overview:
The module now supports caching of up to 10,000 words per language to enhance performance by reducing API calls. The caching mechanism stores words and their corresponding suggestions, improving the speed and efficiency of generating suggestions, especially for frequently used words.

How It Works:

• Cache Initialization:
  When a user begins typing, the tool fetches suggestions for each word. If a word is used for the first time, it is added to the cache with its suggestions and an initial frequency count of 1.

• Cache Structure:
  The cache is structured by language, where each language has its own dictionary of words and their suggestions:

  {
    language-1: {
      word-1: {
        suggestions: string[],
        frequency: number
      },
      word-2: {
        suggestions: string[],
        frequency: number
      },
      word-3: {
        suggestions: string[],
        frequency: number
      },
      ...
    },
    language-2:{
    ...
    },
    ...
  }
  

• Cache Lookup:

  • When a word is typed, the tool first searches for it in the cache.
  • If found, the suggestions are retrieved, and the frequency count is incremented.
  • If not found, the tool fetches suggestions from the API, adds the word to the cache, and initializes the frequency count to 1.

• Cache Management:

  • The cache is limited to 10,000 words per language.
  • When the cache for a specific language exceeds 10,000 words, the least frequently used word is removed to make room for new entries.

• Example Workflow:

  1. User starts typing a word (e.g., "hello"):
     • The tool checks the cache for the word.
     • If "hello" is found, the suggestions are retrieved, and the frequency is incremented.
     • If "hello" is not found, suggestions are fetched from the API, and "hello" is added to the cache with a frequency of 1.
  2. Cache size exceeds 10,000 words:
     • The least frequently used word is replaced with the new word and its suggestions.

• Key Benefits:

  • Improved Performance: Reduces the number of API calls by utilizing cached suggestions for frequently used words.
  • Adaptive Caching: Automatically adjusts to user behavior, increasing the frequency count for commonly used words.
  • Efficient Storage: Least frequently used words are replaced to maintain an optimal cache size.

8. License

MIT © ai4bharat

Sincere thanks to burhanuday for making his work open-source!