Hebrew-Latin Token Classifier small model

What it does

Give it a sentence and it marks every span of romanized Hebrew, with character offsets and a confidence score. Multi-word terms come out as one span.

I need to go to Bituach Leumi before the makolet closes.
                [Bituach Leumi]          [makolet]

It is a fine-tune of xlm-roberta-base with three labels (O, B-HE, I-HE): small enough to run on a CPU, and the first model for this task. The Hebrish speech models write exactly this kind of text, English with Hebrew words romanized, so it is also the way to find the Hebrew in a Hebrish transcript.

Try it in the demo Space Model on Hugging Face

Why nothing could do this before

The letters give nothing away. Telling Hebrew from English is easy when the Hebrew is written in Hebrew letters: they have their own Unicode block (U+0590 to U+05FF), so a program can see the switch character by character. Written in Latin letters, makolet is made of the same Basic Latin characters as every English word around it. Nothing at the level of the text says a language changed. The script itself code-switches, and no tool expected that.

Language identification doesn't cover it. GlotLID, fastText lid.176 and lid218e have no label for Hebrew in Latin script (heb_Latn). No token-level code-switching model covers Hebrew, and no English-Hebrew code-switching corpus had been published. Even a heb_Latn label would not be enough: sentence-level language ID calls a sentence that is 90% English an English sentence. A speech pipeline needs the character offsets of the one Hebrew word, which makes this token classification, not language ID.

Always translating doesn't work either. Many of these words have no good English equivalent, and the translation is not what anyone in Israel would say. Bituach Leumi is Bituach Leumi.

Where it sits: text-to-speech for a code-switched script

It was built for My Weird Prompts, an AI-generated podcast produced in Jerusalem. A script-writing agent writes each episode in English, and Chatterbox TTS voices it. When an episode is about life in Israel, the script fills up with Hebrew in Latin letters, so the script is itself a code-switching text. English TTS reads makolet as "MACK-oh-let". Words English already knows (Shabbat, kosher) come out fine; the long tail of everyday Israeli life does not. The fix is a chain of small components, and this model is the one that was missing.

1. 1 · Script

   A code-switched script

   The podcast's script agent writes English. Hebrew terms arrive in Latin letters.

   I need to go to Bituach Leumi before the makolet closes.

2. 2 · Detect this model

   Hebrew-Latin Token Classifier

   Marks each Hebrew span with its character offsets.

   I need to go to Bituach Leumi before the makolet closes.

3. 3 · Transliterate

   Romanized Hebrew to Hebrew script

   A curated lexicon first, with TaatikNet as the fallback, one word at a time. TaatikNet guesses spellings without context (it wrote savta as סַוְטָה), which is why the lexicon comes first.

   Bituach Leumi → ביטוח לאומי · makolet → מכולת

4. 4 · Niqqud

   Vowel points

   DICTA's nakdan points the lexicon forms, so the voice does not have to guess how to say them. TaatikNet's output is already pointed.

   בִּטּוּחַ לְאֻמִּי · מַכֹּלֶת

5. 5 · Tag

   Segments with language tags

   Plain code, not a model: split at every switch, one language per piece.

   ("I need to go to ", en) (בִּטּוּחַ לְאֻמִּי, he) (" before the ", en) (מַכֹּלֶת, he) (" closes.", en)

6. 6 · Speak

   TTS with a language parameter

   Chatterbox Multilingual sets language_id per call, so each segment is voiced with the same voice prompt and the clips are joined. Among hosted engines, Azure's multilingual voices take an inline <lang xml:lang="he-IL"> tag; Google documents Hebrew in that tag as unsupported, and Polly has no Hebrew.

7. Out

   One voice, Hebrew words said as Hebrew

Deterministic, rather than a model that switches by itself. Every switch is decided in code before the TTS sees the text, so what was detected can be read, tested and fixed one word at a time. The alternative, one multilingual TTS model trusted to switch mid-sentence, gets no signal from romanized Hebrew, which is Latin script and looks like English, and offers no handle on its decisions.

Precision matters most. Every detected span costs a sentence split, an extra TTS call and an audio seam. A missed Hebrew word costs nothing: you get today's behaviour. A false positive pays all that to make the output worse, an English word read by a Hebrew voice. So the labelling is conservative, and inference takes a threshold.

Where to draw the line: the Shabbat question

Shabbat is a Hebrew word. Flagging it would hand it to a Hebrew voice mid-sentence, though English already says it fine. The labelling test is pronunciation, not etymology: Shabbat, kosher, hummus, challah and aliyah are not flagged, nor are city names. The fixed point every rule is tested against is makolet: common in the speech of English speakers in Israel, absent from every major speech corpus. If a rule would exclude makolet, the rule is wrong. A later version could add a third label, "Hebrew that English already says", and leave the choice to the user.

How well it works

Scored on a term-disjoint test split: no Hebrew term in the test set appears in training, so this measures how it does on words it has never seen. The metric is span F1: a three-word term with two words found counts as a miss and a false positive.

• No false positives on 58 hard negatives: sentences built to trap it, with substring traps, the English sense of homographs (dud, tor, salon), lookalikes and anglicised loanwords.
• 14 of the 41 wrong test sentences are label gaps, not model errors: real Hebrew (lechem, od paam, mas hachnasa) the blind annotator refused and no human has re-reviewed. Measured precision is a lower bound.
• Weak spots: Hebrew-sounding place names, which the policy excludes (Dizengoff, Ramat Gan); short words that look like loanwords (shakshuka, botz); the third word of three-word compounds.

For comparison, the closest published recipe (EnTaCs, English-Tamil, arXiv 2603.26587) reports F1 0.638 on its romanized class. Results from the model card.

Use it

Take each word's label from its first sub-token, the only one trained. The stock pipeline works with aggregation_strategy="first"; do not use "simple", which cuts words short (mako). predict.py in the repository also pre-splits words the way the training data was split, so trailing punctuation stays out of spans.

from transformers import pipeline
ner = pipeline("token-classification", model="Hebrish/Hebrew-Latin-Token-Classifier",
               aggregation_strategy="first")
ner("My savta says the arnona went up again.")

Hosted: the Replicate API (CPU; inputs text and threshold, returns the span list). Code, annotation policy and taxonomy: GitHub.

Training data and limits

Hebrish/hebrew-latin-code-switching: 4,737 sentences over 1,235 seed terms. Sentences were generated from a seed term, then annotated blind by a second model family that never saw the seed; disagreements, and a 10% random audit of agreements, went to human review. 458 dictated sentences (the text of sentences read aloud for the Hebrish speech recordings) are the closest thing to real input. Spans nobody has ruled on, and Israeli person names, are masked: no loss, not scored.

• The input is English text with romanized Hebrew. Hebrew script is not the target.
• Romanization varies; an unusual spelling of an unseen word is the hardest case.
• Person names are unlabelled, so behaviour on them is undefined.
• The data is largely synthetic.

Why this model exists, in six minutes

A voice note recorded the day after release, on why it was built.

Transcript

At a glance

Repositories

Hugging Face issues a DOI per repository, so each format can be cited on its own. Repositories, sizes and DOIs checked on the Hub on 2026-10-20.

The model moved to the Hebrish organisation on 2026-10-20 as a copy of danielrosehill/hebrew-latin-token-classifier at revision 01f092f1, the same weights: Hugging Face does not move a repository that has a DOI. New versions are published under Hebrish.

Versions

Cite

Cite Hebrew-Latin Token Classifier with its DOI, 10.57967/hf/10997 (danielrosehill/hebrew-latin-token-classifier). To cite one format, use that repository's DOI from the table above.

@misc{rosehill_token_classifier_2026,
  author    = {Daniel Rosehill},
  title     = {hebrew-latin-token-classifier},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/danielrosehill/hebrew-latin-token-classifier},
  doi       = {10.57967/hf/10997}
}

Licence and access

All Hebrish models