Use it in your own pipeline

On this page:

1. What is in the dataset
2. Load it
3. NeMo manifest
4. Whisper, wav2vec2 and other Hugging Face models
5. Mix in plain English
6. Evaluate term recall
7. Public benchmark
8. Licence and attribution

Snippets marked tested were run against the dataset at revision 4cb4c84b on 2026-10-19 (datasets 5.1.0). Snippets marked sketch show the shape of the code and were not run.

What is in the dataset

danielrosehill/English-Hebrew-Codeswitching-Sentences on Hugging Face. English sentences with Hebrew words in them, read aloud, with the Hebrew words romanized in one spelling. CC-BY-4.0.

• Two splits: train (2,712 rows, 305.3 min) and test (306 rows, 37.1 min). Counts as of 2026-10-19; the dataset grows.
• The test split is term-disjoint. No Hebrew term in it appears anywhere in train. Scores on it measure whether a model generalises to Hebrew words it never saw. More on the dataset page.
• Audio: 16 kHz, mono, 16-bit WAV, in data/train/audio/ and data/test/audio/.
• Metadata: data/train/metadata.jsonl and data/test/metadata.jsonl, one JSON object per clip. file_name is relative to the folder the metadata file is in.

One real row from the test split (text_he is empty on this one):

{
  "file_name": "audio/20261014T190834Z_r9islo.wav",
  "id": "20261014T190834Z_r9islo",
  "text": "Reminder to call Moran Edri tomorrow about the tza'atzuim, and ask whether she wants the white ones or the blue ones.",
  "duration_s": 9.3,
  "categories": ["message", "code_switch", "names"],
  "target_terms": ["tza'atzuim", "Moran", "Edri"],
  "prompt_source": "generated:anthropic/claude-sonnet-5.5",
  "device": "OnePlus Nord 3 5G",
  "microphone": "internal",
  "environment": "quiet-indoor",
  "sample_rate": 16000,
  "recorded_at": "2026-10-14T19:08:34.057Z",
  "text_he": null,
  ...
}

Load it

With Hugging Face Datasets. Recent versions decode audio with torchcodec, which the audio extra installs.

pip install "datasets[audio]"

tested

from datasets import load_dataset, Audio

ds = load_dataset("danielrosehill/English-Hebrew-Codeswitching-Sentences")
ds = ds.cast_column("audio", Audio(sampling_rate=16000))

row = ds["test"][0]
audio = row["audio"]["array"]           # numpy array
sr = row["audio"]["sampling_rate"]      # 16000
print(row["text"], row["target_terms"])

The audio is already 16 kHz, so the cast is a guard, not a resample. Add streaming=True to load_dataset to read it without downloading everything first.

NeMo manifest

NeMo wants one JSON line per clip with audio_filepath, duration and text. Download the repository and write the manifests straight from the metadata files.

tested

import json
from pathlib import Path
from huggingface_hub import snapshot_download

root = Path(snapshot_download(
    "danielrosehill/English-Hebrew-Codeswitching-Sentences", repo_type="dataset"))

for split in ("train", "test"):
    folder = root / "data" / split
    with open(f"{split}_manifest.json", "w") as out:
        for line in open(folder / "metadata.jsonl"):
            row = json.loads(line)
            out.write(json.dumps({
                "audio_filepath": str(folder / row["file_name"]),
                "duration": row["duration_s"],
                "text": row["text"],
            }) + "\n")

Every Hebrish version was trained with NeMo. If your model's tokenizer has no Hebrew characters, as Parakeet's doesn't, train on text.

Whisper, wav2vec2 and other Hugging Face models

The dataset has the columns any Hugging Face speech recipe expects: an audio column and a text column. Follow the standard guides and point them at this dataset:

• Seq2seq (Whisper and similar): Fine-Tune Whisper For Multilingual ASR.
• CTC (wav2vec2, HuBERT and similar): the automatic speech recognition task guide.

Pick the text column to match what your model can write:

• text: Hebrew words in Latin letters. Works for any English tokenizer or character vocabulary.
• text_he: Hebrew words in Hebrew script. Whisper's tokenizer covers Hebrew script, so it can learn this target. Only some rows have it, so filter first.

sketch Whisper preprocessing

from transformers import WhisperProcessor

processor = WhisperProcessor.from_pretrained("openai/whisper-small")
TEXT = "text"   # or "text_he"

if TEXT == "text_he":
    ds = ds.filter(lambda t: t is not None, input_columns="text_he")

def prepare(row):
    audio = row["audio"]
    row["input_features"] = processor.feature_extractor(
        audio["array"], sampling_rate=audio["sampling_rate"]).input_features[0]
    row["labels"] = processor.tokenizer(row[TEXT]).input_ids
    return row

ds = ds.map(prepare, remove_columns=ds["train"].column_names)

From here the guides' data collator, metrics and trainer work unchanged. For a CTC model, build the character vocabulary from the text column; it includes apostrophes, as in tza'atzuim.

Mix in plain English

Fine-tuning on code-switched speech alone tends to damage a model's ordinary English. The Hebrish recipe guards against that by mixing, by audio duration:

This dataset holds only the code-switched part. For the 20%, use plain English in your own target voices and domain. For the 10%, any general English set works; Hebrish uses LibriSpeech. By v3, Hebrish's English scores were as good as or better than stock. Full recipe on the model card.

sketch How much English to add

cs_min = sum(ds["train"]["duration_s"]) / 60
print(f"own English: {cs_min * 20 / 70:.0f} min, LibriSpeech: {cs_min * 10 / 70:.0f} min")

Evaluate term recall

Word error rate hides the thing that matters here. A model can get every English word right and every Hebrew word wrong and still score well. So score the Hebrew words directly: for each target term, did the model write it in the lexicon's spelling?

Use the test split. Because it is term-disjoint, a model trained on train has never seen these terms, so the score measures generalisation, not memory.

Use target_terms as the reference list. A few listed terms are written differently in the transcript itself (the list says cheder madregot where a sentence says heder madregot), so count only terms that appear in the reference text. In the test split that keeps 479 of 484 term occurrences. target_terms includes Israeli names as well as Hebrew words.

tested The scoring logic was checked on the test metadata: reference transcripts as output give recall 1.0, empty output gives 0.0. Plug in your own transcribe.

import re
from datasets import load_dataset, Audio

test = load_dataset("danielrosehill/English-Hebrew-Codeswitching-Sentences", split="test")
test = test.cast_column("audio", Audio(sampling_rate=16000))

def has(term, text):
    # whole word, case-insensitive, exact spelling
    return re.search(r"(?<!\w)" + re.escape(term) + r"(?!\w)", text, re.IGNORECASE) is not None

hits = total = 0
for row in test:
    hyp = transcribe(row["audio"]["array"])   # your model, returns a string
    for term in row["target_terms"]:
        if not has(term, row["text"]):
            continue
        total += 1
        hits += has(term, hyp)

print(f"term recall: {hits / total:.1%} ({hits}/{total})")

This is exact-spelling recall. A model that writes a different romanization of the right word gets no credit, which is the point: one consistent spelling is what makes dictated text searchable and correctable. It is not the leaderboard's scorer, so its numbers are not directly comparable with the leaderboard's.

Scoring your own recordings with the lexicon

For a test set without target_terms, find the reference terms by matching a lexicon against your reference transcripts:

• terms.csv in Hebrew-Latin-Token-Classifier: the canonical spellings (term column), with accepted variants and a status column.
• /data/lexicon.json on this site: every term and Israeli name the dataset covers, in terms[].t, with a domain in terms[].d. Browse it in the lexicon explorer.

Be careful with a blind match: lexicon.json includes Israeli brands that are also ordinary English words, such as Yes, Fox and Ivory. Filter those out, or match case-sensitively.

Public benchmark

For a number you can compare with other models, use the frozen benchmark rather than the test split.

• ASR benchmarkFrozen test set, 145 clips (clean subset 84). Keep it out of training.
• LeaderboardModels side by side on that test set

On the leaderboard's clean subset, Hebrish v3 leads on term recall at 67.4%, against 45.9% for Whisper Large V3 and 32.6% for stock Parakeet TDT 0.6B v2. The leaderboard is early-stage, and 22 of its 139 term labels are English tech words, not Hebrew. Full table on the model card.

Licence and attribution

• The dataset is CC-BY-4.0. You can train on it, ship models built from it and redistribute it, commercially or not, if you give credit.
• Some sentence text comes from other sources (for example Tatoeba, CC-BY 2.0 FR). The prompt_source column records which; keep that attribution when you redistribute the text.
• The Hebrew-Latin-Token-Classifier lexicon is MIT.

A credit line that works:

English-Hebrew-Codeswitching-Sentences by Daniel Rosehill, CC-BY-4.0.
https://huggingface.co/datasets/danielrosehill/English-Hebrew-Codeswitching-Sentences

Built something with it? Say hello. Results on other models, especially other voices, are welcome.