---
name: hebrish-dataset-usage
description: Use the Hebrish English-Hebrew code-switching speech dataset (danielrosehill/English-Hebrew-Codeswitching-Sentences on Hugging Face) to train or evaluate any speech recognition model - access, columns, splits, NeMo manifests, Whisper/wav2vec2 text targets, the English mixing recipe, and term-recall scoring. Use when a user wants to fine-tune, benchmark or test an ASR model on English with Hebrew words in it.
metadata:
  publisher: hebrish.app
  updated: "2026-10-20"
  source: https://hebrish.app/integrate
---

# Using the Hebrish dataset

Full guide with tested snippets: https://hebrish.app/integrate. Dataset page:
https://hebrish.app/dataset.

## Before you start

- **One speaker.** Every clip is one person, an Irish-accented English speaker in Jerusalem.
  A model trained only on this learns one voice; results on its test split are results for
  that voice. Say so in any write-up.
- **Gated.** Since 2026-10-20 the dataset is gated on Hugging Face. Request access on the
  dataset page, then authenticate (`huggingface-cli login` or `HF_TOKEN`). Without access,
  `load_dataset` fails with an authorisation error; that is not a bug in your code.
- **Preview without access:** 40 clips with transcripts and audio URLs at
  `https://cdn.hebrish.app/samples/clips.json` (same `id`, `text`, `target_terms`, `split` as
  the dataset rows), playable at https://hebrish.app/archive.
- **It grows.** Pin a revision for anything you report. Current size:
  `https://hebrish.app/data/stats.json` (`rows`, `minutes`, `splits`, `revision`).

## What is in it

- English sentences with Hebrew words, read aloud. Audio 16 kHz mono 16-bit WAV in
  `data/{train,test}/audio/`; metadata in `data/{train,test}/metadata.jsonl`
  (`file_name` is relative to the folder the metadata file is in).
- **The test split is term-disjoint**: no Hebrew term in test appears anywhere in train, so
  test scores measure generalisation to unseen words.
- Columns that matter:
  - `text`: the target. Hebrew words romanized in the lexicon's one spelling.
  - `text_he`: same transcript with Hebrew script, on about two thirds of rows; null on the rest.
  - `target_terms`: Hebrew terms and Israeli names the sentence was written to contain. Use for
    term recall.
  - `duration_s`, `categories` (every row has `code_switch`), `device`, `microphone`,
    `environment`, `id`, `recorded_at`, `prompt_source` / `prompt_origin` (needed for
    attribution of reused text).

## Load

```python
from datasets import load_dataset, Audio   # pip install "datasets[audio]"
ds = load_dataset("danielrosehill/English-Hebrew-Codeswitching-Sentences")  # needs access
ds = ds.cast_column("audio", Audio(sampling_rate=16000))
```

NeMo: write one JSON line per clip with `audio_filepath`, `duration` (`duration_s`) and `text`
from the metadata files after `snapshot_download(..., repo_type="dataset")`. Every Hebrish
version was trained with NeMo on `text` (Parakeet's tokenizer has no Hebrew characters).

Whisper and similar: train on `text` for Latin output, or on `text_he` (filter out null rows
first) for Hebrew-script output. CTC models: build the character vocabulary from `text`; it
includes apostrophes (`tza'atzuim`).

## Keep English intact

Fine-tuning on code-switched speech alone tends to damage ordinary English. The Hebrish recipe
mixes by audio duration: 70% this dataset, 20% plain English in the target voice, 10% general
English replay (Hebrish uses LibriSpeech). This dataset holds only the 70% part.

## Score the Hebrew words, not just WER

WER hides Hebrew-word errors. Score term recall on the test split: for each term in
`target_terms` that appears in the reference `text`, check whether the hypothesis contains it
as a whole word, case-insensitive, exact spelling:

```python
import re
def has(term, text):
    return re.search(r"(?<!\w)" + re.escape(term) + r"(?!\w)", text, re.I) is not None
```

Count only terms present in the reference text (a few listed terms are spelled differently in
the sentence). Report WER on `text` alongside it. For a frozen comparison with published models,
use the public benchmark: https://huggingface.co/spaces/danielrosehill/English-Hebrew-Codeswitching-ASR-Leaderboard.
Correct spellings for any term: `https://hebrish.app/data/lexicon.json`.

## Licence

The dataset is CC-BY-4.0: train on it, ship models built from it and redistribute it,
commercially or not, with credit. Some sentence text comes from other sources (for example
Tatoeba, CC-BY 2.0 FR); `prompt_source` records which, so keep that attribution when you
redistribute the text. The Hebrew-Latin-Token-Classifier spelling lexicon is MIT. A credit line:

```
English-Hebrew-Codeswitching-Sentences by Daniel Rosehill, CC-BY-4.0.
https://huggingface.co/datasets/danielrosehill/English-Hebrew-Codeswitching-Sentences
```

The Hebrish models have their own terms: https://hebrish.app/terms.