What Hebrish is, how it's made, and the next sprint

This is the recording written up to be read: everything I said is here, tidied into sections. The audio is lightly cleaned: long pauses shortened, level evened out. Then the voice was restored with Sidon (University of Tokyo) to sound closer to a studio microphone. The intro and outro are voiced by Herman Poppleberry of My Weird Prompts, over “Happy Strummin’” by Jason Shaw (music by audionautix.com, CC BY 3.0); the recording starts at 0:15. Timestamps jump to that point in the recording. It dates from 2026-10-22, so some plans and model roles have changed since. Corrections made after recording are at the end. All episodes: podcast.

0:15 Why this voice note

This is the first installment of the Hebrish app podcast. I don't intend to launch a podcast for this project, because I don't think a code-switching project needs one. But this voice note has several uses. On the website it's an update for anyone who wants to hear what the project is. It's also how I gather notes and context for the next few sprints for my AI agent, the esteemed Mr. Claude Code. That's Claude Opus 5.5, and it's starting to scare me: it's the first model that is just prodigiously good. It makes me feel very stupid, I can't keep up with it, and it's wonderful how quickly it has helped me iterate.

So the first half explains to humans or AI agents, whoever you are, what this project is and why I wanted to do it. The second half is what's on my mind for features, which is really instructions for Claude. I like to build in public. My GitHub workspaces usually have a folder for project management, and I almost always use the same voice workflow: I record, and then come sprints, decisions and so on. On this project I'm doing that in public.

1:51 How I came to speech to text

I started using speech to text coming up on two years ago, with Whisper, OpenAI's open-source speech-to-text model, which is an exceptional ASR model. (I'll try to de-jargonize the tech: ASR is automatic speech recognition, the basic speech-to-text model.) I've been using Linux for going on 20 years, and I'd always wanted to use speech to text.

Assistive tech and ADHD

I'm very open about having ADHD, and I mention it because I feel these tools, text to speech and speech to text, are for people with neurodivergence. I think we're overrepresented among the people who consider them the best things ever, and I'm definitely in that category. That interests me because these technologies have such a deep assistive lineage: people with disabilities, people who can't type, the elderly. Until recently I'd never have connected neurodivergence to that population, but maybe we belong under that umbrella, along with people who just find it generally helpful.

The week I stopped typing

I tried speech to text a few times on different projects, and until Whisper it was, I would say, virtually unusable. Earlier versions worked, but they didn't cross the bar where they actually save you time. If it doesn't cross that bar, it's worse than useless, because you just go back to the keyboard.

Then Whisper came along. My wife was in the US around Thanksgiving two years ago, so I had the place to myself, and that was the week I became a convert. I told Hannah she'd come back to find me talking into my phone constantly, because I'd discovered this new app and decided I was never going to type again. It was great to wander around the kitchen getting all my ideas and to-do lists out, without feeling like anyone was listening.

Across devices

Once I saw how fast the technology had matured, I began trying it across devices. The Linux desktop has that weird feature of being where most prototypes happen and, paradoxically, often where finished tech arrives last, because it's the smallest market. There were apps, and I started using Whisper on my phone too. Over the past couple of years I've been building prototypes for my own use, because however many great speech-to-text apps there are, there are always UI quirks I want.

The first app I got excited about was voicenotes.com. I don't know if they still use Whisper under the hood. I used it for everything: email drafts, and a lot of prompts for AI tools. That's where I began getting great results with AI. Voice is so much quicker than typing that you can record all the context, all the background about yourself, that's essential to a well-rounded answer.

6:51 Why Hebrew words go wrong

Anyone who's used speech to text in Israel will tell you it struggles with Hebrew terminology. Some words do get transcribed, like Shabbat and kashrut, and that throws a lot of people off.

The big ASR models, like Whisper, the first major open-source one, are trained on a huge corpus of speech. The general understanding is that a lot of it is YouTube video, which is why earlier models would sometimes hallucinate things like "thanks for watching": many hours of training audio ended that way.

A huge corpus of general English from a source like YouTube will include words like Shabbat. So how reliably a Hebrew word gets transcribed depends on how much it's in general circulation in English, biased towards American English and the content on the English-speaking internet. That distribution doesn't reach a lot of the words English speakers in Israel use, like shuk and bituach leumi. The further down that long tail you go, the less likely the word is to be in a stock model trying to transcribe all of English.

Every Hebrew word you want recognized that sounds like something in English is also fighting inside the model. These models are predictive tools. They look at your phonemes, the sounds you make, and find matches between patterns of articulation and words. They aren't literally understanding, and they have no concept of words as such. So a niche Hebrew word that isn't well represented will always be displaced by an English word that sounds like it. On top of that, when you pronounce a Hebrew word, you're following Hebrew phonetics.

9:38 How the fine-tune works

My first attempt was a fine-tune of Whisper. The method, and this is where it gets tedious, is reading sentences like "I need to go to the makolet today." You build a dataset from your voice recordings plus a ground truth: the model receives the correctly transcribed version, with makolet spelled M-A-K-O-L-E-T. Then you train. Training doesn't start from scratch. You take an existing model, like Whisper or the latest Parakeet, and change its internal weights, which is a way of exposing it to the new words.

The challenge is that you're a drop in the ocean: a drop of weights about makolet and bituach leumi in a model with hundreds of thousands of hours of audio behind it. The potential pitfall is overfitting, or catastrophic forgetting. The model gets so attuned to the niche words that you get the inverse problem, and it starts predicting code-switched words for words that are in English. (I'm trying to think of an English word that sounds like makolet, and I don't have one off the top of my head.)

That's why the training mix contains neutral English sentences. I say "I'm moving the table lamp to the couch," which has no training target. It's just a stam sentence. Those keep the model from forgetting how to process normal speech while it shifts towards the code-switched vocabulary.

12:42 Why Parakeet, and the model plan

The fine-tune is based on NVIDIA's Parakeet, a lightweight ASR model of about 600 million parameters. I chose it because my wife and I moved apartments a few months ago, we've been moving boxes ever since, and for the first time in my life I've been using my phone more than my desktop. Getting ASR models to run on a phone is still challenging: even the best flagship is fundamentally a small piece of hardware. So the question is what's the smallest model that isn't rubbish. Parakeet is small, light and quite good.

NVIDIA has done something OpenAI hasn't really done with Whisper. People have tried bringing Whisper to the phone by quantizing it, and I've used my own Whisper fine-tune on the phone without great results. The first time I found an app with Parakeet, I thought: this is the one. It's fast and accurate enough, and I immediately saw it was a much more promising base for fine-tuning, though I think both have value.

Qwen also has an ASR model, which I fine-tuned yesterday. The training plan is:

• one model for maximum accuracy, for servers or desktops with GPU acceleration
• one everyday driver for your phone
• one for small devices
• one for streaming: NVIDIA's Nemotron is a streaming-output model, and I'm going to update it with the dataset to get a first good version trained

That's where the second part of this note comes in.

15:12 One app, not two

The way I use these every day straddles a few use cases. A lot of the problem with speech interfaces and dictation tools so far is really a UI criticism: there's a split between long form and short form. People roll out voice keyboards, and people build transcription tools for long-form processing. I don't think they need to be two apps. In this case at least, I'd rather have one app that does several things.

16:12 Why now

Before the UI backlog, a little about the models. Claude actually coined the names; I wanted one name for each type of model.

This past week has been a crazy, frenetic development period, and a large part of what's driving it is this: my wife and I have a son, a year and a few months old, and we've been in the emergency room twice in the last month.

I find speech recognition more helpful every day, and even more indispensable since I became a parent. When you're holding a squirming child, trying to clean up and working out how to answer an email, voice to text is by far the most efficient way to do it. In the emergency room, talking to our health fund, I realized this kind of tool could help a lot of people. In those situations you don't have time to go to your keyboard. If you're messaging someone with an update about your kid, or talking to the insurance or someone who speaks English, you don't have time to fix mistranscriptions.

So my motivation is primarily a good tool for my own use. And I genuinely think, and hope, it will be useful to other people.

17:59 Beyond Hebrew words: disfluencies, shekels, paragraphs

One more thing about the training composition; this is going to be a thorough voice note. When I began training on the code-switching sentences, I made a list, because this is primarily for my own use (if other people find it helpful, great, and I'll try to accommodate different usage patterns). What do Parakeet and Whisper get wrong? What stands between 90% good and 100% good?

Where ASR is heading

That's the gap I feel ASR is in. Plugins and MCP are becoming very mature. ChatGPT finally has an email tool, a Drive tool and a Docs tool, so you can dictate into one of these apps and say "send that as an email to my boss." That's taken a while to reach consumer apps, and it's a very big deal for ASR.

The baseline job of an ASR model is turning phonetics into letters on a screen. No rule says it will add punctuation, capital letters, sentence boundaries or paragraphs. None of that is in the fundamental logic of ASR; it arrived as the models improved. Right now there are bolt-ons, like the dedicated post-processing layers in dictation apps such as SuperWhisper. There are also multimodal, or omnimodal, models that process both audio and text, where you can write a prompt saying "the audio you've received is a shopping list; output it as a shopping list." That's a very appealing approach to me.

Until recently I'd have said that's where ASR is going, and classic ASR would be yesteryear's technology. But the other method is still going very strong: an ASR model paired with an LLM, where the second pass is a cleanup step that can also reformat.

What I'm trying is an architectural decision more than anything else, and it's experimental. Not everyone will think it makes sense; it's what makes sense to me now. We're doing the code-switching training on Parakeet, so can we push that a bit further? What else is a problem?

Disfluencies

When I began using Parakeet, I noticed it output disfluencies: hesitations, false starts and word repetition. A false start is "I want to…" and never finishing the sentence. Repetition is "I'm going to put the tomato, tomahdo, in the fridge." You're out there, scattered, holding a phone to your face, and the transcription you want is "I want to put the tomatoes in the fridge."

Shekels

Another one is both Parakeet-specific and Israel-specific. If you say "the falafel cost 20 shekels" and that's going into a WhatsApp or an email, I can't think of a situation where I'd want "shekels" written out. I'd prefer "20 NIS." People might disagree, but as the creator of this open-source project, I have the luxury of building it for my personal preferences.

Paragraphs

The lack of paragraph spacing is one I've always found strange. So I recorded a dataset pair for paragraph-spaced text: a long-form ground truth, say a few paragraphs about a DIY project, which Parakeet returns as one blob.

This is where the post-processing argument comes in. One school says a small post-processing model should do all the after-steps: ASR is responsible only for an accurate rendition of the words, and everything else, paragraph breaks included, comes next. My beef with that is that the moment you add sentence boundaries and punctuation, which Parakeet does, the model has already gone beyond rendering words: it's creating sentences. I'd argue paragraph breaking has been a big blind spot in ASR. It's a fundamental property of readable text, arguably barely less essential than full stops.

That's a small dataset, and the question is whether it can nudge the model towards doing this. There might be an architectural reason it's impossible that I haven't thought of. I'm an experimenter and a builder, I don't know all the inner details of how these models work, and it's very complicated stuff. So that's another experiment.

25:55 Tech vocabulary, my own words, and Meorav

Another small dataset adds technical vocabulary: "I'm running a Docker Compose script," "I'm running a Kubernetes cluster." And I have a small dataset of my own quirky vocabulary, the things I see the model getting wrong. I have an AI-generated podcast called My Weird Prompts, with hosts called Herman Poppleberry and Corn, and those names are commonly botched. Again, that one's for my own benefit.

A voice clone, and a TTS model

That brings me to the restructure I did today. For a while I've wanted a voice clone, because I like to pull pranks and have a voice agent call up a friend. If Mike ever listens to this, he's probably spoken to my voice clone. Voice cloning of course has huge potential for black-hat use and fraud. My view is that I have a YouTube channel and hundreds of hours of my speech on the internet, and you can clone anyone's voice from ten seconds of YouTube, so I want my voice available for my own occasional use.

While doing this, I wondered whether the code-switching dataset could also create a TTS voice. The elephant in the room is that not many people in Israel have an Irish accent. A TTS that renders code-switched text correctly, but in an Irish accent, has niche appeal; an acquired taste, you might say. My voice isn't the ideal baseline, but it's worth having out there as an experimental TTS.

I was arguing with Claude that the TTS one needed a more imaginative name, and we landed on Meorav. I thought that was clever. For those outside Israel, meorav yerushalmi is a famous dish of all sorts of organ meats thrown together, and that's kind of what code-switched speech is. I trained a first test run yesterday.

29:22 The kitchen sink: Sabich and Jachnun

For anyone interested in how the datasets are structured: earlier this week I was recording the code-switching sentences, like "I need to go to the makolet to buy milk," and the tech sentences, like "I need to check the Docker memory use." My first idea was a Parakeet tech model and a Parakeet code-switching model, because I assumed mixing them would hurt accuracy on both.

A few days ago we ran a test evaluation of what I initially called the kitchen sink. I told Claude to lob it all together into one dataset, train it and see how it does on everything. To my surprise, accuracy didn't degrade on either the tech words or the Hebrew words. So the open question is whether the kitchen-sink model, Sabich, is the right default. For people who work in tech and code-switch, it's the obvious choice. If you don't work in tech, does a sample of vocabulary you won't use degrade it, or may it as well be in the model?

From a maintenance standpoint it's much easier to ship one model than two, so my long-term preference is to collapse to one, for Hebrish at least. For people outside Israel, a model with just the tech vocabulary is a more logical fit, so I also want to ship one of those under my personal account. It isn't a Hebrish model. I'm totally confused by all the names Claude has come up with.

What the names stand for

Each name stands for a dataset composition rather than a specific model:

• Sabich is the kitchen sink: all the things and all the experiments baked in.
• Jachnun, if we maintain it, is pure code switching: more conservative, with no tech dataset or my niche personal words, and perhaps none of the experimental datasets like paragraph inference. Whether it will have a life of its own is an open question.

I wanted series names because within each family I'll experiment. Can we have a desktop version? Can we run the fine-tuning dataset on Qwen, or on a new ASR model when one comes out? The name refers to the series and what it contains, not the particular iteration.

33:50 Licensing, accents and noisy recordings

That's a lot of ground already, and I need to get to the implementation part, though hopefully all this context is useful to the agent too. Licensing is free for non-commercial use, because perhaps someone will find a good commercial use for this.

What the dataset still needs

As I've built the dataset, it's occurred to me how much more it needs: Ashkenazi and Sephardic pronunciation, different accents, different genders. And there's so much religious vocabulary: all the masechtot of the Talmud, the major sefarim, the parshiot, major rabbis.

Why scrappy recordings help

When I want to add a word, take makolet, the classic example I use over and over, I don't record one sentence. Ideally it's ten. And this is where data collection for TTS and speech to text differ significantly. For speech to text you actually want a dataset that's a bit scrappy, because it should reflect real-world use.

People try to dictate in the shuk with a bar blaring music and other people talking around them in different languages. Each of those is a distinct challenge, because the model has no built-in way of telling the intended speaker from everyone else. You might have wind noise, honking, a baby crying. So when I'm recording I think: great, this is noisy, let's record a few samples here. As you record a word in different conditions, the model gets more robust, and it doesn't collapse when you say a code-switched word in a gust of wind. That's the augmentation.

Why licensing at all

Back to licensing. This is something I wanted for my own use, and to benefit other people in Israel and the Jewish world. But time costs money. Training and fine-tuning cost money, developing the app takes a lot of time, and so does recording the samples. So I wanted to leave the door open to people supporting it or licensing it, if it's useful in a context like telephony.

For now it's open like that, and the datasets are access-gated on Hugging Face. I'm not unaware that someone could look for a dataset of code-switched English and Hebrew with bad intentions, to put it that way, so that's a voluntary security measure on my side more than anything else.

37:50 Apps and platforms

The apps are currently biased towards what I actually use. There's an Ubuntu app and an Android app, initially provided through F-Droid. I'm looking into publishing on Google Play, which needs a few small changes, a review and a demo recording, and I definitely want to do it. Windows is a thing as well.

iOS is more complex, because inference there works differently and has its own pathway. It's a more involved migration, and of course very important: it should be available on both platforms, 100%. It's a work in progress, and again something that costs money. I looked into it, it's a bit more expensive than Android to ship, and for the moment I have to keep my investment of time and money reasonably capped.

On other speakers, I'm thinking about all the ways people pronounce Hebrew: Yeshivish, Haredi, Ashkenazi, Sephardic. I want all of them represented.

39:25 One recorder for speech recognition and speech synthesis

Something I noticed recording these samples: it's very hard to focus on tagging and recording at the same time. Claude built a fantastic custom recording app for me, with settings where I can say "baby crying" and the annotation gets noted. But it's hard to do that reliably when you're just trying to read and pronounce things correctly and audibly.

There's huge overlap between the TTS and STT data, with one difference. For text to speech you want the opposite of the STT conditions: a clean noise floor and no background noise. It's the same vocabulary, but samples are only good for the TTS set if they're quiet. Noisy ones are fine for ASR but not for TTS.

As I said, Opus 5.5 is the genius I'm explaining all this to, and I don't want to read all these mundane sentences twice. So we're building a plumbing system for my voice: the app has a mode where I choose whether I'm recording for TTS or ASR, the clips are routed into the right datasets, and training runs on the right clips.

41:19 Desktop: shortcuts and the macro pad

Finally, the UI itself, which is where the actual instructions for Claude begin. Here's what I'm thinking after a long time using voice tech full time.

On the desktop, I think a macro pad is the best way to go. The simplest pattern is push-to-talk, or what I call tap-tap, and that works. But you also need buttons for stopping and retaking, which becomes evident when you use it all day, every day. I've narrowed it down to about six buttons, more than a traditional USB foot pedal provides: things like cancel, eject and retake.

Claude pointed out today, very intelligently, that the best design is for all of these, including escape, to be mappable onto custom keys, so you never conflict with global shortcuts. That's a big case for a macro pad. With system-level speech to text, which this is, rather than application-scoped, you have a greater chance of keyboard shortcut conflicts.

I usually use F13, because it's recognized as a valid key, and I map my USB devices onto it. In tap-tap mode I hit a little USB button, a $5 USB HID button from AliExpress that sends F13. So I think the settings should offer a basic setup, with Enter and Escape because that's easier on a laptop, and an advanced shortcut-binding option, so someone with dedicated hardware like a macro pad can remap everything onto whatever keys they want.

43:49 Android: Bluetooth and one button

On Android, Bluetooth support is very important. In a noisy environment I use a Bluetooth microphone, and we added the ability to change the microphone input source, which is key.

A button binding would also help, and this is tricky. I have a Yealink Bluetooth headset with one button, and on Android you can use an interceptor tool to intercept and map it. Things like a dedicated Bluetooth remote are probably the closest equivalent to a macro pad on Android. But for a first implementation, assume the user has one button, and bind it to start and stop recording. That's the fundamental if there's nothing else. I think it has to be tap-tap, because holding down a button on an earphone would look very weird. The fuller implementation can come later.

45:50 Usage patterns, streaming and Shakshuka

I'd like to document what you could call canonical usage patterns and map them onto models, because each one favors a different model. One of the experimental datasets, by the way, is non-diarized speech: speech the model can tell wasn't intended for transcription.

Typing at the cursor

The first usage pattern is typing into Claude Code, using text injection at the cursor on Ubuntu. We added a few typing-speed settings: normal, slower, slowest. The most impressive implementation I've seen is actually the one in Claude Code, which I think uses Deepgram under the hood. It streams, and when you finish a sentence and it decides what it previously wrote was wrong given the new context, it rewrites the earlier text on the fly. It looks like magic.

I asked Claude what this technology is: just a streaming model, or an additional construct? It said it's an additional construct, but the building block is a streaming model. That's when I realized this is the reason for the streaming model based on Nemotron: Shakshuka.

Shakshuka

Shakshuka is the third model, the streaming fine-tune, and it's the kitchen-sink version. Another naming convention could be Jachnun streaming and Sabich streaming, but for now it's Shakshuka, the third member of the family, just for streaming. We might revisit that, but I feel the difference is significant enough to keep it as a separate model.

So there's currently no code-switching-only Shakshuka. There could be a Shakshuka-all and a Shakshuka code-switching, a Nemotron variant trained only on that dataset, and that may be the better way to go down the line. Right now Shakshuka is everything, or as many things as can be included in that baseline.

Setup and training together

For typing at the computer, the default would use Jachnun. [The recording says Jachnun here; the surrounding context is the streaming model. Still to be confirmed.] That also changes setup. Right now it's "choose your model." I think the better presentation is "download models": we can still recommend models for your hardware, but the baseline assumption, on the desktop at least, is that people want both a streaming and a non-streaming model.

We should also keep the trainings in parallel. I'm experimenting at the moment, and that creates divergence. Whenever I have a significant amount of new training data, there should be a much more standardized process: a defined next version, released for Jachnun, Sabich and Shakshuka at about the same time on the same dataset, so they all stay up to date. My personal tech-only model can be retrained at the same time for convenience, but it doesn't have to share the release cycle.

50:43 Long-form transcription

The second mode is where transcription and dictation need careful thought, because you can do two very different things. One of them is sitting down to record something long, say a 15-minute note.

When I'm streaming into Claude Code and I have a big note, like this one, I don't want it just appearing there. That's where I lose confidence: there's a limit on input text in Claude Code in the terminal. If I'm spending an hour explaining all the ins and outs of an idea, I want to be confident that the audio, which is what's valuable, won't be lost. That's why I'm recording this on my phone in a sound recorder app, which I do habitually.

Two interfaces that make sense

• Long-form transcription in the app. This is where model matching really matters: by default it locks onto the Parakeet fine-tune, and all the inference scaffolding around it is optimized for reliably capturing notes. The app could say "up to 90 minutes," or whatever maximum can be supported. That requires chunking and local storage, and offline it's not trivial. At that length you also need to think about API limits, upload limits, and doing things like VAD and downsampling as you capture the chunks. So the architecture is different.
• File-based transcription. Almost cleaner: you upload a file and it runs inference over it.

History as a source of improvement

Today we added local storage for the dictation history. Ideally it would support the cloud as well. I also want a way of seeding words.

More of a project note: I'm going to mine my own dictation history, thousands of words a day at a minimum, to find what the model still gets wrong and pinpoint words for the training dataset. The history becomes a source of iterative improvement. Maybe in future people could share anonymized histories of their own. Probably no one will want to, but it could be an option. It's easy for me to see as I use the tool: great, it gets this right now; why is it still misspelling that? This should make that much easier.

A dictation mode that saves itself

Right now, when we transcribe in the app, it adds to the history and puts the text on the clipboard. That's the text-injection workflow. For long form, the ideal would be something more like a document editor, but really a self-standing module. You go into dictation mode, and while it's running you can choose a file or send a recording. It transcribes and saves automatically. In future, with the cloud backup I want to add, I'll use it with my own account as a beta test, and potentially users could sign up for it as a SaaS tool. For now it's for my own testing.

The ideal UI for me transcribes and auto-saves, because the first concern is always preventing data loss. Once it's auto-saved, I might tidy it up a bit and hit save. One idea from earlier: automatically create a new Google Doc from the transcription after you give it a name, like the Voicenotes feature.

56:15 Clipboard, text injection and output modes

I think the default should be recording and putting the text on the clipboard. In a previous implementation on Ubuntu, the default put it on the clipboard and injected it. The clipboard is a backup: if you rely on text injection and accidentally move your mouse outside the target, you can lose the text. What I've seen on Android, and I think it's in this app, is that if the transcription comes back and you weren't in a text field, the overlay shows a little text-insertion chip so you can pop it in.

I also added either-or modes: clipboard only, and text injection only. I don't actually see uses for them. In general I'd like a fairly deterministic philosophy, because too many options for every little thing can be distracting and unhelpful. So for the moment I'd make an executive call: we always use the clipboard as a backup.

One second, I'm going to retract that, which is why this voice note is different. We should actually have all the modes, and here's why. If there's a searchable history of your recent transcriptions, showing the last ten, you can always copy from the app, so that's a reliable fallback. And the clipboard can become annoying: you start dictating, copy something from another app, and before you paste it the transcription comes back and overwrites your clipboard. So there's an edge case where people might want the deterministic setting, or some detection for that race condition: the clipboard has just been filled, so don't overwrite it, and show a notification saying "transcription held back," or something like that. That's an idea for the future.

59:41 Storage, recordings and history

On desktop these are definitely viable. On Android I'm not sure how far a fully offline transcription mode can go, or whether we have the local storage for it. I know the all-files access permission is a blocker for Google Play, so I don't know if the default permissions allow reliable local storage. I think they do, in which case it could just save locally, but on a phone you're more constrained.

On audio recordings, my strong preference is that they're never retained. The only retention, on any platform, is for failure prevention: if transcription fails, keep it. Maybe keep the last three, if a fallback ever needs a few. Otherwise it takes space, and it makes me uncomfortable to have voice recordings sitting around. Eventually the history should have settings for truncation too.

That's the project outline, the next, somewhat jumbled, sprint. On desktop, the UI needs to be closely integrated with the system tray icon. For the text-editor mode for long-form dictation, there may be existing components we can use; the ideal is a very text-editor-like experience.

1:01:38 Voice activity detection and personal dictionaries

Two architectural things to tag on, to get everything out in one go.

First, accessory layers like VAD. I'm not sure if it can be implemented on Android. If it's there on desktop, should it be a toggle? Fast inference is key.

Second, a personal dictionary, which is an interesting layer. Android exposes the personal dictionary quite liberally, and I've always been an advocate of using these systems. Most users aren't going to create their own fine-tunes, so my method of baking my dictionary into my own model isn't replicable for most people.

I'm not sure how it would be integrated, or whether it could be part of the inference pass. On Android we'd integrate with the user's personal dictionary, or offer the option; then on the desktop, sync across devices. That's where cloud integration would be brilliant, because it's a huge productivity accelerator. One failure I've seen: if you use these tools every day, you add words on one app on Android, move to another computer, find they aren't there, create them again, and they're never quite in sync.

1:03:16 Report a word, and more voices

If anyone actually starts using this, which would be great, a brilliant feature that fits my workflow would be "report a word that didn't get transcribed correctly." Say you dictated a brand name and it didn't work: you send just the word, or the word and a note. That could feed a user-submitted corrections dataset, piped into my recording app so I can record it.

The final thing is the recording app. I'm sure I can find a few friends happy to record some sentences in different accents. Hannah has recorded a few, with a little manual tagging. My general principle is that the priority is always gathering more training data, and fixing up the annotations later, before the next training run.

So ideally the training app has multi-tenancy: a few users, and whoever's logged in is the annotator. Then we tag that speaker with their accent, and maybe a short bio for credits as a contributor. We could probably build that in now, because I'm the only user. I think I have a manual drop-down for Hannah, but it would be better to add her address to the allowlist so her speaker profile gets added: gender female, US accent, from Dallas, in Israel for ten years. The useful part is that the dataset could then be stratified by accent: how much is from female speakers, from code switching, and so on.

The quick fix for now, for the day a friend shows up at my apartment and says "the app is great, can I record a few sentences?": I'm not going to create a profile on the fly and get on to Claude. So as a fallback, a custom entry where I can add a name, accent, gender and short description, to add them to the annotation pool. I can work out logins later if needed. That's good enough for now, because it would be fun to go out into the field and ask people to read a few sentences. It's free, and they'd be helping English speakers get better dictation.

1:06:19 Sign-off

All right, I have to go pick up the little one very shortly, so that's my brain dump: all the ideas, all the progress updates. If you want to get in touch, my name is Daniel Rosehill. My personal site is danielrosehill.com, I have an AI consultancy, Carrot Cake AI, as in carrot cakes, and you can reach me through hebrish.app: H-E-B-R-I-S-H.

Oh, one final thing: a small dataset of our own identity. "hebrish.app" should be in it, and "Hebrish" should be in it, because ironically that doesn't get recognized currently. And the general principle: all the datasets and models for the project are in the Hugging Face organization, and all the GitHub repos are in the GitHub organization, not my personal account, to keep it properly segregated.

Okay, I really have to go. Over and out from Hebrish. Daniel Rosehill, progress update and vision for the next sprint, late October 2026.

Corrections after recording

• Desktop output mode (typed the same day): the output-mode toggle already exists. Make it easier to change: put it in the system tray icon's menu, and add shortcut bindings for toggling the output mode. Desktop only.