I've been experimenting with two ways to make HAL talk: a local pipeline with PocketTTS, and OpenAI's GPT Live.
Kyutai's PocketTTS lets me clone HAL's voice from a reference recording. A companion computer transcribes what I say, generates a reply, then turns that text into speech for the ESP32. I can control the voice separately from the model writing the answer, but I have to connect and coordinate each stage.
The other path uses GPT Live, released in the API on September 10, 2026. It handles listening and speaking together, with a backend model for deeper reasoning. My setup uses its Meridian voice, prompted to speak with HAL's calm, measured delivery.
The catch is voice cloning. As of writing this, custom voices require separate access, so I can't simply give Live a HAL clip as I do with PocketTTS. A voice conversion step between Live and the speaker might get closer to HAL's sound. I'd need to test how much delay it adds and whether it preserves the timing and expression.