Geminiero: Multimodal AI Inside a Discord Server
Four Gemini capabilities (chat, image generation, speech and live voice) behind one bot, reachable by slash command or a single-character prefix. The hard part is the voice loop: Discord speaks Opus at 48 kHz stereo, Gemini Live speaks PCM at 24 kHz mono, and neither waits for the other.
- Bidirectional live voice with VAD turn-taking
- 4 capabilities · 2 interfaces each
- Context persisted in SQLite (WAL)
- TypeScript
- Bun
- discord.js
- @google/genai
- SQLite
- Opus
- 21:14:02 DEBUG Live API opened
- 21:14:02 DEBUG setup completed (from server)
- 21:14:05 DEBUG VAD: speech started.
- 21:14:07 DEBUG VAD: speech ended.
- 21:14:07 DEBUG activity end sent
- 21:14:08 DEBUG Starting discord audio stream
- 21:14:11 DEBUG turn complete
Voice & Generation Bridge. Chat + Grounding: 60-msg context. Live Voice Loop: 24 kHz → 48 kHz. Speech Synthesis: 24 voices.
The problem
Using a general-purpose AI from inside a community means leaving it: copy the question out, paste the answer back. The context dies with the tab, and voice is not part of the conversation at all.
The approach
Geminiero puts four Gemini surfaces behind one bot. Text chat keeps the last sixty messages per
channel in SQLite, so the model answers with the thread’s history rather than a blank slate, and
returns source citations when grounding fires. Every capability is reachable twice, as a slash
command for discoverability, and as a one-character prefix (*, +, =, .) for people who
already know what they want.
Live voice is the part that had to be engineered rather than configured. Discord delivers Opus at 48 kHz stereo; the Gemini Live API expects raw PCM at 24 kHz mono and answers in the same shape. The bot decodes, downsamples, streams, then upsamples and re-encodes on the way back, with voice activity detection deciding when a turn has ended instead of a push-to-talk key.
The outcome
A conversation that never leaves the server, in text, image, speech or voice, and a codebase where adding a fifth capability means adding one command file, because commands and events are discovered from the filesystem at boot.