---
title: "Audio (Speech & Transcription)"
---

> Documentation Index
> Fetch the complete documentation index at: https://truememory-docs.mvstg.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Audio (Speech & Transcription)

Truememory provides OpenAI-compatible audio endpoints for speech-to-text (transcription) and text-to-speech (TTS). These are drop-in replacements for OpenAI's `/v1/audio/transcriptions` and `/v1/audio/speech` — any client that works with OpenAI's audio API works with Truememory by changing the base URL.

:::note[Model ID format on this endpoint]
Audio endpoints accept bare OpenAI IDs (`whisper-1`, `gpt-4o-mini-tts`) and `google/`-prefixed Gemini IDs (`google/gemini-2.5-flash`). Note this differs from chat, where Gemini uses the `vertexai/` prefix — see [Model ID formats](/ai-gateway/overview/#model-id-formats).
:::


:::tip[Looking for Gemini-native voice?]
This page covers the OpenAI-compatible file-based endpoints. For real-time voice conversations, Gemini TTS, and WebSocket STT — see [Real-Time Voice](/ai-gateway/voice/).
:::

All requests require a Bearer token from [Truememory Auth](/authentication/overview/). Both `/v1/audio/transcriptions` and `/v1/audio/speech` accept tri-mode auth — see [Ways to authenticate](/authentication/overview/):

| Caller | Method |
|--------|--------|
| Browser / mobile app | User token: `Authorization: Bearer <user-jwt>` |
| Backend service (BlinkLife / Vibrantly) | Service key on user's behalf: `Authorization: Bearer $AUTH_SECRET` + `X-On-Behalf-Of: <user-uuid>` |
| Backend service (stateless) | Service key only: `Authorization: Bearer $AUTH_SECRET` |

The audio endpoints are stateless (no per-user memory or thread writes), so the service-only mode is allowed for both — unlike the WebSocket voice endpoints, which require `X-On-Behalf-Of` because they touch user state.

## Speech-to-Text (Transcription)

Convert audio to text. Truememory proxies to OpenAI Whisper and returns the transcript.

### Endpoint

```
POST /v1/audio/transcriptions
```

### Parameters

Send as `multipart/form-data`:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file` | file | required | Audio file (mp3, mp4, mpeg, mpga, m4a, wav, webm) |
| `model` | string | `whisper-1` | Transcription model. Use `google/gemini-2.5-flash` for Gemini-powered transcription. Files capped at 25 MB. |
| `language` | string | null | ISO 639-1 language code (e.g. `en`). Improves accuracy if known. |
| `prompt` | string | null | Guide the model's style or continue a previous segment |
| `response_format` | string | `json` | Output format: `json`, `text`, `srt`, `verbose_json`, `vtt` |
| `temperature` | float | 0.0 | Sampling temperature (0–1). Lower is more deterministic. |

### Example

```bash
curl -X POST https://your-tie-host/v1/audio/transcriptions \
  -H "Authorization: Bearer $TOKEN" \
  -F file=@recording.webm \
  -F model=whisper-1
```

### Response

Default (`json` format):

```json
{
  "text": "Hello, I'd like to log my breakfast. I had oatmeal with blueberries."
}
```

With `response_format=verbose_json`, the response includes timestamps and segment-level detail.

### AI SDK Integration

The [Vercel AI SDK](https://sdk.vercel.ai/docs) supports transcription via `experimental_transcribe`. Point the OpenAI provider at your Truememory instance:

```typescript
import { createOpenAI } from "@ai-sdk/openai";
import { experimental_transcribe as transcribe } from "ai";

const tie = createOpenAI({
  baseURL: "https://your-tie-host/v1",
  apiKey: "$TOKEN",
});

const result = await transcribe({
  model: tie.transcription("whisper-1"),
  audio: audioBuffer, // Uint8Array, Buffer, base64 string, or URL
});

console.log(result.text);
// "Hello, I'd like to log my breakfast."
```

---

## Text-to-Speech

Convert text to spoken audio. Truememory proxies to OpenAI's TTS models and streams audio bytes back.

### Endpoint

```
POST /v1/audio/speech
```

### Parameters

Send as JSON:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model` | string | `gpt-4o-mini-tts` | TTS model (`gpt-4o-mini-tts`, `tts-1`, `tts-1-hd`) |
| `input` | string | required | Text to convert to speech (max 4096 characters) |
| `voice` | string | `alloy` | Voice: `alloy`, `ash`, `ballad`, `coral`, `echo`, `fable`, `nova`, `onyx`, `sage`, `shimmer`, `verse` |
| `response_format` | string | `mp3` | Audio format: `mp3`, `opus`, `aac`, `flac`, `wav`, `pcm` |
| `speed` | float | 1.0 | Speed multiplier (0.25–4.0) |

### Example

```bash
curl -X POST https://your-tie-host/v1/audio/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "model": "gpt-4o-mini-tts",
    "input": "Great job logging your meals today! You hit your protein target.",
    "voice": "nova"
  }' \
  --output response.mp3
```

### Response

Raw audio bytes in the requested format. The `Content-Type` header reflects the format (e.g. `audio/mpeg` for mp3).

### AI SDK Integration

```typescript
import { createOpenAI } from "@ai-sdk/openai";
import { experimental_generateSpeech as generateSpeech } from "ai";

const tie = createOpenAI({
  baseURL: "https://your-tie-host/v1",
  apiKey: "$TOKEN",
});

const result = await generateSpeech({
  model: tie.speech("gpt-4o-mini-tts"),
  text: "Great job logging your meals today!",
  voice: "nova",
});

// result.audio is a Uint8Array of audio bytes
```

---

## Gemini TTS (same endpoint, different model)

`/v1/audio/speech` routes to Gemini when `model` is a Gemini TTS variant — same pattern as `/v1/audio/transcriptions` switching between Whisper and Gemini. No second endpoint to learn. For real-time conversational voice, see [Real-Time Voice](/ai-gateway/voice/).

> **Note:** Google's [OpenAI compatibility layer](https://ai.google.dev/gemini-api/docs/openai) does not expose a TTS endpoint — only audio *understanding* (input). The Gemini path on `/v1/audio/speech` is Truememory's own OpenAI-shaped facade that calls Gemini's native SDK (`google-genai` with `response_modalities=["AUDIO"]`). Callers see a unified API; Truememory handles the bridging.

```bash
POST /v1/audio/speech
Authorization: Bearer <token>
Content-Type: application/json
```

```json
{
  "model": "google/gemini-2.5-flash-tts",
  "input": "Great job today!",
  "voice": "Kore",
  "language": "en",
  "response_format": "pcm"
}
```

### Differences vs OpenAI TTS

| | OpenAI | Gemini |
|--|--|--|
| `model` | `gpt-4o-mini-tts`, `tts-1`, `tts-1-hd` | `google/gemini-2.5-flash-tts` |
| `voice` | `alloy`, `nova`, `coral`, etc. (see [OpenAI Voices](#openai-voices)) | `Kore`, `Aoede`, `Puck`, etc. (see [Gemini Voices](#gemini-voices)) |
| `language` | Auto-detected from input | Required — BCP-47 code (`en`, `ms`, `es`) |
| `response_format` | `mp3` / `opus` / `aac` / `flac` / `wav` / `pcm` | `pcm` / `wav` only — others return `400 VALIDATION_ERROR` |
| `speed` | 0.25–4.0 | Ignored |

### Example

```bash
# Raw PCM — 24 kHz signed 16-bit little-endian mono
curl -X POST https://your-tie-host/v1/audio/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "model": "google/gemini-2.5-flash-tts",
    "input": "Great job today!",
    "voice": "Kore"
  }' \
  --output response.pcm

# WAV — playable by any audio player without client-side transcoding
curl -X POST https://your-tie-host/v1/audio/speech \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "model": "google/gemini-2.5-flash-tts",
    "input": "Great job today!",
    "voice": "Kore",
    "response_format": "wav"
  }' \
  --output response.wav
```

### Response

`pcm` (default): raw PCM — **24 kHz, signed 16-bit little-endian, mono**. `Content-Type: audio/pcm;rate=24000`.

`wav`: same PCM wrapped in a standard 44-byte WAV header. `Content-Type: audio/wav`. Playable directly in browsers and mobile apps without transcoding.

#### Usage headers

Every successful Gemini TTS response includes token-usage headers so callers can record billing data without inspecting the audio body:

| Header | Source |
|--|--|
| `X-Usage-Prompt-Tokens` | `usage_metadata.prompt_token_count` |
| `X-Usage-Completion-Tokens` | `usage_metadata.candidates_token_count` |
| `X-Usage-Total-Tokens` | `usage_metadata.total_token_count` |

Headers are always present — they default to `"0"` when the SDK omits `usage_metadata`. The OpenAI TTS path (`gpt-4o-mini-tts` etc.) does not emit these headers.

---

## Common Pattern: Voice Chat

A typical voice-enabled chat flow combines both endpoints:

1. **User speaks** → record audio on the client
2. **Transcribe** → `POST /v1/audio/transcriptions` → get text
3. **Chat** → `POST /v1/chat/completions` with the transcribed text → get AI response
4. **Speak** → `POST /v1/audio/speech` with the AI response text → play audio

```mermaid
sequenceDiagram
    participant User
    participant Client
    participant Truememory

    User->>Client: Speaks
    Client->>Truememory: POST /v1/audio/transcriptions (audio file)
    Truememory-->>Client: { "text": "..." }
    Client->>Truememory: POST /v1/chat/completions (transcribed text)
    Truememory-->>Client: AI response text
    Client->>Truememory: POST /v1/audio/speech (response text)
    Truememory-->>Client: Audio bytes
    Client->>User: Plays audio
```

Steps 3 and 4 can overlap — start TTS as soon as the chat response text is available, rather than waiting for the full response.

## Models

### Transcription

| Model | Description |
|-------|-------------|
| `whisper-1` | OpenAI Whisper — general-purpose, supports 50+ languages |
| `google/gemini-2.5-flash` | Gemini 2.5 Flash — accurate, supports audio/webm and other browser formats |

### Text-to-Speech

| Model | Description |
|-------|-------------|
| `gpt-4o-mini-tts` | Latest, most natural-sounding. Default. |
| `tts-1` | Optimized for low latency |
| `tts-1-hd` | Optimized for quality |

### OpenAI Voices

| Voice | Tone |
|-------|------|
| `alloy` | Neutral, balanced |
| `ash` | Warm, conversational |
| `ballad` | Soft, expressive |
| `coral` | Clear, friendly |
| `echo` | Steady, authoritative |
| `fable` | Expressive, storytelling |
| `nova` | Warm, engaging |
| `onyx` | Deep, rich |
| `sage` | Calm, measured |
| `shimmer` | Bright, upbeat |
| `verse` | Versatile, adaptive |

## Gemini Voices

Used with `POST /v1/audio/speech` (when `model=google/gemini-2.5-flash-tts`) and the real-time voice agent.

| Voice | Character |
|-------|-----------|
| `Kore` | Firm, grounded |
| `Puck` | Playful, upbeat |
| `Charon` | Calm, measured |
| `Fenrir` | Confident, direct |
| `Aoede` | Smooth, warm |
| `Leda` | Clear, friendly |
| `Orus` | Steady, authoritative |
| `Zephyr` | Light, energetic |

Full catalog: [Gemini voice options](https://cloud.google.com/vertex-ai/generative-ai/docs/live-api/voice-options)

Source: https://truememory-docs.mvstg.com/ai-gateway/audio/index.mdx