---
title: "Unified API (OpenAI compat)"
---

> Documentation Index
> Fetch the complete documentation index at: https://truememory-docs.mvstg.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Unified API (OpenAI compat)

Truememory AI Gateway offers an OpenAI-compatible `/v1/chat/completions` endpoint, enabling integration with multiple AI providers using a single URL. This is a drop-in replacement for OpenAI, Cloudflare AI Gateway, or LiteLLM — change the base URL and it works.

## Authentication

All requests require a Bearer token in the `Authorization` header. See [Ways to authenticate](/authentication/overview/) for how to get one.

| Caller | Method | Details |
|--------|--------|---------|
| Browser / mobile app | [User tokens](/authentication/email-password/) | Sign in via Truememory Auth, pass the JWT |
| Backend service | [Service accounts](/authentication/service-accounts/) | Use `AUTH_SECRET`, optionally with `X-On-Behalf-Of` for user context |

## Endpoint URL

```
POST /v1/chat/completions
```

Switch providers by changing the `model` parameter. For example: `anthropic/claude-sonnet-4-5`, `openai/gpt-5.1`, `vertexai/gemini-2.5-flash`.

## Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model` | string | required | Model to use (e.g. `anthropic/claude-sonnet-4-5`, `openai/gpt-5.1`) |
| `messages` | array | required | Conversation messages in OpenAI format. Each message's `content` can be a plain string or an array of content parts mixing text and images — see [Vision](/ai-gateway/features/vision/). If the array includes a `system` or `developer` message, it replaces the agent's built-in prompt for that request (memory and personas are still injected). Each message also accepts an optional `cache_control` field — see [Prompt caching](/ai-gateway/features/prompt-caching/). |
| `tools` | array | null | Tool definitions in OpenAI function calling format. See [Tool calling](/ai-gateway/features/tool-calling/). |
| `stream` | boolean | false | Enable SSE streaming |
| `stream_options` | object | null | Streaming options. Set `{"include_usage": true}` to receive a final chunk with token usage (only applies when `stream` is `true`). |
| `temperature` | float | null | Sampling temperature. Ignored on `anthropic/claude-opus-5-5`, `anthropic/claude-sonnet-5-5` and `anthropic/claude-haiku-5-5`, which do not accept sampling parameters. |
| `max_tokens` | integer | null | Maximum tokens to generate |
| `top_p` | float | null | Nucleus sampling parameter. Ignored on `anthropic/claude-opus-5-5`, `anthropic/claude-sonnet-5-5` and `anthropic/claude-haiku-5-5`. |
| `stop` | string/array | null | Stop sequences |
| `thread_id` | string | null | Conversation thread for state persistence (see [Threads](#threads)) |
| `parent_message_id` | string | null | Id of the last stored message the client received on this thread (the `message_id` of the previous response). When set, anything already stored after it is not appended again, so a failed request can be re-sent unchanged. `messages` holds the messages after it, and may also start with history that ends at it; that history is dropped exactly as it is without this field. Ignored without `thread_id`. See [Retrying a failed turn](/ai-gateway/threads/#retrying-a-failed-turn). |
| `store` | boolean | `true` | When `true` (default), the call is recorded in the conversation thread and shows up in `/threads`. Set to `false` for one-off agent calls — for example, a background helper that should run the agent without leaving a trace in the user's chat history. |
| `memory` | object | null | Per-request memory controls. `{"read": true, "write": false}` lets the agent recall facts but skips saving anything new from this turn; both default to `true` on user-scoped requests. The thread itself is still recorded either way (so it still appears in `/threads`) — only the memory graph writes are suppressed. |
| `persona_id` | string | null | Persona to use for this request. Falls back to the user's [active persona](/ai-gateway/personas/) if omitted. |
| `context_management` | array | null | Context-management strategies for this request (OpenAI-compatible). Include a `{"type": "compaction", ...}` entry to fold older thread history into a rolling summary checkpoint once it crosses a token threshold. See [Thread compaction](/ai-gateway/features/thread-compaction/). |
| `metadata` | object | null | Up to 16 key-value string pairs attached to this request. Persisted with the thread and returned in history. Echoed in the SSE first chunk when streaming. See [Custom metadata](/ai-gateway/features/custom-metadata/). |
| `skill_instructions` | string | null | Caller-composed skill prompt. Injected after persona, before memory context. Purely additive — never replaces persona or base instructions. See [Skill instructions](/ai-gateway/features/skill-instructions/). |
| `knowledge_pack_ids` | string[] | null | Ground the answer in specific uploaded documents. See [Knowledge Packs](/ai-gateway/knowledge-packs/#retrieval). |
| `skill_id` | string | null | Ground the answer in every knowledge pack tagged with this [skill](/ai-gateway/skills/). Can be combined with `knowledge_pack_ids`. See [Knowledge Packs — Retrieval](/ai-gateway/knowledge-packs/#retrieval). |
| `brain_id` | string | null | Ground the answer in every pack in a [brain](/ai-gateway/brains/) (a named collection of packs). Singular alias for `brain_ids`; send both and their union is resolved. Ignored when `knowledge_pack_ids` is also provided. Resolved on the memory path — requires a memory-enabled agent and memory reads on (see [Brains](/ai-gateway/brains/)). |
| `brain_ids` | string[] | null | Ground the answer in several [brains](/ai-gateway/brains/) at once. Their packs merge into one deduplicated set. Every ID must be readable by the caller — an unreachable one returns `404 BRAIN_NOT_FOUND`. |
| `graph_ids` | string[] | null | Blend one or more [graph namespaces](/memory/graph-namespacing/) (shared knowledge graphs) into this request's memory retrieval, alongside the user's own memory. |

## Custom Prompt Mode

If you want to replace the agent's built-in prompt with your own instructions, send a `system` or `developer` message in the `messages` array.

When Truememory sees one of those roles, your message **replaces** the agent's base instructions for that request:

- your `system` or `developer` message becomes the top-level instruction instead of the agent's built-in prompt
- Truememory **still injects** persistent memory context so the model knows what it remembers about the user
- Truememory **still injects** the active persona (if any)
- thread history and turn recording **still work** normally
- client-defined tools in the `tools` parameter still work

This is useful when you want full prompt control for a specific request while keeping Truememory's memory and conversation features.

### Example: caller-owned prompt

```bash
curl -X POST https://your-tie-host/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "model": "openai/gpt-5.1",
    "messages": [
      {"role": "developer", "content": "You are a concise support assistant. Answer in 3 bullet points max."},
      {"role": "user", "content": "How do I rotate an API key?"}
    ]
  }'
```

## Examples

A minimal request — memory and the active persona are injected automatically, nothing extra to send:

```bash
curl -X POST https://your-tie-host/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5",
    "messages": [{"role": "user", "content": "What did I say I was working on?"}]
  }'
```

With `"stream": true`, tokens arrive as SSE chunks in the standard OpenAI shape — content lives in `choices[0].delta.content`, and the stream ends with a `finish_reason` chunk followed by `data: [DONE]`:

```
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1711234567,"model":"anthropic/claude-sonnet-4-5","choices":[{"index":0,"delta":{"content":"Hel"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1711234567,"model":"anthropic/claude-sonnet-4-5","choices":[{"index":0,"delta":{"content":"lo!"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1711234567,"model":"anthropic/claude-sonnet-4-5","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]
```

Use the interactive picker below for per-language variants (OpenAI SDKs, Vercel AI SDK, cURL) and image inputs:

### Response

```json
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1711234567,
  "model": "anthropic/claude-sonnet-4-5",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 8,
    "total_tokens": 20,
    "cache_read_tokens": 0,
    "cache_creation_tokens": 0
  },
  "metadata": null
}
```

If `metadata` was included in the request, it is echoed back here.

### Understanding the `usage` object

Every time you call the AI, it counts the pieces of text (called **tokens**) that go in and out. You pay your provider for these tokens, so the `usage` object is how you know what just got spent.

Think of it like a taxi meter: `prompt_tokens` is how far you rode in, `completion_tokens` is how far the AI drove back, and `total_tokens` is the full fare.

| Field | What it means (in plain English) |
|-------|-----------------------------------|
| `prompt_tokens` | How much **you** sent in (your messages + system prompt + memory + persona). Bigger question = bigger number. |
| `completion_tokens` | How much the **AI** wrote back. A one-word answer is tiny. A long essay is big. |
| `total_tokens` | Simply `prompt_tokens + completion_tokens`. The whole bill. |
| `cache_read_tokens` | Of your `prompt_tokens`, how many were **reused from a cached copy** the provider had already saved. These are WAY cheaper — Anthropic charges up to **10× less** for cache reads. |
| `cache_creation_tokens` | How many tokens the provider **saved into cache** for next time. Slightly pricier than a normal token, but it makes future calls much cheaper. |

:::tip[Why cache tokens matter]
If you keep sending the same long system prompt or the same big conversation history, Anthropic and OpenAI can cache it. The next time you ask, the cached part is nearly free. These fields let you see that savings (or the one-time cost of creating the cache) so you can calculate your real bill. See [Prompt caching](/ai-gateway/features/prompt-caching/) for how to control it.
:::

:::note[When these are zero]
Most providers (Gemini, Groq, Mistral, etc.) don't support prompt caching, so `cache_read_tokens` and `cache_creation_tokens` will always be `0` for them. OpenAI reports cache reads but not cache creation, so `cache_creation_tokens` is always `0` for OpenAI calls. That's not a bug — there's just nothing to report.
:::

## Features

Request-level capabilities — each has its own page under [Features](/ai-gateway/features/overview/):

- [Prompt caching](/ai-gateway/features/prompt-caching/) — reuse a stable prompt prefix at up to 10x lower cost
- [Vision (image input)](/ai-gateway/features/vision/) — send images via content parts
- [Tool calling](/ai-gateway/features/tool-calling/) — client-defined functions the model can call
- [Custom metadata](/ai-gateway/features/custom-metadata/) — key-value pairs persisted with the thread
- [Message compression](/ai-gateway/features/message-compression/) — summarise long histories to save tokens
- [Skill instructions](/ai-gateway/features/skill-instructions/) — additive caller-composed skill prompts

## Threads

Pass a `thread_id` to maintain conversation state across requests. Truememory uses it to persist the message history so the LLM has context from previous turns.

- If **omitted**, Truememory auto-generates a UUID for the thread. Each request starts a new conversation.
- If **provided**, Truememory resumes the conversation from where it left off. Use any string (e.g. a UUID you generate client-side).

See [Threads](/ai-gateway/threads/) for the full API reference (list, rename, delete, history).

## Streaming Usage

When streaming (`stream: true`), token usage is available by setting `stream_options.include_usage` to `true`.

```bash
curl -X POST https://your-tie-host/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5",
    "stream": true,
    "stream_options": {"include_usage": true},
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'
```

When enabled, Truememory emits a final SSE chunk after the `finish_reason` chunk with an empty `choices` array and the accumulated `usage`:

```
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1711234567,"model":"anthropic/claude-sonnet-4-5","choices":[],"usage":{"prompt_tokens":9,"completion_tokens":12,"total_tokens":21,"cache_read_tokens":0,"cache_creation_tokens":0}}

data: [DONE]
```

The final `usage` chunk includes the same fields as the non-streaming response, including `cache_read_tokens` and `cache_creation_tokens`. Numbers are accumulated across every internal step (tool calls, memory lookups, etc.) so you get one clean bill for the whole request. See [Understanding the `usage` object](#understanding-the-usage-object) for what each field means.

If `stream_options` is omitted or `include_usage` is `false`, no usage chunk is emitted.

## Agent Selection

By default, requests use the `chatbot` agent. To use a different agent, pass the `X-Agent-Id` header:

```bash
curl -X POST https://your-tie-host/v1/chat/completions \
  -H "X-Agent-Id: research-assistant" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"model": "anthropic/claude-sonnet-4-5", "messages": [...]}'
```

If the request includes a `system` or `developer` message, that message replaces the agent's built-in prompt for that request. Memory and personas are still injected.

## Per-App Thread Isolation

To scope threads to a specific application, pass the `X-App-Id` header. Threads sent with an app ID are isolated — they won't appear in other apps' thread lists, and can only be resumed, renamed, or deleted with the same `X-App-Id`.

```bash
curl -X POST https://your-tie-host/v1/chat/completions \
  -H "X-App-Id: vibrantly" \
  -H "X-Agent-Id: chatbot" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"model": "anthropic/claude-sonnet-4-5", "messages": [{"role": "user", "content": "Hello"}], "thread_id": "my-thread"}'
```

When `X-App-Id` is omitted, threads are unscoped and visible to all apps. See [Threads — Per-App Thread Isolation](/ai-gateway/threads/#per-app-thread-isolation) for details.

For the full agent registry (memory scope and safety behavior per agent), see [Agents](/ai-gateway/agents/#available-agents) — the single source of truth. Query `GET /info` to see the agents on your instance.

## Supported Providers

- [Anthropic](https://docs.anthropic.com/)
- [OpenAI](https://platform.openai.com/)
- [Google Vertex AI](https://cloud.google.com/vertex-ai)
- [Z.ai (GLM)](https://docs.z.ai/) — e.g. `zai/glm-4.6`, `zai/glm-4.7`, `zai/glm-5.2`

Query `GET /info` to see which models are currently available on your instance.

Source: https://truememory-docs.mvstg.com/ai-gateway/chat-completions/index.mdx