Reference

Configuration Reference

The entire stack is controlled by a single config.tomlfile. All settings have sensible defaults — you only need to set API keys and choose your providers.

Full Configuration Example

config.toml

[server]
port = "8080"              # Server listen port
public_ip = ""             # Public IP for ICE/STUN (required for TURN)
turn_secret = ""           # Shared secret for built-in STUN/TURN server
jwt_secret = ""            # Optional: JWT secret for token auth
api_key = ""               # Optional: API key for /token endpoint
session_grace_ms = 30000   # How long a peerless session survives, for reconnects
max_sessions = 0           # Global cap on live sessions. 0 = unlimited

[plugins]
directory = "./plugins"    # Plugin and skills directory

[pipeline]
barge_in = true            # Allow users to interrupt the agent
greeting = ""              # Optional: greeting message on connect
debug = false              # Emit timing events over DataChannel

# Speech-to-speech. Setting a provider here replaces [stt], [llm], and [tts]
# with one model, and hands turn detection to it. Empty = classic pipeline.
[realtime]
provider = ""              # grok

[stt]
provider = "deepgram"      # aliyun | assemblyai | deepgram | openai | vibevoice | volcengine

[llm]
provider = "openai"        # openai | groq | ollama | agent

[tts]
provider = "cartesia"      # cartesia | deepgram | elevenlabs | mimo | minimax | speechify | vibevoice

# ── Provider credentials ──

[deepgram]
api_key = ""               # Deepgram API key (STT + optional TTS)

[openai]
api_key = ""               # OpenAI API key
model = "gpt-4o-mini"      # LLM model name

[groq]
api_key = ""               # Groq API key
model = "openai/gpt-oss-20b"
reasoning_effort = "none"  # reasoning off — thinking is dead air on a call

[cartesia]
api_key = ""               # Cartesia API key (TTS)

[elevenlabs]
api_key = ""               # ElevenLabs API key (TTS)

[ollama]
base_url = "http://localhost:11434"  # Ollama server URL
model = "gemma4:e4b"                 # Local model name

# Optional: answer from your own documents. Ingest them with streamcore-cli.
[rag]
provider = "supabase"      # supabase | pgvector, or omit to disable
top_k = 3

[server]

Key	Default	Description
port	`"8080"`	HTTP listen port
public_ip	`""`	Public IP for ICE candidates and STUN/TURN (required when deploying to cloud)
turn_secret	`""`	Shared secret for built-in STUN/TURN server (required when public_ip is set)
jwt_secret	`""`	If set, WHIP requests require a valid JWT bearer token
api_key	`""`	If set, the `/token` endpoint requires this key to issue JWTs
session_grace_ms	`30000`	How long a session with no connected peers is kept before it is reaped. The window is what lets a dropped client recover the same conversation by ICE restart or redial
max_sessions	`0`	Global cap on live sessions; past it `POST /whip` returns 503 with `Retry-After`. `0` is unlimited. Each session burns CPU and provider spend, so set it to what one instance can actually serve

STUN/TURN Server

When deploying to the cloud (EC2, DigitalOcean, etc.), browsers cannot directly connect to your server's private IP. Set public_ip and turn_secret to enable the built-in STUN/TURN server on port 3478.

Cloud deployment config

[server]
public_ip = "1.2.3.4"        # Your public IP
turn_secret = "your-secret"  # Shared secret for TURN authentication

The server automatically starts a STUN/TURN server when both values are set. The username is always voiceagent and the credential is your turn_secret.

Setting	Description
public_ip	Public IP address advertised in ICE candidates (required for NAT traversal)
turn_secret	Shared secret for TURN authentication. Username is always "voiceagent"

[pipeline]

Key	Default	Description
barge_in	`true`	Allow users to interrupt the agent mid-response
greeting	`""`	If set, the agent speaks this message when a session starts
greeting_outgoing	`""`	Spoken on outbound SIP calls; falls back to `greeting`
debug	`false`	Emit timing events (STT, LLM, TTS latency) over the DataChannel and log a per-turn latency breakdown
user_speech_quiet_ms	`600`	Quiet period after the caller stops before the agent speaks
turn_merge_ms	`350`	Debounce window for merging finals into one turn. Raise it if the agent answers halfway through a sentence; lower it if replies feel slow
rag_prefetch	`false`	Start retrieval during the merge window instead of after it
readback_bargein_guard_enabled	`false`	Ignore weak barge-ins while the agent is reading values back

[realtime] — speech-to-speech

Setting a provider here replaces [stt], [llm], and [tts]with a single model that hears audio and answers with audio — one key instead of three, and lower latency because there are no handoffs between stages. Leave it empty for the classic pipeline.

config.toml

[realtime]
provider = "grok"

[grok]
api_key = "xai-..."
model = "grok-voice-latest"   # tracks the newest model; pin a version in production
voice = "eve"
reasoning_effort = "high"     # "none" for lower latency
system_prompt = "You are a helpful AI voice assistant. Keep responses short and conversational."
transcription = true          # caller transcript for the client UI only; costs an extra pass

Keep the prompt short. These models are strong enough that porting a long GPT-era prompt over verbatim makes them worse, not better.

Turn detection

In this mode the model owns turn detection and barge-in, so the [pipeline] timing keys do not apply. Unset values use xAI’s defaults.

Key	Default	Use it when
vad_threshold	`0.85`	Noise, coughs, or “mm-hm” cut the agent off. Raise it (range 0.1–0.9)
silence_duration_ms	`500`	Callers get cut off mid-sentence. Raise it to allow longer pauses
prefix_padding_ms	`333`	The first word of a turn gets clipped. Raise it
idle_timeout_ms	`0`	You want the agent to re-engage after silence. `0` disables the check-in
language_hint	`""`	BCP-47 tag. Spanish and Portuguese need a region (`es-MX`, `pt-BR`)
keyterms	`[]`	Bias transcription toward domain words. Up to 100 terms
web_search	`false`	Hosted search runs on xAI’s side with no local plugin. `x_search` works the same way

Plugins, skills, and RAG still work — they are registered as function tools instead of prompt injection, and RAG becomes a knowledge_search tool the model calls on demand. Delivery tags and the plugin thinking_sound are not used, since the model controls its own prosody and audio timing.

STT Providers

Set [stt] provider to one of these values:

Deepgram Nova-3

provider = "deepgram"

Real-time streaming STT with word-level timestamps and punctuation. Recommended for lowest latency.

[deepgram]
api_key = "your-deepgram-api-key"
model = "nova-3"
language = ""              # BCP-47 (en-US, es-MX). Non-en/es routes to the multilingual model
endpointing = "300"        # silence (ms) before a transcript is finalised. Raise it if
                           # callers get cut off mid-sentence
utterance_end_ms = "1000"  # silence before an UtteranceEnd event flushes a turn when no
                           # speech_final arrives (min 1000)
keyterms = []              # Nova-3 only: bias the decoder toward domain words

OpenAI Whisper

provider = "openai"

OpenAI’s speech recognition model. Uses the same OpenAI API key.

[openai]
api_key = "your-openai-api-key"

AssemblyAI Universal-Streaming

provider = "assemblyai"

Streaming STT with its own end-of-turn model.

[assemblyai]
api_key = "your-assemblyai-api-key"
model = "u3-rt-pro"          # "u3-rt" is the cheaper baseline
language = ""                # BCP-47; region is stripped (en-NZ -> en). Empty auto-detects
# format_turns = true        # auto-punctuate and capitalise the final turn
# end_of_turn_silence_ms = 0 # override how long the model waits before ending a turn
keyterms = []                # bias the decoder toward domain words

Alibaba Cloud (DashScope)

provider = "aliyun"

Streaming ASR from Model Studio. Strong on Mandarin. Key from the Bailian console.

[aliyun]
api_key = "your-dashscope-api-key"
model = ""            # defaults to paraformer-realtime-v2; fun-asr-realtime is the alternative
language = ""         # hint for a multilingual model ("zh", "en"). Empty auto-detects
vocabulary_id = ""    # a hotword list from the console, for terms it keeps getting wrong

Volcengine (Doubao)

provider = "volcengine"

Useful where Deepgram is slow to reach or its Mandarin is not good enough. The console gives a free hourly allowance to start with.

[volcengine]
api_key = "your-console-api-key"  # sent as X-Api-Key; the old app-id + access-token pair is rejected
resource_id = ""      # defaults to volc.seedasr.sauc.duration (hourly);
                      # volc.seedasr.sauc.concurrent bills by concurrency instead
model = ""            # defaults to bigmodel
end_window_ms = 0     # silence that settles an utterance, defaults to 800. Same role as Deepgram endpointing

VibeVoice (local)

provider = "vibevoice"

Local ASR and TTS via external Python services — no API keys, nothing leaves the machine. Start the servers first, then point the config at them.

python external/vibeVoice/vibeVoiceAsr/server.py   # default port 8200
python external/vibeVoice/vibeVoiceTTS/server.py   # default port 8300

[vibevoice]
asr_url = "ws://127.0.0.1:8200"
tts_url = "http://127.0.0.1:8300"
voice = "en-Emma_woman"

LLM Providers

Set [llm] provider to one of these values:

OpenAI

provider = "openai"

GPT-4o, GPT-4o-mini with streaming completions and function calling for plugin invocation.

[openai]
api_key = "your-openai-api-key"
model = "gpt-4o-mini"     # or "gpt-4o" for maximum capability
base_url = ""             # any OpenAI-protocol endpoint: DeepSeek, Moonshot, Qwen, MiniMax

Prefer a non-reasoning model here. Reasoning models emit their thinking before the answer, and on a voice call that is dead air. Note base_url is unrelated to the one in [ollama], which speaks Ollama’s own protocol.

Groq

provider = "groq"

Open models on Groq’s inference hardware — the fastest hosted option here, and the one that stops the LLM being the slowest link in a turn.

[groq]
api_key = "your-groq-api-key"
model = "openai/gpt-oss-20b"   # ~1000 tok/s. Also openai/gpt-oss-120b, qwen/qwen3.8-27b
reasoning_effort = "none"      # off by default; gpt-oss floors at "low"
reasoning_format = "hidden"    # "raw" would stream <think> tags into what TTS speaks

The models Groq serves fastest all reason, so both defaults turn it down. Hidden thinking still costs seconds, which is why reasoning_effort matters more than the format. Models with no reasoning at all (llama-*, groq/compound) are sent neither field.

Ollama (local)

provider = "ollama"

Run local models with zero API costs. Requires Ollama running separately.

[ollama]
base_url = "http://localhost:11434"
model = "gemma4:e4b"      # or "mistral", "qwen2.5", etc.

Bring your own agent

provider = "agent"

Each user turn is POSTed as JSON to a URL you own. Your endpoint keeps the conversation — memory, prompting, tools — in whatever language and framework you already use, and StreamCore only speaks the reply.

[agent]
url = "http://localhost:9000/agent"
api_key = ""          # sent as Authorization: Bearer. Empty disables auth
timeout_ms = 60000    # whole-turn budget, including streaming the reply

The request body is {session_id, type, text, system}. Reply with SSE (data: lines, raw or {"delta": "..."}), chunked text/plain, or a buffered {"text": "..."}. Streaming replies are spoken sentence by sentence as they arrive.

TTS Providers

Set [tts] provider to one of these values:

Cartesia Sonic

provider = "cartesia"

Ultra-low latency streaming TTS with natural prosody. Recommended for fastest time-to-first-byte.

[cartesia]
api_key = "your-cartesia-api-key"

Deepgram Aura

provider = "deepgram"

High-quality neural TTS. Uses the same Deepgram API key as STT.

[deepgram]
api_key = "your-deepgram-api-key"   # same key for STT + TTS
tts_model = "aura-2-thalia-en"      # e.g. aura-2-theia-en (Australian), aura-2-pandora-en (British)

ElevenLabs

provider = "elevenlabs"

Premium voice cloning and multilingual speech synthesis.

[elevenlabs]
api_key = "your-elevenlabs-api-key"
voice_id = ""      # defaults to Rachel
model = ""         # defaults to eleven_turbo_v2_5

Speechify

provider = "speechify"

[speechify]
api_key = "your-speechify-api-key"
voice_id = ""      # defaults to Geffen (geffen_32). Simba 3.2 uses its curated voice set
model = ""         # defaults to simba-3.2

MiniMax

provider = "minimax"

40+ languages. Two platforms with non-interchangeable keys — check which one you registered on.

[minimax]
api_key = "your-minimax-api-key"
voice_id = ""      # defaults to English_Graceful_Lady
model = ""         # defaults to speech-2.6-turbo (low latency). A Token Plan key (sk-cp-) covers
                   # only speech-2.8-hd; anything else routes to pay-as-you-go and fails with 2056
# base_url = ""    # defaults to https://api.minimax.io/v1. Mainland-China accounts use
                   # https://api.minimaxi.com/v1

MiMo (Xiaomi)

provider = "mimo"

[mimo]
api_key = "your-mimo-api-key"
voice = ""         # defaults to mimo_default. Chinese: 茉莉 冰糖 苏打 白桦; English: Mia Chloe Milo Dean
model = ""         # defaults to mimo-v2.5-tts; also -voiceclone / -voicedesign (paid)

VibeVoice (local)

provider = "vibevoice"

Same local service as VibeVoice STT. Pair it with Ollama for a setup that needs no API keys at all.

[vibevoice]
tts_url = "http://127.0.0.1:8300"
voice = "en-Emma_woman"

Cartesia also takes voice_id, ws_url, and max_concurrency (default 3) — set the last one to your plan’s TTS concurrency limit so requests queue locally instead of hitting 429s.

[rag] — answer from your own documents

Optional. Point the server at a vector store and retrieved context is injected into each turn — or, in speech-to-speech mode, exposed as a knowledge_search tool the model calls on demand. Omit the section entirely to disable retrieval.

Supabase

[rag]
provider = "supabase"                       # or "pgvector"
top_k = 3
embedding_model = "text-embedding-3-small"  # optional, this is the default

[supabase]
url = "https://xxx.supabase.co"
api_key = "your-service-role-key"
function = "match_documents"   # optional, this is the default
table = "documents"            # optional; used by streamcore-cli when ingesting

pgvector

[rag]
provider = "pgvector"
top_k = 3

[pgvector]
connection_string = "postgres://user:pass@localhost:5432/mydb"

Getting documents in there is what streamcore-cli is for — it parses PDFs, Word, Excel, CSV, and markdown, chunks them, embeds each chunk, and writes to the same store. Set pipeline.rag_prefetch to start retrieval during the turn merge window rather than after it.

[plugins]

Key	Default	Description
directory	`"./plugins"`	Path to the plugins and skills directory. Scanned at startup.

The directory structure inside the plugin directory:

plugins/
├── plugins/           # External plugins (Python/TS/JS)
│   ├── math-calculate/
│   ├── weather-get/
│   └── time-get/
└── skills/            # Skill definitions (Markdown)
    ├── friendly-assistant/
    └── concise-responder/

Environment Variables

Every secret can come from the environment instead of the file — no exceptions, so a container deployment never has to bake a key into an image. Environment values take precedence over config.toml. Non-secret settings (models, voices, tunables) stay in the file only.

Variable	Overrides
STREAMCORE_TURN_SECRET	[server] turn_secret
STREAMCORE_JWT_SECRET	[server] jwt_secret
STREAMCORE_API_KEY	[server] api_key
STREAMCORE_AGENT_API_KEY	[agent] api_key
DEEPGRAM_API_KEY	[deepgram] api_key
ASSEMBLYAI_API_KEY	[assemblyai] api_key
ALIYUN_API_KEY	[aliyun] api_key
VOLCENGINE_API_KEY	[volcengine] api_key
OPENAI_API_KEY	[openai] api_key
XAI_API_KEY	[grok] api_key
CARTESIA_API_KEY	[cartesia] api_key
ELEVENLABS_API_KEY	[elevenlabs] api_key
SPEECHIFY_API_KEY	[speechify] api_key
MINIMAX_API_KEY	[minimax] api_key
MIMO_API_KEY	[mimo] api_key
SUPABASE_API_KEY	[supabase] api_key
PGVECTOR_CONNECTION_STRING	[pgvector] connection_string

JWT Authentication (Optional)

When jwt_secret is set, the WHIP endpoint requires a valid JWT bearer token. Clients can obtain tokens from the /token endpoint (if api_key is also set).

config.toml

[server]
jwt_secret = "your-secret-key"    # Enable JWT auth
api_key = "your-api-key"          # Protect the /token endpoint

Token request

curl -X POST http://localhost:8080/token \
  -H "Authorization: Bearer your-api-key"

WHIP with JWT

curl -X POST http://localhost:8080/whip \
  -H "Authorization: Bearer <jwt-token>" \
  -H "Content-Type: application/sdp" \
  -d @offer.sdp

Quickstart →Plugin guide →SIP & Telephony →