CLI

Give the agent your own documents.

streamcore-cli parses, chunks, embeds, and uploads documents into the vector store the server reads at query time. Point it at a folder of PDFs and the agent can answer from them on the next call.

Install

go install github.com/streamcoreai/streamcore-cli@latest

Or build from the repo:

cd streamcore-cli
go build -o streamcore-cli .

Setup

The wizard asks for your vector store provider, your OpenAI key and embedding model, and the provider credentials, then writes ~/.streamcore/config.toml. Run it again any time — it pre-fills what you already have.

streamcore-cli setup

Config is looked up in this order: --config, then ~/.streamcore/config.toml, then ./config.toml, then ../server/config.toml. The last one means a monorepo checkout can reuse the server’s config instead of configuring credentials twice.

Ingest documents

# One or more files
streamcore-cli ingest docs/faq.pdf product-catalog.xlsx notes.md

# Override the provider or point at a specific config
streamcore-cli ingest --provider supabase --config ./my-config.toml data.csv

# Control chunking
streamcore-cli ingest --chunk-size 256 --chunk-overlap 32 manual.docx

Supported formats

Format	Extensions
Plain text	`.txt`
Markdown	`.md, .markdown`
CSV	`.csv`
PDF	`.pdf`
Word	`.docx`
Excel	`.xlsx`

Flags

Flag	Default	Description
--config	`~/.streamcore/config.toml`	Path to the config file
--provider	`from config`	Override the RAG provider (pgvector, supabase)
--chunk-size	`512`	Target chunk size in words
--chunk-overlap	`64`	Overlap between chunks in words

How it works

1. Parse — Word documents are read from their underlying XML, Excel rows become field-value pairs, CSVs use the header row as field names.
2. Chunk — Split into overlapping chunks on paragraph and sentence boundaries, so a chunk never ends mid-sentence.
3. Embed — Each chunk goes to the OpenAI embeddings API.
4. Store — Chunk, embedding, and source filename are inserted into your vector store. The source column keeps the original filename for filtering and attribution.

Database setup

Both stores need the vector extension and a table the server can query.

pgvector

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT NOT NULL,
    embedding vector(1536),
    source TEXT
);

Supabase additionally needs the RPC the server calls at query time, plus row-level security policies that let the CLI insert and the server read:

Supabase

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT NOT NULL,
    embedding vector(1536),
    source TEXT,
    created_at TIMESTAMP DEFAULT NOW()
);

-- RPC used by the server for query-time retrieval
CREATE OR REPLACE FUNCTION match_documents(
    query_embedding vector(1536),
    match_count int DEFAULT 3
)
RETURNS TABLE (content text, similarity float)
LANGUAGE plpgsql AS $$
BEGIN
    RETURN QUERY
    SELECT d.content, 1 - (d.embedding <=> query_embedding) AS similarity
    FROM documents d
    ORDER BY d.embedding <=> query_embedding
    LIMIT match_count;
END;
$$;

ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

CREATE POLICY "Allow read access to documents"
ON documents FOR SELECT TO authenticated, anon USING (true);

CREATE POLICY "Allow insert access to documents"
ON documents FOR INSERT TO authenticated, anon WITH CHECK (true);

Wire it to the server

Once documents are ingested, point the server at the same store. In the classic pipeline the retrieved context is injected into the prompt; in speech-to-speech mode it is exposed as a knowledge_search tool the model calls when it needs it.

server config.toml

[rag]
provider = "supabase"   # or "pgvector", or omit entirely to disable
top_k = 3
embedding_model = "text-embedding-3-small"

[supabase]
url = "https://xxx.supabase.co"
api_key = "your-service-role-key"
function = "match_documents"
table = "documents"

Configuration →Plugins →Quickstart →