Documentation

Plugin Development Guide

Plugins are external tools that the LLM can call during conversation — when a user asks a question that requires an action (checking the weather, creating a calendar event, querying a database), the LLM invokes your plugin and speaks the result back.

How Plugins Work

1. You create a plugin folder with a plugin.yaml manifest and an entrypoint file
2. The server discovers and starts your plugin as a subprocess at boot
3. During conversation, the LLM decides when to call your plugin based on the description and parameters you define
4. The server sends a JSON-RPC request to your plugin over stdin
5. Your plugin processes the request and returns a result on stdout
6. The LLM incorporates the result into its spoken response

Plugins are long-lived processes— they start once and stay running to avoid cold-start latency.

Directory Structure

Plugins live inside the configured plugin directory (default: server/plugins/):

server/plugins/
└── plugins/
    └── my-plugin/
        ├── plugin.yaml       # Required — manifest
        ├── main.py           # Entrypoint (Python)
        └── requirements.txt  # Optional — Python deps

Plugin Manifest

Every plugin needs a plugin.yaml file:

plugin.yaml

name: weather.get                    # Unique dotted name (service.action)
description: Get current weather      # What the LLM sees — be descriptive
version: 1                           # Integer version number
language: python                     # python, typescript, or javascript
entrypoint: main.py                  # File to execute
parameters:                          # JSON Schema — defines what the LLM sends
  type: object
  properties:
    location:
      type: string
      description: City or location name
  required:
    - location
confirmation_required: false         # Reserved for future use

Manifest Fields

Field	Required	Description
name	Yes	Unique identifier in `service.action` format
description	Yes	Human-readable description shown to the LLM
version	Yes	Integer version number
language	Yes	`python`, `typescript`, or `javascript`
entrypoint	Yes	File to run (relative to plugin folder)
parameters	Yes	JSON Schema defining input parameters
confirmation_required	No	Reserved for future confirmation prompts

Naming Convention

Use dotted names following service.action pattern:

weather.get — Get weather data
calendar.create_event — Create a calendar event
database.query — Run a database query
email.send — Send an email

Building a Python Plugin

1. Install the SDK

cd plugin-sdk/python
pip install -e .

2. Create the Plugin

mkdir -p server/plugins/plugins/my-plugin
cd server/plugins/plugins/my-plugin

3. Write the Manifest

plugin.yaml

name: time.get
description: Get the current time in a specific timezone
version: 1
language: python
entrypoint: main.py
parameters:
  type: object
  properties:
    timezone:
      type: string
      description: IANA timezone name like America/New_York or Europe/London
  required:
    - timezone

4. Write the Code

main.py

from datetime import datetime
from zoneinfo import ZoneInfo
from streamcoreai_plugin import StreamCoreAIPlugin

plugin = StreamCoreAIPlugin()

@plugin.on_execute
def handle(params):
    tz_name = params.get("timezone", "UTC")
    try:
        tz = ZoneInfo(tz_name)
        now = datetime.now(tz)
        return f"The current time in {tz_name} is {now.strftime('%I:%M %p')}."
    except Exception as e:
        return f"Could not get time for timezone {tz_name}: {e}"

plugin.run()

5. Test It

echo '{"jsonrpc":"2.0","method":"execute","params":{"timezone":"America/New_York"},"id":1}' | python3 main.py

Expected output:

{"jsonrpc": "2.0", "result": "The current time in America/New_York is 02:30 PM.", "id": 1}

Python SDK Reference

SDK API

from streamcoreai_plugin import StreamCoreAIPlugin

plugin = StreamCoreAIPlugin()

@plugin.on_execute
def handle(params):
    """Called when the LLM invokes this plugin.

    Args:
        params: dict with the parameters defined in plugin.yaml

    Returns:
        A string result that the LLM will use in its response
    """
    return "result string"

@plugin.on_initialize
def init():
    """Optional — called once when the plugin process starts.
    Use for setup like loading models or establishing connections."""
    plugin.log("Plugin ready")

plugin.log("message")  # Log to the server (writes to stderr)
plugin.run()            # Start the JSON-RPC loop — blocks forever

Python Plugin with External APIs

main.py — weather example

import requests
from streamcoreai_plugin import StreamCoreAIPlugin
import os

plugin = StreamCoreAIPlugin()

@plugin.on_initialize
def init():
    global api_key
    api_key = os.environ.get("WEATHER_API_KEY", "")
    if not api_key:
        plugin.log("WARNING: WEATHER_API_KEY not set")

@plugin.on_execute
def handle(params):
    location = params.get("location", "")
    if not location:
        return "Please specify a location."

    resp = requests.get(
        "https://api.weatherapi.com/v1/current.json",
        params={"key": api_key, "q": location},
        timeout=10
    )

    if resp.status_code != 200:
        return f"Could not get weather for {location}."

    data = resp.json()
    current = data["current"]
    return (
        f"In {location} it's currently {current['temp_c']}°C "
        f"and {current['condition']['text'].lower()}."
    )

plugin.run()

Building a TypeScript Plugin

1. Install the SDK

cd plugin-sdk/typescript
npm install
npm run build

2. Write the Manifest

plugin.yaml

name: math.calculate
description: Evaluate a mathematical expression
version: 1
language: typescript
entrypoint: index.ts
parameters:
  type: object
  properties:
    expression:
      type: string
      description: A mathematical expression like "2 + 2" or "sqrt(144)"
  required:
    - expression

3. Write the Code

index.ts

import { StreamCoreAIPlugin } from '@streamcore/plugin';

const plugin = new StreamCoreAIPlugin();

plugin.onExecute(async (params) => {
  const { expression } = params;
  plugin.log(`Evaluating: ${expression}`);

  try {
    const sanitized = expression.replace(/[^0-9+\-*/().%\s]/g, '');
    const result = new Function(`return (${sanitized})`)();
    return `${expression} = ${result}`;
  } catch (e) {
    return `Could not evaluate "${expression}".`;
  }
});

plugin.run();

4. Test It

echo '{"jsonrpc":"2.0","method":"execute","params":{"expression":"2 + 2"},"id":1}' | npx tsx index.ts

TypeScript SDK Reference

SDK API

import { StreamCoreAIPlugin } from '@streamcore/plugin';

const plugin = new StreamCoreAIPlugin();

// Required — handle execute calls from the LLM
plugin.onExecute(async (params) => {
  // params is an object matching the JSON Schema in plugin.yaml
  return "result string";  // Returned to the LLM
});

// Optional — called once on process start
plugin.onInitialize(() => {
  plugin.log("Plugin ready");
});

plugin.log("message");  // Log to server (writes to stderr)
plugin.run();            // Start JSON-RPC loop — blocks forever

JavaScript Plugin

JavaScript plugins work the same way, but use language: javascript and node as the runtime.

plugin.yaml

name: greet.user
description: Generate a personalized greeting
version: 1
language: javascript
entrypoint: index.js
parameters:
  type: object
  properties:
    name:
      type: string
      description: The user's name
  required:
    - name

index.js

const { StreamCoreAIPlugin } = require('@streamcore/plugin');

const plugin = new StreamCoreAIPlugin();

plugin.onExecute(async (params) => {
  return `Hello ${params.name}! Great to meet you.`;
});

plugin.run();

Building a Go Native Plugin

Go plugins are compiled directly into the server binary. They have zero IPC overhead but require rebuilding the server.

1. Implement the Tool Interface

server/internal/plugin/weather.go

package plugin

import "encoding/json"

type WeatherTool struct{}

func (t *WeatherTool) Name() string {
    return "weather.get"
}

func (t *WeatherTool) Description() string {
    return "Get current weather for a location"
}

func (t *WeatherTool) Parameters() json.RawMessage {
    return json.RawMessage(`{
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City or location name"
            }
        },
        "required": ["location"]
    }`)
}

func (t *WeatherTool) ConfirmationRequired() bool {
    return false
}

func (t *WeatherTool) Execute(params json.RawMessage) (string, error) {
    var p struct {
        Location string `json:"location"`
    }
    if err := json.Unmarshal(params, &p); err != nil {
        return "", err
    }
    return "Currently 22°C and sunny in " + p.Location, nil
}

2. Register with the Plugin Manager

// After pluginMgr is created:
pluginMgr.RegisterNative(&plugin.WeatherTool{})

3. Rebuild the Server

cd server && go build -o streamcoreai-server .

Tool Interface

Go Tool interface

type Tool interface {
    Name() string                                      // Unique dotted name
    Description() string                               // LLM-facing description
    Parameters() json.RawMessage                       // JSON Schema for parameters
    Execute(params json.RawMessage) (string, error)    // Execute the tool
    ConfirmationRequired() bool                        // Reserved for future use
}

Communication Protocol

Plugins communicate with the server via JSON-RPC 2.0 over stdin/stdout. You don’t need to implement this yourself — the SDKs handle it — but understanding it helps with debugging.

Initialize

When the plugin starts, the server sends:

{"jsonrpc": "2.0", "method": "initialize", "id": 1}

The plugin responds with:

{"jsonrpc": "2.0", "result": "ok", "id": 1}

Execute

When the LLM calls the tool:

{"jsonrpc": "2.0", "method": "execute", "params": {"timezone": "UTC"}, "id": 2}

Success response:

{"jsonrpc": "2.0", "result": "The current time in UTC is 2:30 PM.", "id": 2}

Error response:

{"jsonrpc": "2.0", "error": {"code": -1, "message": "Invalid timezone"}, "id": 2}

Rules

stdout is reserved for JSON-RPC responses only (one JSON object per line)
stderr is for logging — use plugin.log() which writes to stderr
Never use print() (Python) or console.log() (JS/TS) — these write to stdout and break the protocol
Plugin processes have a 30-second timeout per execute call

Debugging

Common Issues

Plugin not loading

Check that plugin.yaml is valid YAML
Verify the name field is unique across all plugins
Check the server logs for loading errors

Plugin not being called

Make sure the description clearly explains what the plugin does
Check that parameters has a valid JSON Schema
Try asking the agent directly: "use the [plugin name] tool"

Plugin crashes

Test manually with piped JSON-RPC input
Check stderr output (the server captures and logs it)
Verify all dependencies are installed

Manual Testing

Test any plugin by piping JSON-RPC messages:

# Test initialize
echo '{"jsonrpc":"2.0","method":"initialize","id":1}' | python3 main.py

# Test execute
echo '{"jsonrpc":"2.0","method":"execute","params":{"key":"value"},"id":1}' | python3 main.py

Server Logs

The server logs plugin lifecycle events:

loaded plugin: time.get (python)
loaded plugin: math.calculate (typescript)
loaded 2 plugins, 1 skills
plugin time.get: Plugin ready          # stderr from the plugin

Best Practices

Keep it fast

Plugins run in the voice pipeline. A slow plugin means the user waits in silence. Aim for under 2 seconds.

Return spoken-friendly text

The result is spoken aloud. Return natural language, not JSON or code.

Log generously

Use plugin.log() for debugging. It goes to stderr and won't interfere with the protocol.

Handle errors gracefully

Return a meaningful error string rather than letting exceptions crash the process.

Don't print to stdout

Only JSON-RPC responses go to stdout. Use plugin.log() for anything else.

Use environment variables for secrets

Don't hardcode API keys. The plugin inherits the server's environment.

Keep descriptions specific

The LLM uses your description to decide when to call the plugin. Vague descriptions lead to missed or wrong invocations.

Test before deploying

Use the manual piped-input method to verify your plugin works before restarting the server.

Built-in Plugins

Name	Language	Purpose
math.calculate	TypeScript	Evaluate math expressions
weather.get	TypeScript	Current weather via API
time.get	Python	Time in any timezone
vision.analyze	TypeScript	Camera image analysis (GPT-4V)
gmail	TypeScript	Gmail OAuth2 read/send

Skills guide →Client SDKs →Quickstart →