WebSocket Voice Assistant Quickstart

Build and run a real-time WebSocket voice assistant in under 10 minutes.

Overview

This guide helps you run a complete voice assistant demo locally. It supports real-time voice conversations and tool calling. The demo includes:

Component	Vendor	Purpose
ASR	Deepgram	Speech-to-text
LLM	OpenAI	Response generation
TTS	ElevenLabs	Text-to-speech
Tool	WeatherAPI	Weather lookup (optional)

Requirements

Prerequisites

Follow the TEN Framework Quick Start to install the base environment, then make sure the check below passes:

tman check env

Expected output (at least Python and Go are Ready):

✅ Operating System: Supported
✅ Python:   Ready
✅ Go:       Ready

Extra tools

Install the following tools for build & run:

Task (task runner)

Linux / macOS:

go install github.com/go-task/task/v3/cmd/task@latest
task --version  # verify

Windows:

# Option 1: Install via winget (recommended)
# Install winget from: https://apps.microsoft.com/detail/9nblggh4nns1?hl=en-US&gl=US
# Requires Windows 10 version 1709 (Build 16299) or higher, or Windows 11
winget --version  # verify winget
winget install Task.Task
task --version  # verify

# Option 2: Install via scoop
irm get.scoop.sh | iex
scoop --version  # verify scoop
scoop install task
task --version  # verify

Bun (JavaScript package manager)

Linux / macOS:

curl -fsSL https://bun.sh/install | bash
bun --version  # verify

Windows:

powershell -c "irm bun.sh/install.ps1 | iex"
bun --version  # verify

uv (Python package manager)

Linux / macOS:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv --version  # verify

Windows:

powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
uv --version  # verify

💡 After installation, reload your shell config:

Linux/macOS: source ~/.zshrc

Windows: reopen your PowerShell terminal

Quickstart

Clone the repo

git clone https://github.com/TEN-framework/ten-framework.git
cd ten-framework/ai_agents/agents/examples/websocket-example

Install dependencies

task install

⏱️ Estimated time: 2–3 minutes

This command will:

Install TEN extension packages (45+)
Build the Go app and API server
Install Python and frontend dependencies

Configure API keys

Configure vendor keys in ai_agents/.env:

Linux / macOS:

cd ../../../  # back to ai_agents
vim .env

Windows:

cd ..\..\..   # back to ai_agents
notepad .env

Copy all content from .env.example, then fill in:

# Deepgram - ASR
DEEPGRAM_API_KEY=your_deepgram_api_key

# OpenAI - LLM
OPENAI_API_KEY=your_openai_api_key
OPENAI_MODEL=your_openai_model

# ElevenLabs - TTS
ELEVENLABS_TTS_KEY=your_elevenlabs_api_key

# WeatherAPI - Tool (optional)
WEATHERAPI_API_KEY=your_weatherapi_key

Start services

cd agents/examples/websocket-example
task run

After startup, the following services are available:

Frontend UI: http://localhost:3000
API server: http://localhost:8080
TMAN Designer: http://localhost:49483

Talk to your assistant

Open http://localhost:3000, click the microphone button, and start speaking.

Try:

"Hello, who are you?"
"What's the weather in Beijing today?" (requires WeatherAPI)
"Tell me a joke."

Congratulations! Your TEN Agent voice assistant is running.

Advanced

Swap extensions

TEN Framework lets you swap ASR/LLM/TTS extensions flexibly.

Option 1: Use TMAN Designer (recommended)

Open http://localhost:49483, then in the visual editor:

Click an extension node to inspect it
Choose a replacement extension
Configure parameters
Save and restart

Option 2: Edit configuration file

Edit tenapp/property.json and modify the extension configs under predefined_graphs.

Example: switch ASR to Azure

{
  "type": "extension",
  "name": "stt",
  "addon": "azure_asr_python",
  "extension_group": "stt",
  "property": {
    "params": {
      "key": "${env:AZURE_STT_KEY}",
      "region": "${env:AZURE_STT_REGION}",
      "language": "en-US"
    }
  }
}

Example: switch TTS to Minimax

{
  "type": "extension",
  "name": "tts",
  "addon": "minimax_tts_websocket_python",
  "extension_group": "tts",
  "property": {
    "params": {
      "api_key": "${env:MINIMAX_TTS_API_KEY|}",
      "group_id": "${env:MINIMAX_TTS_GROUP_ID|}",
      "model": "speech-02-turbo",
      "audio_setting": {
        "sample_rate": 16000
      },
      "voice_setting": {
        "voice_id": "female-shaonv"
      }
    }
  }
}

Restart after changes (Ctrl+C then task run).

Monitoring (Grafana)

Use Grafana to observe application metrics and logs.

Start monitoring stack

Linux / macOS:

cd tools/grafana-monitoring
docker-compose -f docker-compose.push.yml up -d

Windows:

cd tools\grafana-monitoring
docker compose -f docker-compose.push.yml up -d
# If Docker is not installed, install it from https://apps.microsoft.com/detail/xp8cbj40xlbwkx, restart your PC, then launch Docker Desktop

Services:

Grafana (port 3001) - UI
Prometheus (port 9091) - metrics store
Loki (port 3100) - log store
OpenTelemetry Collector (ports 4317/4318) - OTLP receiver

Configure app telemetry

Edit agents/examples/websocket-example/tenapp/property.json:

Logs (export)

Add to the ten.log.handlers array:

{
  "matchers": [{ "level": "info" }],
  "formatter": { "type": "json", "colored": false },
  "emitter": {
    "type": "otlp",
    "config": {
      "endpoint": "http://localhost:4317",
      "service_name": "websocket-example"
    }
  }
}

Metrics (export)

Add to ten.services:

{
  "telemetry": {
    "enabled": true,
    "metrics": {
      "enabled": true,
      "exporter": {
        "type": "otlp",
        "config": {
          "endpoint": "http://localhost:4317",
          "service_name": "websocket-example"
        }
      }
    }
  }
}

Restart and open Grafana

cd ../../../agents/examples/websocket-example
task run

Open http://localhost:3001 (username/password: admin/admin).

Metrics (Dashboards)

Go to Dashboards → "TEN Framework" to view:

Extension startup latency
Cmd processing latency
Message queue wait time

Logs (Explore/Loki)

Go to Explore → select Loki, then query:

{service_name="websocket-example"} - all logs
{service_name="websocket-example", ten_extension_name="stt"} - ASR logs
{service_name="websocket-example", ten_extension_name="llm"} - LLM logs
{service_name="websocket-example", ten_extension_name="tts"} - TTS logs

💡 Use the ten_extension_name label to filter logs per extension.

Stop the monitoring stack:

Linux / macOS:

cd tools/grafana-monitoring
docker-compose -f docker-compose.push.yml down

Windows:

cd tools\grafana-monitoring
docker compose -f docker-compose.push.yml down

Troubleshooting

If bun install fails due to missing versions, switch to the official npm registry:

cd frontend
echo "registry=https://registry.npmjs.org/" > .npmrc
rm bun.lock
bun install

Set the Python dynamic library path:

export DYLD_LIBRARY_PATH=/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib:$DYLD_LIBRARY_PATH
# Add to ~/.zshrc for persistence

Make sure the Python 3.10 installation path is in your system PATH:

# Check Python path
python -c "import sys; print(sys.prefix)"

# Add Python DLL directory to PATH (adjust to your actual path)
$env:Path += ";C:\Python310;C:\Python310\DLLs"

If you didn't check "Add Python to PATH" during installation, re-run the installer and choose "Modify" to add it.

Find and kill the process:

Linux / macOS:

lsof -i :3000  # or :8080
kill -9 <PID>

Windows:

netstat -ano | findstr :3000   # or :8080
taskkill /PID <PID> /F

# Set proxy
# Linux/macOS
export GOPROXY=https://goproxy.cn,direct
# Windows PowerShell
$env:GOPROXY = "https://goproxy.cn,direct"

# Clean Go module cache
go clean -modcache

# Reinstall dependencies
cd ai_agents/agents/examples/websocket-example
task install

Next steps

Extension development guide - build custom extensions
More examples - explore other agent demos
GitHub Issues - report bugs or request features
Documentation hub - full TEN Framework documentation

WebSocket Voice Assistant Quickstart

Table of Contents