TEN Agent Events

Events are the primary communication mechanism between the RTC server, MLLM extension, and your custom agent extension. TEN Agent uses a Server→Client (S2C) model for incoming events and Client→Server (C2S) primitives for outgoing commands.

S2C Events (Server to Client)

These are events sent from the server that your agent extension receives and handles.

User & Session Events

`UserJoinedEvent`

Triggered when a user joins the RTC session.

Property	Type	Description
user_id	`string`	Unique identifier for the user joining
timestamp	`float`	Event timestamp (Unix epoch)

Example:

case UserJoinedEvent():
    self._rtc_user_count += 1
    await self._greeting_if_ready()

`UserLeftEvent`

Triggered when a user leaves the RTC session.

Property	Type	Description
user_id	`string`	Unique identifier for the user leaving
timestamp	`float`	Event timestamp

Example:

case UserLeftEvent():
    self._rtc_user_count -= 1

Transcript Events

`InputTranscriptEvent`

Speech recognized by the ASR extension (user speech converted to text).

Property	Type	Description
text	`string`	The recognized speech text
final	`bool`	Whether this is a final (complete) transcript
metadata	`dict`	Additional context (session_id, confidence, etc.)
stream_id	`string`	Unique identifier for this speech segment

Example:

case InputTranscriptEvent():
    self.current_metadata = {"session_id": event.metadata.get("session_id", "100")}
    self.session_ready = True
    await self._greeting_if_ready()

Final vs Streaming: The final flag indicates whether the transcript is complete or still being recognized. Non-final transcripts should not trigger processing.

`OutputTranscriptEvent`

Assistant speech generated by the MLLM (text-to-speech will convert this to audio).

Property	Type	Description
text	`string`	The assistant's speech text
is_final	`bool`	Whether this completes the response
stream_id	`string`	Unique identifier for this speech segment

Example:

case OutputTranscriptEvent():
    await self._send_transcript("assistant", event.text, event.is_final, event.stream_id)

Tool & Function Events

`ToolRegisterEvent`

A tool/function becomes available for the agent to call.

Property	Type	Description
tool	`str`	Tool/function identifier
source	`str`	Extension that registered this tool
metadata	`dict`	Tool schema and parameters

Example:

case ToolRegisterEvent():
    await self.agent.register_tool(event.tool, event.source)

`FunctionCallEvent`

The MLLM is requesting a tool/function be called (function calling in action).

Property	Type	Description
call_id	`string`	Unique identifier for this function call request
function_name	`string`	Name of the function to call
arguments	`dict`	Function parameters as key-value pairs

Example:

case FunctionCallEvent():
    await self.agent.call_tool(event.call_id, event.function_name, event.arguments)

After processing a FunctionCallEvent, you must send the result back via the C2S SendFunctionCallOutput primitive to continue the conversation.

Interrupt Event

`ServerInterruptEvent`

The MLLM detected the start of user speech and sent an interrupt signal.

Property	Type	Description
reason	`string`	Why the interrupt occurred (e.g., "user_speech_detected")

Example:

case ServerInterruptEvent():
    await self._interrupt()

C2S Primitives (Client to Server)

These are commands your agent extension sends to the MLLM server.

Set Message Context

Initialize or update the conversation context with message history.

async def _set_context_messages(self, messages: list[MLLMClientMessageItem]):
    await _send_data(
        self.ten_env,
        DATA_MLLM_IN_SET_MESSAGE_CONTEXT,
        "v2v",
        MLLMClientSetMessageContext(messages=messages).model_dump(),
    )

Parameters:

messages: List of message items (system, user, assistant)

Send Message Item

Add a single message to the conversation.

async def _send_message_item(self, message: MLLMClientMessageItem):
    await _send_data(
        self.ten_env,
        DATA_MLLM_IN_SEND_MESSAGE_ITEM,
        "v2v",
        MLLMClientSendMessageItem(item=message).model_dump(),
    )

Parameters:

message: A single message (text, image, or audio)

Trigger Response

Ask the MLLM to generate a response (thinking + speaking).

async def _send_create_response(self):
    await _send_data(
        self.ten_env,
        DATA_MLLM_IN_CREATE_RESPONSE,
        "v2v",
        MLLMClientCreateResponse().model_dump(),
    )

Call this after setting context or sending a message to trigger the MLLM to think and speak.

Send Function Call Output

Return the result of a tool/function call back to the MLLM.

async def _send_function_call_output(self, result: str, call_id: str):
    await _send_data(
        self.ten_env,
        DATA_MLLM_IN_FUNCTION_CALL_OUTPUT,
        "v2v",
        MLLMClientFunctionCallOutput(
            output=result,
            call_id=call_id,
        ).model_dump(),
    )

Parameters:

output: The tool result as a string
call_id: The call_id from the original FunctionCallEvent

Event Flow Diagram

User Speech
    ↓
RTC → ASR Extension
    ↓
InputTranscriptEvent (S2C)
    ↓
MainExtension.on_data()
    ↓
_set_context_messages() (C2S)
    ↓
_send_create_response() (C2S)
    ↓
MLLM processes & generates response
    ↓
OutputTranscriptEvent (S2C)
    ↓
TTS Extension converts to audio
    ↓
Audio streamed to user

User Message (text/voice)
    ↓
MainExtension processes input
    ↓
Sends to MLLM (C2S)
    ↓
MLLM responds with text
    ↓
OutputTranscriptEvent (S2C)
    ↓
TTS converts & streams audio
    ↓
User hears response

Common Patterns

Greeting When Ready

async def _greeting_if_ready(self):
    if self._rtc_user_count == 1 and self.config.greeting and self.session_ready:
        await self._send_message_item(
            MLLMClientMessageItem(
                role="user",
                content=self.config.greeting,
            )
        )
        await self._send_create_response()

Handling Tool Results

case FunctionCallEvent():
    try:
        result = await call_tool(event.function_name, event.arguments)
        await self._send_function_call_output(json.dumps(result), event.call_id)
    except Exception as e:
        await self._send_function_call_output(json.dumps({"error": str(e)}), event.call_id)

Graceful Shutdown

case UserLeftEvent():
    self._rtc_user_count -= 1
    if self._rtc_user_count == 0:
        await self._cleanup()

Reference

For complete event definitions and types, refer to the source code in the TEN Framework:

Events

Table of Contents