Skip to main content

​
Architecture

OpenRTC keeps the public API intentionally narrow.

​
Core building blocks

​
AgentConfig

AgentConfig stores the registration-time settings for a LiveKit agent:

​
AgentDiscoveryConfig

AgentDiscoveryConfig stores optional discovery metadata attached by @agent_config(...):

​
AgentPool

AgentPool owns a single LiveKit AgentServer, a registry of named agents, and one universal session handler. At startup it configures shared prewarm so worker-level runtime assets are loaded once and reused across sessions. The pool picks the underlying server class from the isolation constructor argument:
  • isolation="coroutine" (the v0.1 default): swaps livekit.agents.ipc.proc_pool.ProcPool for CoroutinePool, running sessions as asyncio.Tasks in the main worker loop.
  • isolation="process": uses the vanilla AgentServer from livekit-agents, one OS subprocess per session (the v0.0.x behavior).
The same agent classes, providers, and routing rules apply in both modes.

​
Session lifecycle

1

Route

OpenRTC resolves the target agent from job metadata, room metadata, room-name prefix matching, or the first registered agent.
2

Build session

It creates an AgentSession using the selected agent configuration and injects prewarmed VAD and turn detection models from proc.userdata.
3

Start

The resolved agent instance is started for the room.
4

Connect

OpenRTC connects the room context.
5

Greet

If a greeting is configured, it generates the greeting after connect.

​
Coroutine-mode lifecycle

When isolation="coroutine" (the v0.1 default), per-job work runs inside the worker process as asyncio.Tasks instead of in a forked subprocess.
The user’s prewarm callback (Silero, turn detector, etc.) is invoked exactly once into the singleton JobProcess. Every executor’s JobContext then references that same process and userdata dict. This is the density story: prewarm cost is amortized across N concurrent sessions instead of paid once per session as in process mode.
Every launch_job allocates a fresh CoroutineJobExecutor. Concurrent sessions never share an executor, so errors stay isolated to their own task wrapper.
Per-session work runs as asyncio.Tasks on the worker loop. There is no IPC, no process boundary, and no per-session process startup cost.
CoroutinePool.current_load() returns len(active) / max_concurrent_sessions. The _CoroutineAgentServer registers a load_fnc closure that reads this value, so LiveKit dispatch sees >= 1.0 at saturation and routes new jobs elsewhere.
drain() flips a flag (rejecting new launches) and awaits every executor’s join(). aclose() then cancels anything still pending and clears state. After both, the worker’s asyncio loop has no residual tasks belonging to the pool.The wait window is bounded by AgentPool(drain_timeout=N) (default 30 seconds). Sessions that exceed the budget are cancelled with a WARNING log and the per-executor kill() escalation runs so the worker can finish shutting down.
After consecutive_failure_limit (default 5) consecutive non-SUCCESS terminations, the pool fires its registered callback. The default callback in _CoroutineAgentServer schedules aclose() so the worker exits and the deployment platform restarts it, bounding the blast radius of a systemic bug.
In process mode, the per-session lifecycle is unchanged from v0.0.x: each session is its own subprocess via livekit-agents’s default ProcPool, with its own JobProcess, its own setup_fnc invocation, and its own rtc.Room.

​
Configuration precedence

Worker-runtime settings (isolation, max_concurrent_sessions) can be supplied at three layers: The same precedence applies to LiveKit connection settings (--url / LIVEKIT_URL, --api-key / LIVEKIT_API_KEY, --api-secret / LIVEKIT_API_SECRET, --log-level / LIVEKIT_LOG_LEVEL), which follow the upstream livekit-agents naming convention.

​
Shared runtime dependencies

During prewarm, OpenRTC loads:
Both plugins are bundled with openrtc as package dependencies. If they are missing at runtime, OpenRTC raises a RuntimeError with install instructions.
  • livekit.plugins.silero: voice activity detection (VAD)
  • livekit.plugins.turn_detector.multilingual.MultilingualModel: end-of-turn detection