​

Architecture

​

Core building blocks

​

AgentConfig

• unique name
• agent_cls subclass
• optional stt, llm, and tts values (ProviderValue | None: provider ID strings or plugin instances)
• optional greeting generated after ctx.connect()
• optional session_kwargs forwarded to AgentSession
• optional source_path when the module file is known (e.g. after discovery), for tooling and footprint estimates—not used for routing

​

AgentDiscoveryConfig

• optional explicit name
• optional stt, llm, and tts overrides
• optional greeting override

​

AgentPool

• isolation="coroutine" (the v0.1 default) constructs an internal _CoroutineAgentServer subclass that swaps livekit.agents.ipc.proc_pool.ProcPool for our CoroutinePool for the duration of run().
• isolation="process" constructs the vanilla AgentServer from livekit-agents (one OS subprocess per session, the v0.0.x behavior).

​

Session lifecycle

1. OpenRTC resolves the target agent from job metadata, room metadata, room-name prefix matching, or the first registered agent.
2. It creates an AgentSession using the selected agent configuration.
3. Prewarmed VAD and turn detection models are injected from proc.userdata.
4. The resolved agent instance is started for the room.
5. OpenRTC connects the room context.
6. If a greeting is configured, it generates the greeting after connect.

​

Coroutine-mode lifecycle

                         AgentServer.run()
                                │
              first time, builds CoroutinePool (one per worker)
                                │
                  CoroutinePool.start()
                                │
              ┌─── runs the user's setup_fnc ONCE ───┐
              │   into a singleton JobProcess        │
              │   (loads VAD, turn detector, …)      │
              └──────────────────────────────────────┘
                                │
                       worker is registered
                       and accepts dispatch
                                │
                                ▼
                  per session (N concurrent):
                                │
                    CoroutinePool.launch_job(info)
                                │
              builds a CoroutineJobExecutor wired with
              the same setup_fnc + entrypoint_fnc the pool was
              constructed with, plus a context_factory closing
              over the singleton JobProcess
                                │
                  executor.launch_job(info)
                                │
                schedules `_run_entrypoint(ctx)` as
                an asyncio.Task on the running loop
                                │
                                ▼
                  user entrypoint runs (AgentSession etc.)
                                │
              wrapper catches any exception, sets status
              to FAILED, calls session_end_fnc, removes the
              executor from pool.processes; supervisor counts
              consecutive failures
                                │
                                ▼
                  on shutdown: pool.drain() awaits every
                  in-flight executor's join(); pool.aclose()
                  cancels anything still pending

• Setup runs once per worker. The user’s prewarm callback (Silero, turn detector, etc.) is invoked exactly once into the singleton JobProcess, then every executor’s JobContext references that same process and userdata dict. This is the density story: prewarm cost is amortized across N concurrent sessions instead of paid once per session as in process mode.
• One executor, one session. Every launch_job allocates a fresh CoroutineJobExecutor; concurrent sessions never share an executor. Errors stay isolated to their executor’s task wrapper.
• No subprocess. Per-session work runs as asyncio.Tasks on the worker loop. There is no IPC, no process boundary, no per-session process startup cost.
• Cooperative backpressure. CoroutinePool.current_load() returns len(active) / max_concurrent_sessions. The _CoroutineAgentServer registers a load_fnc closure that reads this value, so LiveKit dispatch sees >= 1.0 at saturation and routes new jobs elsewhere.
• Cooperative shutdown. drain() flips a flag (rejecting new launches) and awaits every executor’s join(); aclose() then cancels anything still pending and clears state. After both, the worker’s asyncio loop has no residual tasks belonging to the pool. Upstream AgentServer installs the SIGTERM/SIGINT handler and invokes aclose() when one fires; the wait window is bounded by AgentPool(drain_timeout=N) (default 30 seconds). Sessions that exceed the budget are cancelled with a WARNING log and the per-executor kill() escalation runs so the worker can finish shutting down.
• Supervisor. After consecutive_failure_limit (default 5) consecutive non-SUCCESS terminations, the pool fires its registered callback. The default callback in _CoroutineAgentServer schedules aclose() so the worker exits and the deployment platform restarts it — the blast radius of a systemic bug stays bounded.

​

Configuration precedence

1. CLI flag — --isolation, --max-concurrent-sessions. Passed on the openrtc start / dev / console command line, this wins over everything else.
2. Environment variable — OPENRTC_ISOLATION, OPENRTC_MAX_CONCURRENT_SESSIONS. Read at startup when the matching flag is not provided.
3. Library default — isolation="coroutine", max_concurrent_sessions=50. Baked into AgentPool.__init__ when neither the flag nor the env var is set.

​

Shared runtime dependencies

• livekit.plugins.silero
• livekit.plugins.turn_detector.multilingual.MultilingualModel

​

Why this shape?

• routing logic is explicit
• worker-scoped dependencies are loaded once
• discovery metadata is opt-in and typed
• agent registration stays stable and readable
• the public API remains small enough for contributors to extend safely