In-Conversation Widgets

Agents sometimes need something from you mid-task. It might be an API key, a confirmation, a choice between options. Without widgets, the only way to get that input is for the agent to ask in chat and for you to paste a value back. That breaks two things at once: your flow (the agent now needs to wait, you need to context-switch) and the security guarantee (the value lands in the model’s context window, in logs, in transcripts that survive).

Widgets fix that. When the agent calls a widget tool, a small form renders inline in the transcript. You fill it out. The values flow directly browser → daemon. They never reach the agent. A sanitized “[Agor] User submitted X” prompt is auto-queued into the agent’s next turn so it can pick up where it left off.

Widgets are a primitive. v1 ships one widget type (env_vars) as the canonical example. Future widget types (confirmations, OAuth connect, MCP-server picker, file picker) use the same plumbing.

This is the widget the agent calls when it needs an environment variable it doesn’t have. Typical use: an MCP integration needs HUBSPOT_API_KEY and the agent doesn’t want to fail with “please go to Settings”.

Agents should prefer the widget over Settings instructions. Agor’s system prompt and the credential-related tool descriptions (e.g. agor_repos_get) tell agents that when a task needs a secret the user hasn’t set, the correct move is to call agor_widgets_request_env_vars — not to write “open Settings → Environment Variables” or ask the user to paste a value into chat. A written Settings instruction is only a last-resort fallback for when the widget tools are unavailable.

What the agent does

The agent calls the MCP tool agor_widgets_request_env_vars:


{
  "names": ["HUBSPOT_API_KEY"],
  "reason": "Needed to call the Hubspot Private Apps API.",
  "variable_metadata": {
    "HUBSPOT_API_KEY": {
      "description": "Create one under Private Apps with the CRM > contacts > read scope.",
      "format_hint": "Starts with pat-"
    }
  }
}

The scope (Session / Global) is the user’s choice in the form — the agent does not set it.

The tool returns within milliseconds. The agent ends its turn voluntarily. No HTTP timeout, no executor pause, no daemon-side await.

What you see

A small form renders inline in the transcript:

The variable name(s) the agent is asking for
A short reason
Optional per-variable description and format_hint text (from variable_metadata)
A password input per variable (or text / textarea, per input_type)
A scope selector (Session / Global)
Dismiss and Save buttons
The security posture is conveyed by the lock/shield chrome — never paste values into chat

When you hit Save, the value goes directly from the form to the daemon via POST /widgets/:widget_id/submit. It does not traverse the model. The daemon writes it through the same encrypted-at-rest path as Settings → Environment Variables (AES-256-GCM via the daemon’s master secret).

What the agent sees next

A new user-role message arrives in the agent’s next turn:


[Agor] User submitted HUBSPOT_API_KEY (scope: global). You can now retry the operation that needed it.

The agent retries. Only the name appears in that prompt. The value is read from the encrypted store at executor spawn time, as if you had set it via the Settings panel.

The `already_present` short-circuit

If you already have the requested variables set in the chosen scope, the widget skips the form entirely. The transcript shows a ”✓ already configured” badge and the agent auto-resumes with:


[Agor] HUBSPOT_API_KEY was already configured. You can proceed.

One fewer click in the common “agent didn’t know I had this” case.

Dismissal

If you hit Dismiss, the widget marks itself dismissed and queues an explicit prompt into the agent so it doesn’t loop:


[Agor] User dismissed the request for HUBSPOT_API_KEY. Do not re-request immediately — ask whether to proceed without, or move on to other work.

Security guarantees

The widget primitive has one hard requirement: submitted values never enter the LLM’s context. Triple-checked:

Path	Carries values?
Agent → MCP tool call	No, input schema only accepts names + reason + variable_metadata
Widget message row	No, the row stores params (names, reason, …) and lifecycle state
MCP tool return value	No, returns `{ widget_id, status }` only
Browser → daemon submit	Yes, direct HTTP, authenticated, never traverses the agent
Daemon → users service	Yes, encrypted at the boundary, never logged
Auto-resume prompt to agent	No, built from sanitized `result_meta` (just names + scope)
`widget:resolved` WebSocket event	No, same sanitized payload
Transcript reload	No, the row never held values

The disclaimer text on the form (“Type values here, never paste them into chat”) is there for one specific reason: if the agent goes off-script and asks you to type a value into chat instead of into the widget, don’t. The widget is the only path that preserves the no-LLM-context guarantee.

For the full security analysis, see docs/internal/in-conversation-widgets-design-2026-05-19.md.

Transcript persistence

The widget message persists in the transcript with its final state:

submitted → ✅ + names + scope + timestamp
dismissed → ⊘ + names
already_present → ✓ + names + “already configured”

The names of submitted variables are visible to anyone who can read the transcript. The values are not, and never were. This is the same surface as Settings → Environment Variables, which also shows names without values.

If a particular request is sensitive enough that even the name shouldn’t appear in the transcript, dismiss the widget and add the variable via Settings.

Multi-user behavior

When branch_rbac: true and someone other than the session creator submits the widget (under others_can: prompt), the env var lands in the session creator’s identity. That’s whose env the agent will read at retry. The submitting user is recorded in result_meta.submitted_by for audit.

This is consistent with dangerously_allow_session_sharing: false (the default-safe posture). For the full multi-user story see Multiplayer Execution Isolation.

Why widgets, not chat?

Three reasons:

Secrets stay out of the model. The values flow directly to the daemon’s encrypted store. The agent never sees them.
No flow-breaking context switch. You don’t have to leave the conversation to go set up Settings → Env Vars.
Async-friendly. The widget is durable. Submit it now, submit it in an hour, submit it tomorrow. The session picks up wherever it was, on whatever device. Slack and gateway sessions work the same way.

This is the v1 instance of a broader primitive. Future widgets (confirmations, OAuth-connect, MCP-server picker) use the same machinery and the same security guarantees.