Settings¶
TandemApp keeps its settings where they always were: the page's switches and the backends in the controller's config database, the conductor's documents in their folder, each engine's settings on its own host. /v1/settings is one interface over all of them. Every part of it answers the same six questions, a change can span several parts, and every change is versioned, can be undone and is audited.
The contract is contracts/settings-v1.
Scopes¶
| scope | target | store | restart class |
|---|---|---|---|
app |
controller |
the page's switches: chat and conductor, on or off |
none |
backends |
controller |
the config database: backends.<id>.<field>, backends.<id>.key, routes, users.<id>.enabled |
none |
conductor |
controller |
this controller's conductor documents: settings, router, roles, packages, monitor, residency, deciders, profiles.<name> |
hot |
engine |
an engine host's id | the engine's managed overlay, through tandem remote settings over SSH |
per key: hot or engine |
hooks |
controller |
the hook registry and the decision providers, through their adapter: hooks.<id>.<field>, providers.<id>.<field> |
none; an in-process hook needs an engine restart |
plugins |
an installed plugin's id | the plugin registry: enabled, settings.<property>, secrets.<NAME>, capabilities.<capability>, trust (plugins) |
none |
They apply in this order. A change to the backends lands before a conductor document that may name one, and both land before an engine restart. Hooks apply before plugins, so a change set that turns off a plugin and the hooks that name it lets go of the hooks first.
The restart classes:
none: the change applies as it is.app: this controller restarts.hot: the running service reads the change at once.engine: the engine restarts. This is scheduled and waits for the engine to be idle.
An engine reached over HTTP only, a TandemLLM backend whose host is not paired over SSH, is a read-only target, http:<backend id>. Its hot keys are read from the engine's GET /v1/engine/settings. To change it, use its own host (tandem remote settings), or pair it.
The six operations¶
Every scope answers:
GET /v1/settings/{scope}/schema[?target=] each key: type, default, restart class, fingerprint flag, secret, writable
GET /v1/settings/{scope}[?target=] each key's value and where it comes from
POST /v1/settings/{scope}/plan {target?, set?, unset?}
POST /v1/settings/{scope}/apply {target?, set?, unset?, confirm, confirm_fingerprint?, confirm_grant?, when?, deadline_s?, note?}
GET /v1/settings/{scope}/versions[?target=]
POST /v1/settings/{scope}/restore {target?, version, confirm?, ...} without confirm: the plan
GET /v1/settings lists the scopes in order with their targets.
- Sources. In
app, a source isinstallwhile the env file decides (TANDEM_CHAT,TANDEM_CONDUCTOR) andpageonce switched. Unsetting a key hands it back to the install. Inbackends, a field the environment sets isenvand not writable here. Itsvariablenames the variable to change instead. Engine keys carry the engine's own layer names (default,install,profile,setup,local,managed,mode). - Versions. The conductor and the engine have versions of their own:
<document>@<n>and the overlay'sn. Forapp,backendsandhooks, the controller keeps the values after each change in its config database. Secrets are left out. A change made on another screen is recorded as a version of its own ("as found") before the next change. - Restore writes a new version equal to the old one. It never deletes history. A key that no longer exists, or that the environment now sets, is left alone and the plan says so. A conductor document or an engine overlay that is no longer valid is refused with
restore_invalid.
Change sets¶
POST /v1/settings/plan {changes: [{scope, target?, set?, unset?}, {scope, target?, restore: "<version>"}, ...]}
POST /v1/settings/apply {changes, confirm, confirm_fingerprint?, confirm_grant?, when?, deadline_s?, note?}
A change set has at most one part per scope and target.
Unsetting a key that is unset already is no change, in every scope: the plan drops it, and a part left with nothing applies as unchanged. In backends, a field that holds nothing is unset (set false).
The plan writes nothing. For each part it lists:
- every change, with its restart class;
- for a part that restarts an engine, how long the restart itself takes (
restart_estimate, below); - the keys that feed an engine's disk-cache fingerprint;
- warnings.
restart_estimate is {seconds, source}. The source is one of these, in this order:
| source | seconds |
|---|---|
measured |
what this controller saw the last time it restarted that engine: from the engine's job acting to the end of the warm-up. Kept while the controller runs. |
engine |
the engine's last start as its status reports it: its last start, restart or settings job that acted at once, from its creation to its end |
default |
180 |
It leaves out the wait for idle, which deadline_s bounds.
It gives one confirm for the whole set. A key that feeds the fingerprint adds confirm_fingerprint: after that restart every saved prefix and parked chat misses, so it asks twice.
A refusal while a part is planned names the part: its scope and target, and the key when it is about one key's value (unknown_key, locked, bad_value, invalid_document, extension_missing). The same key on two engine hosts is told apart. A part of one key names that key even when its store's sentence does not.
Applying plans again and compares:
| situation | answer |
|---|---|
no confirm |
428 confirm_required |
| the settings changed since the plan | 409 stale_plan, with the new plan |
| a fingerprint key without its second confirm | 428 fingerprint_confirm_required |
| another change set is applying | 409 busy |
The parts then apply in order. The first failure stops the set. The answer lists each part:
| state | meaning |
|---|---|
applied |
written |
scheduled |
an engine restart will run it (see below) |
unchanged |
it held already |
failed |
refused or failed; error says why |
not_run |
after the failure, untouched |
Each applied part carries the version it had before and the restore that brings it back. The set as a whole is applied, partial, failed or unchanged.
Applying again. The same confirm with the same body, sent again within five minutes, answers the same result (replayed true). Nothing is applied twice. After a partial failure, plan the same changes again: the parts that already hold are unchanged, and the rest is planned anew.
Secrets¶
A backend's API key, and any key a hook adapter marks secret, is write-only:
- an answer shows
settrue or false, never the value; - a plan shows the change as set or unset;
- the confirm binds the value through its hash only.
No answer, version, audit row, log line or export holds a secret. A restore leaves secrets as they are.
The engine's keys that may hold a secret are not managed by its overlay. Their values are not read into an answer either.
Engine changes and restarts¶
Hot changes. A change of hot keys only (today QSE_ALLOW_FROM) goes through the engine's PATCH /v1/engine/settings (engine API 1.9). The plan's overlay version is the base. There is no restart.
Other changes go through tandem remote settings apply. when sets when the engine acts, deadline_s how long it waits:
when: idle(the default) waits for idle, up todeadline_s(600 s by default);when: nowacts at once;when: at=<ISO time>waits until that time, then for idle.
The engine's own job stops the engine, makes the new overlay version current and starts it. If the start fails, the engine puts the last good version back.
The restart. The controller follows the engine's job (GET /v1/settings/restarts/{id}, in the part's restart):
| state | what happens |
|---|---|
scheduled |
waiting for an at= time. Nothing is drained yet. |
draining |
the gateway answers 503 node_draining with Retry-After: 30 for the backends on that engine host. Other backends keep serving. The engine's job waits for idle. remaining_s counts down to the deadline. |
restarting |
the job acts: stop, new version, start. |
warming |
the job is done. One token goes through each backend on the host, then the backends serve again. |
done, failed, timeout, cancelled |
the end. |
failed: the engine's message, andrestored, the version it runs again and whether it answers.timeout: the deadline passed while the engine was busy. It changed nothing.cancelled: see below.
Cancel. POST /v1/settings/restarts/{id}/cancel works while the restart is scheduled or draining. The controller sends start to the engine host. That replaces a job that still waits, and finds the engine healthy, so nothing changes. Once the job acts it is too late (409 too_late).
The drains live in the controller's process. A controller restart ends them; the engine's job goes on by itself.
The engine's mode¶
Two keys of the engine scope stand for the engine's mode:
| key | values | what it is |
|---|---|---|
mode |
engine or chat |
engine: the bare engine. chat: TandemApp's conductor and Chat run inside it. |
mode.ext |
a module, tandemapp.engine_ext by default |
the extension chat mode runs |
They show what the engine's QSE_MODE and QSE_EXT resolve to. Those two are not changed directly: a part that names them is refused (422 locked).
- The mode alone. A part that only sets
modeandmode.extgoes through the engine's owntandem remote mode set. It plans, applies and restarts like any other engine change: the change set's id, the audit rows,whenanddeadline_s, the drain, the countdown, Cancel and the warm-up token. The restart'skindismode. - With other keys. Together with other keys of the same engine, the mode goes into the same overlay change as
QSE_MODEandQSE_EXT, which is whatmode setwrites. The engine restarts once. - The extension. When the result is chat mode, the plan asks the engine host for its extensions (
tandem remote ext list). If none of that module loads there, the plan says so in a warning and names the wheel TandemApp installs first: its own, for its own version (TANDEM_NODE_WHEEL, else built from this controller's copy, as Add Spark sends it). The apply sends it (tandem remote ext install, its sha256 checked) before the switch. The install itself restarts nothing. - No wheel. A module that is not TandemApp's, or no wheel on this controller, makes the plan refuse with 409
extension_missingon the keymode.ext. The sentence says what to do: setTANDEM_NODE_WHEELto the tandemapp wheel of this release, or install the extension on the engine host.
POST /v1/fleet/engines/{id}/mode {mode?, when?, ext?}, the route the Add engine wizard uses, is a change set of one part with these keys. It answers as before: whether anything changed, the mode before and the engine's job. Its audit row names the change set.
Audit¶
GET /v1/settings/audit[?change_set=cs-...] lists the settings' rows of the config database's audit table, which /v1/config/audit shows too:
- one row per part applied or restored (
settings.apply,settings.restore) with its result; - one row per change set (
settings.change_set) with what applied, what failed and what did not run; - one row per restart's end (
settings.restart) and per cancel (settings.cancel).
Each row has the change set's id, the scope and target, the keys (never a value), who, when and the result. Every row a store writes during a change set carries the same id, for example a backend's update or a switch of the page.
Rights¶
| request | rule |
|---|---|
| reads | the dashboard's |
| plans, applies, restores, a cancel | the configuration's: the admin from another machine, or a tool on this machine |
Every write must come from the page itself: a JSON body, not from another site or origin.
Hooks¶
The hooks scope is an adapter (settingscopes.HooksAdapter). The controller plugs in the hook registry's (proxy/hooksettings.py). Without one, the scope answers an empty schema.
Each hook has these keys:
| key | what |
|---|---|
hooks.<id>.enabled |
whether it is called at all |
hooks.<id>.mode |
off, shadow or on |
hooks.<id>.priority |
its order among the hooks of a stage, low first |
hooks.<id>.scope |
the requests it runs on: lists of patterns under models, backends, profiles, users, sessions, client_kinds. Unset: every request. |
hooks.<id>.policy |
open or closed when it fails. Unset: the stage's default. |
hooks.<id>.timeout_ms |
its own time limit. Unset: the stage's budget. |
hooks.<id>.payload |
meta, redacted or full |
hooks.<id>.secret |
an http hook's signing secret, shown only as set or unset |
Each value carries the hook's name as its label. The checks are the registry's own. A change is written through the registry and audited there, and the hook runner reads it at the next request. The settings interface keeps a version after each change.
Adding or removing a hook, where and when it runs, and a new secret stay on the hooks' API (/v1/fleet/hooks, hooks). That API makes the secret and shows it once, so the settings interface never writes one.
The full payload¶
Setting payload to full grants the hook the prompts and answers as they are. It keeps the rules of the hooks' API:
- Only a caller with pairing's rights may plan it: the admin, or this machine while no admin password is set. Anyone else gets 403
trust_required. - The plan warns with the destination's host.
- A destination outside the LAN needs a second confirm. The plan lists the key under
grant_keysand givesconfirm_grant. Apply without it is 428grant_confirm_required. The grant records who confirmed it.
Decision providers¶
The decision providers are keys of the same scope. They sit beside the hooks because a hook provider is a decision.ask hook. Each provider has these keys:
| key | what |
|---|---|
providers.<id>.enabled |
whether it is asked at all |
providers.<id>.mode |
on, shadow or ab |
providers.<id>.ab_percent |
in ab, the share it answers, 0 to 100 |
providers.<id>.sticky |
in ab, what the share sticks to: session, user or request |
providers.<id>.questions |
the questions it answers, each {model?, endpoint?, timeout_ms?} |
providers.<id>.api_key |
a systemone-http provider's own key |
providers.<id>.questions.<question>.api_key |
the key for one question alone; a change set that adds the question to questions may set it in the same change |
A key is write-only. A string sets it and an unset clears it. An answer says only whether one is set, with its hint, and a version never holds it. A question's key exists once the provider answers that question: add the question first, then its key.
One enabled provider serves a question on or ab. A change set may move a question from one provider to another. The provider that lets it go is written first.
Adding or removing a provider and its kind stay on its API (/v1/fleet/decision-providers, decision providers).
Plugins¶
The plugins scope (settings-v1 1.1) has one target a plugin. Its keys come from the plugin's manifest, so they differ plugin by plugin; each plugin keeps its own versions, and restoring one never touches another or the hooks. Installing, upgrading and removing a plugin stay on its API (/v1/fleet/plugins, plugins).
| key | what |
|---|---|
enabled |
the plugin host may start it. The tree is checked against its recorded sha256 first. |
settings.<property> |
one property of the plugin's settings schema, with its type and default. A value must hold against the property's schema. Unset: the default. |
secrets.<NAME> |
a declared secret, write-only, sealed under the master key |
capabilities.<capability> |
a requested capability, false until granted. Unset revokes it. |
trust |
only for a plugin whose manifest asks trust: host: host lets it run as a process of the app's user, outside any sandbox; sandboxed (the default) keeps it from starting. Set to host here only for a plugin signed by a key trusted for host; an unsigned plugin's grant takes the typed confirmation on its trust route (plugins), and here answers 409. |
Enabling a plugin and every grant take pairing's rights (403 trust_required otherwise). A grant of prompts:full, engine:control, network:<host> outside the LAN or trust host is in the plan's grant_keys and needs confirm_grant. The plugin host restarts a running plugin when its settings, secrets or grants change. A restore that brings a grant back asks again.
The configuration as a file¶
GET /v1/fleet/app/export writes the configuration as a file (tandemapp-fleet-config, version 2): the fleet's nodes, deployments, alert rules, layouts and app.* settings, and the hooks, the decision providers, the plugins (name, version, source, sha256, settings and grants, never their files), each paired engine host's overlay and the conductor's stored documents. No secret is in it: a hook's secret, a provider's keys, a plugin's secrets and an engine's secret keys appear by name only.
POST /v1/fleet/app/import {config, mode?} plans the file and changes nothing; POST /v1/fleet/app/import/apply with the plan's confirm applies it. Version 2's switches, settings, grants, overlay keys and documents go through one change set of this interface, in its order, stopping at the first failure; the hooks and providers the file adds are created after it, the hooks first. A plugin the file names that is not installed is a conflict naming what to install, and so is a hook of that plugin; an engine host that is not paired is a conflict. A plan whose change set feeds an engine's disk-cache fingerprint, or that grants a payload outside the LAN, gives confirm_fingerprint or confirm_grant, and the apply sends them too. A secret the file holds by mistake is stripped and listed in the plan's stripped. A version 1 file still imports; a file of a later version is refused with the version it needs.
The pages¶
The dashboard shows this interface under Settings, App: one page per scope.
| page | what it holds |
|---|---|
| App | the app scope: the Chat and conductor switches under What runs here, above the controller's own page |
| Backends | the backends scope: a group for each backend, the routing rules, the users' keys |
| Conductor | the conductor scope: each document, edited whole |
| Engines | the engine scope: a page for each engine host, its mode first (the two choice cards, and the extension once chat is chosen). An HTTP-only one is read-only and says why. |
| Hooks | the hooks pages (hooks) and the decision providers' (decision providers). They stage their switches as keys of the hooks scope. Under the pipeline the same keys show as a plain form, a group for each hook and each provider. |
| Plugins | the plugins pages (plugins). A plugin's switch, settings, secrets, grants and trust level are staged as keys of the plugins scope, one target a plugin. Under a plugin's page the same keys show as a plain form, with the plugin's own versions. |
| History | the audit rows and the restarts that ended. #/app/history?change_set=cs-... shows one change set. |
Each page draws its form from the scope's schema. A key gets the control its type asks for: a switch for on and off, a number field for an integer, a JSON editor for a document. The row also shows the default and the value's source. The environment locks some keys; such a row names the variable, for example TANDEM_BACKENDS. No page shows a secret. You set one or remove it.
Changing a field writes nothing. The page stages the change, and a bar at the bottom carries the staged changes from page to page. Review then reads the plan of the whole set from POST /v1/settings/plan:
- every part in the order it applies, with its changes and warnings;
- whether a part restarts an engine, and how long the restart takes ("about 3 min"). The plan then asks when: once idle (it waits 10 minutes by default), at a set time, or now;
- a second confirm for a key that feeds the fingerprint, and for a hook's full payload outside the LAN. That one names the hooks and where their payload goes.
A plan the controller refuses for one field marks that field, on the page of the part the refusal names, with the sentence under it.
Apply answers with each part's state. An applied part has a Restore button. After a failure you can plan the same changes again.
While an engine restarts, every page of the area shows it: the state, the countdown to the deadline, the engine's job, and Cancel while can_cancel holds. After a failed start the card names the version the engine runs again.
Versions sit at the bottom of each page, with what differs from now. A restore goes through the same plan as any other change.