Skip to content
Trazum

Product direction: useful work per AI budget

Documentation / Proposed product experiments

Direction: position Trazum around the cost and capacity needed to complete useful work, then use measurements to improve the next task. The task efficiency workflow is implemented in the 2.6.0 release. The broader control loop below remains a product direction, not shipped automation.

Two budgets, one outcome#

An API team can optimize actual billed cost per accepted task. A subscription user can optimize observed token use, elapsed time and retries per accepted task. A fixed fee remains a fixed fee. Provider capacity rules may be opaque; Trazum must not invent a remaining-message counter from tokens.

Track the denominator: a task has an explicit completion criterion and outcome (accepted, rejected or unknown). Include failed attempts in the cost of the workflow. Report success rate and unresolved outcomes alongside unit cost. A cheaper model that causes rework can increase cost per accepted task.

A local control loop#

  1. Set policy: choose task type, quality criterion and an explicit budget.
  2. Prepare: recommend a context strategy and allowed configuration from observed comparable tasks; use user-supplied defaults when evidence is absent.
  3. Observe: collect available usage and tool-event metadata without prompt content by default. Mark missing data and covered/uncovered host calls.
  4. Intervene: suggest a bounded action at a meaningful threshold. Applying it requires a host capability and user policy; recommendations are not enforcement.
  5. Evaluate: compare full-task consumption, quality, retries and elapsed time, retaining inconclusive results.

Do not add a supervisor model call to every turn. Run deterministic summaries locally, cache them and emit a short recommendation only when something changed. Account for the controller's own instructions and calls in the measurement.

Experiments worth testing#

ExperimentMechanism to prototypeSuccess measure and failure boundary
Context policySmall opt-in host instructions; prefer targeted reads and bounded tool outputLower whole-task input with unchanged acceptance. Re-reading changed files remains necessary.
Tool-output controlHost adapter filters or paginates oversized results, preserving errors and providing an expansion handleFewer input tokens without extra failed searches or missing evidence. No silent truncation.
Repetition detectorDetect repeated tool calls or repeated identical file ranges within a taskFewer unnecessary retries; distinguish changed files and deliberate validation. Advisory first.
Output policyTask-specific response length or output contractLower output plus retries, with the required explanation and deliverable intact.
Task configuration historyCompare allowed models and context policies on matched tasksBetter cost/quality trade-off; no model switch without an available host control and agreed policy.
Session handoffPropose a compact goal/decision/constraint/evidence summary when changing tasksSuccessful continuation with less carried context; user can inspect and retain original evidence.
Capacity pacingShow observed consumption trend against a user-set work budgetBetter planning without claiming to predict an undocumented provider reset or remaining allowance.

Do not ship all of these at once. Start with one host and one repeated task type so overhead and quality can be measured rather than inferred.

First product experiment#

Opt-in context and output policy, evaluated over complete tasks.

Start with the task efficiency workflow: import explicit Claude Code/Codex sources, inspect repeated calls with matching recorded results, review the fixed repeat-review-v1 policy, optionally append it to a selected instruction file, and compare a later task with an explicit outcome. The CLI supports local apply/revert; MCP provides analysis and comparison. These additions are available since 2.6.0. The existing compact instruction template remains a manual option.

For the first adapter, require documented access to the necessary events. Read metadata where available; do not assume transcripts reveal exact file-range reuse, hidden system context, quota accounting or task success. If the host cannot expose an event or enforce an action, label it unavailable.

The first record keeps task identity/type, criterion, declared acceptance and assessor, policy version, measured usage, source diagnostics, harness/model coverage and repetition evidence. It excludes raw tool text and source paths; its reuse checksums do not authenticate or anonymize a report. Comparison refuses to treat missing coverage or replayed evidence as an improvement.

Additional record capabilities to design and validate:

  • Start/end timestamps and measurement coverage.
  • Actual billed cost kept separate from the existing API-equivalent estimate.
  • Attempt count and controller overhead where attributable.
  • Independent evidence that a host loaded a policy and that acceptance passed.

Do not infer success from a zero exit code alone or put raw file paths, prompts and answers in a shared record by default. Keep unknown counters unknown.

Compare the baseline with the policy on representative comparable tasks. Keep the model and relevant settings fixed for the first experiment so the policy is the variable being tested. Repeat runs and include setup overhead. Define the acceptable quality change and minimum worthwhile benefit before results are observed. Ship a default only after the net benefit survives that check.

Reuse the existing product#

Existing surfaceRole in this directionAdditional work required
Native imports and usage profilingBaseline counts, cost estimates and conversation-growth evidenceAutomatic task attribution and coverage beyond explicitly supplied telemetry.
Prompt optimization and token checksReview controlled instruction text and check chosen budgetsWhole-task quality and net-overhead evaluation.
Saved plans and verificationKeep a proposed change and inspect later evidenceRepeated experiments beyond the paired task comparison introduced in 2.6.0.
MCP skill and toolsLet an agent request analysis and receive adviceVerified host behavior beyond the optional instruction policy introduced in 2.6.0.
Local API and gatewayExpose policy decisions; enforce supported rules on routed callsNo blanket control over subscription-host turns; adapters need explicit support.

Adoption test#

Keep the existing API-cost cohort and the subscription-efficiency cohort separate. Ask both: "Did Trazum change a decision, and did the resulting task finish acceptably with less total resource use?" Publish the supported measurement, not a promised percentage of savings or additional messages.

If a policy cuts tokens but reduces acceptance or creates more human repair, it is not a FinOps win. If it improves quality at the same budget, that is a useful result even when the token count does not fall.