Gating on cost, in whatever CI you already run
Documentation / CLI setup · Gateway · Schedules
Trazum is one binary with meaningful exit codes. That is the whole integration story, and it is deliberate: a tool that only gates on one vendor's CI is a tool half its readers cannot use, and every vendor-shaped convenience is a second code path that rots.
Exit codes are the contract.
| Code | Means |
|---|---|
0 | Every gate that ran, passed — or was waived, on the record. |
1 | A gate failed, or the run could not be completed. |
Commands that gate: trazum check (token budgets and the baseline),
trazum profile (money budgets, from flags or spend in the config), and
trazum verify --gate (a plan's promises).
Everything else exits 0 and reports. trazum doctor surveys and never gates —
its model recommendation is a keyword heuristic, and gating a build on a keyword
heuristic teaches people to re-run until green, which costs more than the tool
saves.
GitHub Actions#
The repository ships an Action, which additionally comments on the pull request:
- uses: Davmunrey/Trazum@<commit-sha> # pin to a commit, never a tag
with:
target: prompts/ # a file, or a directory using the config's budgets
max-tokens: '900'The input is target. file is a deprecated alias kept so existing
workflows keep working. There is no path input, and giving one fails the run
with "Set the 'target' input to the prompt file or directory to check, or
'usage-log' to gate the spend" — the Action refuses rather than checking
something nobody asked about.
Or use the binary directly, exactly as below.
What the change does to each prompt's monthly cost#
On a pull request the Action also reads the base branch's prompts and puts a
table in the same report, and so in the same comment: for each prompt the
change touches, its tokens on the base and on the head, what it costs on
each, and the difference. It is trazum check <target> --base <sha>, with
the pull request's base commit, and it adds no second comment: the report is
still one comment per comment-key, replaced on every push.
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@<commit-sha>
with:
fetch-depth: 0 # the base commit has to be in the clone
persist-credentials: false
- uses: Davmunrey/Trazum@<commit-sha>
with:
target: prompts/
comment: 'true'
github-token: ${{ secrets.GITHUB_TOKEN }}
max-monthly-increase-usd: '25' # optional; off unless setWhat each figure is, and what it is not:
The base comes from the clone, never from the network. It is read with
git show, and a base commit the clone does not have is said, with the fix (fetch-depth: 0), rather than fetched or stood in for by the working tree.actions/checkoutfetches one commit by default, so without that line the Action does not compare and the report is what it was before this existed. Withmax-monthly-increase-usdset, which cannot be judged without the base, it compares anyway and fails with that fix.A month needs a volume somebody stated. It is
volumesintrazum.config.json, calls a month for a label thatlabelsmaps to the prompt file, the same maptrazum plan --editsreads:{ "labels": { "support": "prompts/support.txt" }, "volumes": { "support": 30000 } }A changed prompt with no stated volume is priced per call, and the report says how many are. Several labels sending one file add up; one of them without a volume leaves the file per call. A prompt marked inside a source file is always per call, because a label maps a file and not a prompt in it.
Input tokens at the input price, both sides at the same prices. The model is
usage.model, or Trazum's default, and the report says which. Output is left out, because a prompt edit does not set how much the model writes back. Every row names its price per million input tokens and the day that price was read from the provider's page: the catalogue's review date for that provider, or a local overlay's own date. Both sides use the same price list, so a provider repricing cannot read as a change the pull request made.max-monthly-increase-usdis the one gate, and it is off unless set. It fails the build when the prompts with a stated volume cost more than that many dollars a month more on the head than on the base, and it names how many changed prompts it could not judge. Set off a pull request, or withusage-log, it is refused; when the base cannot be read it fails, because a gate that judged nothing is not a pass.
Locally, against any ref the clone has:
trazum check prompts/ --base origin/main --max-monthly-increase-usd 25A weekly summary in the repository#
The same Action can write the week down and post it, so nobody has to remember
to run bill. It reads a usage log the job can see (committed, or fetched by
an earlier step), writes trazum weekly's
summary, and posts it to the one place you name. It is off unless
weekly-summary is set.
on:
schedule:
- cron: '41 7 * * 1'
permissions:
contents: read
issues: write
jobs:
weekly:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: Davmunrey/Trazum@<commit-sha> # pin to a commit, never a tag
with:
usage-log: logs/usage/
weekly-summary: comment
weekly-summary-issue: '12'
github-token: ${{ secrets.GITHUB_TOKEN }}weekly-summary is the owner's choice and there are two answers. issue opens
one issue for the week, titled with its dates. comment adds one comment for
the week on the standing issue weekly-summary-issue names, which is the only
place it comments. Any other value stops the run. The repository is always the
one the workflow runs in: no input and nothing in the event moves it.
- A week is posted once. The summary ends in a marker naming the week's
last day, and a re-run of the same week replaces its own post instead of
adding another; the next week gets its own. Only a post made by a bot counts
as its own, so an issue or comment somebody else wrote is never edited, even
one that quotes the marker. That recognition needs the workflow's own
GITHUB_TOKEN: with a personal access token the author is a user, and a re-run of the same week would post again. - It sets nothing else. No label, assignee or milestone: those are yours.
- The endpoints are the four it needs: list the repository's issues or the
standing issue's comments, create an issue or a comment, and edit the one it
created. The token is the workflow's, handed to one script and never to the
CLI. It needs
issues: writeand nothing more. - It never fails the build for not posting. A token too narrow, an issue that is not there or an API that refuses each leave a notice, and the summary is on the run's summary page either way. A summary that could not be written does fail the run, since nothing was said.
- It is a mode of its own. With
weekly-summarythe run takesusage-log(and optionallyuntil,label,config,locale) and refusestargetand every gate (max-usd,against,max-growth-usd,max-cache-loss-usd,since): one run says one thing. Run a gate in its own step.
A plan that opens its own pull request#
Some of what a plan says to do is an edit Trazum can already write: a prompt
trimmed by the safe rules. With plan-pull-request: 'true' the Action makes
the plan with trazum plan --edits, makes
those edits on a new branch, and opens one pull request whose body is the
plan: each change, the measured figure it is expected against, and how
trazum verify checks it. It is off unless asked for, and it is the one step
of the Action that writes to the repository, so give it a job of its own and
the two permissions there:
on:
schedule:
- cron: '23 6 * * 1'
permissions:
contents: read
jobs:
plan:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v7
- uses: Davmunrey/Trazum@<commit-sha> # pin to a commit, never a tag
with:
usage-log: logs/usage/
plan-pull-request: 'true'
github-token: ${{ secrets.GITHUB_TOKEN }}The files come from labels in trazum.config.json, which says which prompt
file each workload sends. Only a file git tracks is edited, never a source
file, and only by the safe rules: no model writes any of it.
- The branch carries the plan's edits and nothing else. Each new text is made again from the default branch's file, which has to be the file the plan was made against (by SHA-256), and has to come out as the text the plan promised (by SHA-256). One file that moved on refuses the whole pull request, because a body listing a change the branch does not make would not be the plan. A file's mode is kept.
- No pull request with no change. A plan with no edit, or one whose edits are all on the default branch already, creates nothing: no branch, no commit.
- One pull request per plan. The branch is
trazum/plan-and a hash of the plan's edits, so the same edits made into a plan next week are the same plan. An open pull request from it is reused; a closed or merged one is the owner's answer and is not opened again; a branch of that name with no pull request is left alone. - Never the default branch, never a merge. The one ref it creates is the new branch, once. It calls no merge endpoint, and sets no label, reviewer or assignee. The repository is the one the workflow runs in and the base is its default branch: no input and nothing in the event moves either.
- No saving is projected. The figure each edit is held to is what the
workload cost in the log, measured. The file's tokens before and after are
the estimator's and are labelled estimated.
trazum verify plan.json --against <the newer log>says afterwards whether the edit is in the file and whether the calls got shorter. - It never fails the build for not opening. A token without the two permissions, a file that moved on or an API that refuses each leave a notice, and the plan is on the run's summary page either way. A plan that could not be made does fail the run.
- It is a mode of its own. It takes
usage-log(and optionallyconfigandlocale) and refusestarget,weekly-summaryand every gate and window: it proposes, over the whole log, and gates nothing.
GitLab CI#
prompt-cost:
image: node:20
script:
- npx --yes @trazum/cli check prompts/ --max-tokens 900
- npx --yes @trazum/cli profile logs/ --max-usd 400
artifacts:
when: always
paths: [trazum-summary.md]--markdown-out trazum-summary.md on either command writes the same report as
a Markdown file, which GitLab renders as a job artifact.
Jenkins#
stage('Prompt cost') {
steps {
sh 'npx --yes @trazum/cli check prompts/ --max-tokens 900'
sh 'npx --yes @trazum/cli profile logs/ --max-usd 400 --markdown-out trazum.md'
}
post {
always { archiveArtifacts artifacts: 'trazum.md', allowEmptyArchive: true }
}
}A failing gate throws on the sh step, which is what marks the stage failed.
Nothing Jenkins-specific is needed and nothing Jenkins-specific is offered.
CircleCI#
jobs:
prompt-cost:
docker: [{ image: cimg/node:20.11 }]
steps:
- checkout
- run: npx --yes @trazum/cli check prompts/ --max-tokens 900
- run: npx --yes @trazum/cli profile logs/ --max-usd 400The sign-off hook, for contributors to this repository#
.github/workflows/dco.yml requires every non-merge commit in a pull request
to carry a Signed-off-by trailer, and it is a required check. Forgetting
git commit -s once is expensive out of all proportion to the mistake: adding
the trailer to a later commit does not help, and neither does reverting, because
the workflow walks base..head and the unsigned commit is still in that range.
The only fix is rewriting history and force-pushing.
So install the hook and stop thinking about it:
ln -s ../../scripts/prepare-commit-msg .git/hooks/prepare-commit-msgIt adds the trailer from the identity git is already about to record as the
author, which is the same one git commit -s would use. It never adds a second
one, it skips merges because the workflow does, and TRAZUM_SIGNOFF_HOOK=0
turns it off. Nothing installs it for you: a repository that adds git hooks
during npm install is a repository that runs code somebody did not ask for.
signoff-hook.test.js reads the pattern out of the workflow and asserts the
hook's own output matches it, so the two cannot drift on what a trailer looks
like — a hook writing a trailer the check rejects would be worse than no hook,
because it would look like the problem was solved.
A pre-commit hook#
The one place where speed matters more than completeness, so gate on the files this commit touches and nothing else — one pipe, no shell loop:
#!/bin/sh
# .git/hooks/pre-commit
git diff --cached --name-only | npx --yes @trazum/cli check --files-from - || {
echo "Prompt over budget. Commit with --no-verify if you mean it."
exit 1
}--files-from - reads the list from stdin — exactly the shape
git diff --name-only produces. Paths that are not prompt files, paths the
config ignores, and deletions are dropped and counted out loud; a commit
that touches no prompts passes without ceremony. Budgets apply per file from
the config as always. The baseline gate is deliberately skipped here — it
compares the whole repository against the committed record, and a partial
list would read as removals — so keep trazum check . in CI for it.
--no-verify is named on purpose. A hook somebody cannot get past is a hook
somebody uninstalls, and an uninstalled hook gates nothing.
Verifying a plan in CI#
The plan is the one output meant to be committed (format). Once it is in the repository, a broken promise is as visible as a broken test:
trazum plan logs/ -o plan.json # once, reviewed in a pull request
trazum verify plan.json --against logs/ --gate--gate fails on a promise that did not arrive, and on fields-stopped — a
team that degraded its own log must not pass on the strength of the silence. A
workload that genuinely vanished, or a tier the log never recorded, reports
cannot-tell and fails nothing. Three outcomes, never two.
Gating performance without gating the runner#
A wall-clock budget in CI fails on weather: shared runners lie about time. The bench's gate holds a ratio instead — each workload timed against a calibration loop in the same process, so the runner's speed cancels out of the number being judged:
trazum bench --record trazum.bench.json # once, committed
trazum bench --against trazum.bench.json --max-ratio 3 # every buildExits 1 when any workload is past its recorded ratio times the stated
factor, on stderr, the way check gates. The factor is yours: 3× is a
tripwire for the quadratic-regex class of regression; a tighter factor buys
sensitivity at the price of noise. A baseline whose version the binary does
not know is a loud error naming --record, never a best-effort read — the
file is committed, so it crosses upgrades. This repository runs exactly this
step on its own CI.
Waivers, and their record#
A gate failure a team has looked at and decided to live with goes in the config with a reason and an end date:
{
"spend": { "maxUsd": 400 },
"waive": [
{ "gate": "maxUsd", "reason": "the vendor migration lands in March", "until": "2026-04-01" }
]
}All three fields are required. A waiver with no end date is a finding deleted with extra steps, and a reasonless one is a silence nobody can audit. The failure still prints — waived is shown as waived, never hidden — and only the exit code goes quiet.
Every use is written down. When a waiver silences a gate, Trazum appends a
dated line to .trazum/waivers.jsonl: the gate, the reason and expiry as they
stood at that moment, the commit when CI exported one, and the figures the gate
judged. trazum history reads those lines back and can finally say that a
finding has been waived nine times across four months under one unchanging
sentence whose deadline moved three times — which is a decision nobody is
revisiting, and something no config could ever have told you.
Two rules make that record trustworthy:
- Nothing is back-filled. The history begins the day recording began, and says so. A past reconstructed from today's config would be a guess presented as evidence.
- A waiver nobody's build has hit is dead config, not a habit, and the report keeps the two apart.
Commit .trazum/waivers.jsonl if you want the record shared, or leave it out of
version control if you only want it locally. Trazum never rewrites it and offers
no command to clear it: a record of decisions the tool can erase is a record
nobody can rely on. Delete the file yourself if you mean to.
What is deliberately not here#
No vendor plugin, no marketplace listing, no wrapper action for each CI. They would each be a second code path with its own bugs, its own release cadence and its own way of drifting from the exit codes above. One binary, documented recipes, and any limitation stated rather than papered over.