When a run fails

Two different things get called a failed run, and they have different outcomes. A run that hits one of the agent's ceilings stops early but exits cleanly: it is marked completed, its findings are stored, and it is sealed. A run where the container or the scheduler breaks is marked failed, and a failed engagement is never sealed.

What evidence a failed run leaves¶

Sealing is called from one place, on the branch the scheduler takes when the container exits 0 and returns a results payload. The other three branches mark the engagement failed and return without sealing. There is no re-seal: no endpoint, command or retry queue calls the seal path again for a run that did not take that branch, so a failed engagement stays unsealed permanently.

What a failed engagement has:

• Artifacts. Each artifact is hashed and stored as it streams out of the container, before the run's outcome is known, so the ones captured before the failure are in the artifact store and the database. No endpoint serves them. The locker export refuses an unsealed engagement with a 409, and there is no other route that reads artifacts.
• The progress trace, for 24 hours — but not through the console. The engagement page opens the stream only while the status is queued or running, and it flips to failed the moment the failure event arrives, so the terminal disappears as the run ends and does not come back when you reopen the page. The trace is still served by GET /api/engagements/{id}/progress, which replays it from the first event. Calling that endpoint directly is the only way to read it.
• The container's stdout, written to a file inside the scheduler's own container as the run streams, not the agent's. The agent's container is removed only on the normal path, after the scheduler has read its exit code. The log file is an operator-side artefact, not something the console or the API exposes.

What it does not have:

• Findings. Findings are written to the database in the same step that marks an engagement completed. A failed run's findings existed only inside the container and in its log.
• A report. Report generation is refused for any status other than completed.
• A seal or a locker. See Sealing and Exporting a locker.

The three ways a run is marked failed¶

What happened	The event you see
The container exited non-zero	Agent exited with code N
The container exited 0 but sent no results payload	Agent produced no result payload
The scheduler raised while processing the job	Scheduler error: …

The third covers the cases where the container never ran properly — the Docker client being unavailable, for instance. The second is treated as a failure deliberately, so a run with nothing to show is not recorded as a success.

The ceilings, and why they do not produce a failure¶

The agent's run loop has two limits, and reaching either ends the loop the same way a finished run ends it: the agent reports engagement_complete, prints its results payload and exits 0. The engagement is then marked completed, its findings are written, and sealing runs exactly as it does for a run that worked through every phase. The evidence is intact; the run did less than it might have.

500 steps. A step is one turn of the run loop; in the six model-driven phases that is one model call, which may issue a command, record a finding, complete the phase or just reason. At 500 the loop stops wherever it is, possibly mid-phase.

Five consecutive failing steps. A step that raises is caught, published as an error event carrying the count (Step failed (3/5): …), and the loop continues. The counter resets to zero after any step that does not raise, so only an unbroken run of five aborts the engagement. A sixth error event, Aborting engagement after repeated step failures, follows the fifth and precedes the stop.

Two smaller limits do not end a run at all. A single command is killed after five minutes, and the output it had produced is discarded rather than handed over: the agent is told only STDERR: Command timed out, [COMMAND TIMED OUT] and an exit code of -1. The command log is still captured as an artifact, with an empty stdout. The run carries on, and a tool that reliably needs longer than five minutes yields nothing on every attempt. The threat-modeling phase has its own step cap, ten by default, which force-advances that phase rather than stopping the run.

The maximum duration you set in the wizard is not a ceiling. It is written into the rules-of-engagement document and nothing enforces it — see Rules of engagement.

Two failures that look like something else¶

Overlapping scope lists. The check runs when the rules-of-engagement document is rendered, inside the container in the first phase rather than at submit time. It is not the same test for both kinds of entry: two networks overlap when either contains the other, so an out-of-scope /24 inside an in-scope /16 is caught, while domains have to match exactly. Rendering raises, the phase does not advance, and every subsequent step raises the same way until the five-consecutive-error abort fires. The engagement then ends completed, with no findings and no artifacts, and is sealed over an empty evidence set. The trace shows five error lines naming the overlap, then the abort line. The cost is a queue slot and a container. See Scope.

An artifact over 10 MiB. Capture refuses it rather than truncating it, and the refusal propagates as a step failure: the command had already run, but its log is not recorded and the agent never sees its output. The run continues, one consecutive error further along. A tool whose output is reliably that large will burn five steps and abort the run.

What to do next¶

Read the trace first, from GET /api/engagements/{id}/progress rather than the engagement page — it names the phase and the command the run stopped on. If the run was marked completed, the findings and the report are there and tell you how far it got, and so is the locker if sealing succeeded. If it was marked failed, there is nothing to hand an auditor: create a fresh engagement rather than trying to recover the old one, since failed is terminal and cannot be resubmitted.