Retesting after remediation

Pentr has no retest feature. A test after a fix is a new engagement, and the comparison with the earlier run is done by hand. This page describes what the product does with a finished engagement and how to run the second test.

A finished engagement is not run again¶

An engagement can be submitted while its status is pending or blocked. A submit request for an engagement in any other status, completed included, is refused with a 400. A completed engagement keeps its one run.

Its results are fixed too. When the results of a run arrive for an engagement that is already sealed, the server writes nothing. The findings, the locker, the seal and the report stay as they were.

Running the test again¶

Create a new engagement with the same targets, scope, rules of engagement and framework, and run it. The console has no action that copies an existing engagement, so the wizard is filled in again.

The new engagement goes through the same launch checks as the first, including ownership verification. An attestation that has expired since the first run blocks the launch.

A completed engagement does not count against the plan's limit on concurrent engagements. That limit counts engagements that are queued or running.

The new engagement gets a new id, a new run, a new evidence locker, a new seal and a new report. The engagement record has no field that refers to another engagement.

A schedule creates a new engagement from a stored template at every occurrence it does not skip, which gives a repeated test over the same scope. The engagements it creates do not record which schedule created them.

What carries over¶

No finding, status or remediation note carries over. remediation is free text the agent writes on a finding in that run. The product does not track it, close it, or copy it into a later run. A finding's status is the value the agent set during its own run; none of the four values means fixed.

Comparing two runs¶

Compare the two reports yourself. The CSV export is the easiest to compare programmatically: one row per finding, sorted by severity and title, with the status in its own column. SARIF and CSV exports describes the columns.

Finding ids differ between runs. The pair (title, affected_asset) is a reasonable key to compare on. It is the key the agent uses to drop duplicates within a single run; the product itself does not compare findings across runs. A title is free text written by the agent in each run, so the same issue can carry a different title the second time.

A finding missing from the second run is not evidence that it was fixed. A run can end early and still be completed and sealed, as Sealing describes. Check how far the second run got before reading an absence as a fix.

If the framework file changed between the two runs, their control ids can differ for the same finding. Compare on findings, not on control ids.