Your pen test report is a PDF you have to trust

Most penetration tests end the same way. Testing stops on a Friday, and the following week someone writes a report. A few dozen pages arrive as a PDF: an executive summary, a scope statement, a findings table, and a screenshot for each finding. The auditor reads it, checks the dates and the firm's name, samples two or three findings and files it.

What the auditor cannot do is check any of it. The report is a description of the work, written after the work, by the people who did it. Everything in it is a claim.

That has worked for a long time, and it mostly still works. But it is worth being precise about what is being trusted, because the conditions around it are changing.

What the auditor is actually trusting¶

Take a typical finding: an SSRF in an import endpoint that reaches the cloud metadata service. In the report it appears as a paragraph, a CVSS score and a screenshot of the response. To accept it, the auditor has to trust that:

• the screenshot came from the system in scope, on the date stated, and not from a lab or a previous engagement;
• the request that produced it was the one described, and nothing was left out of the chain;
• the scope page at the front of the report matches what the tester actually touched;
• nothing was added, removed or softened between the test and the final version.

None of this is a criticism of pentesters. The format was designed for a reader who trusts the author, and most of the time that trust is earned. The report is a professional opinion, and it is signed by a firm with a reputation to lose.

The strain shows in two places. The first is frequency. PCI DSS expects testing at least annually and after significant changes, and SOC 2 reviewers increasingly want evidence of continuous monitoring. The number of reports grows, and the time anyone spends checking each one shrinks. The second is automation. When part of the work is done by an agent, the auditor's question changes from "do I trust this firm?" to "what did the software actually do?" A PDF written afterwards is a poor answer to that question, however well written it is.

Keep the test, not a description of it¶

The alternative we settled on is simple to state: do not write a document about the test. Keep the test.

In Pentr, every artifact the agent produces is captured at the moment it exists. That means each HTTP request and response, each command with its stdout and stderr, each screenshot, and the manifest of what each reconnaissance phase enumerated. Each one is stored as a discrete entry, with a SHA-256 digest of its bytes and metadata recording its position in the run, its kind, the phase, the controls and findings it relates to, and when it was captured.

Each entry's hash also covers the hash of the entry before it:

entry[n].hash = sha256( entry[n-1].hash + sha256(artifact) + metadata )

The chain starts from a value derived from the run's ID, so a chain lifted from one run does not verify under another.

When the engagement ends, whether it completed or was aborted, the head of the chain is signed with an Ed25519 key. Because every entry points back to the one before it, signing the head covers the entire history.

Three properties follow, and none of them depends on anyone's good intentions:

1. Nothing can be edited quietly. Change one byte of one artifact and its digest changes, which breaks every link after it.
2. Nothing can be removed or reordered. The chain records order, so a deleted screenshot leaves a gap that verification reports.
3. Scope is sealed with the work. The signed manifest carries a digest of the run's in-scope and out-of-scope lists and its rules of engagement. Anyone told what the scope was can recompute that digest and compare it. Change the scope after the fact and it no longer matches, so the scope and the work cannot drift apart.

None of these ideas is new. Signed, append-only logs are how certificate transparency works, and how trust service providers keep records that have to stand up years later. We have applied them to a pen test.

Verification that does not involve us¶

A record like this is only useful if the auditor can check it without asking the vendor. If verification means logging into our console, we have only moved the trust from the tester's PDF to our server.

So verification runs offline, against an exported locker, with a public key:

$ pentr-verify pentr-locker-3f9a1c2e.zip --key acme.pem
OK      0  command_log      f869ee4c0b89  -
OK      1  req_resp         2482ffa1be00  -
OK      2  screenshot       075f8e3f22fc  screenshot.png
…
PASS — evidence locker verified

  run          3f9a1c2e-7b44-4d21-9a6e-0c5b8e2d14f7
  artifacts    38
  chain head   af08f99ccd6af021f46396c82a2cd859c6296f32b28af9f2291e6ae7e0ae0bab
  signed by    5f79492518ca52f60086291848a46ef8c9a852f7cb0394509b0fa552541a3d69
  signed at    2026-08-26T01:04:11.229107+00:00

The verifier recomputes every digest, walks the chain and checks the signature. If anything fails, it marks the artifact BAD and says what broke. It needs no account and no network access. The public key should come from the organization through a channel you trust, not from the locker itself. Run without --key, the tool still checks everything, but it says plainly that it has proved the locker is intact, not who signed it. The verifier is open source, so anyone who prefers not to trust our binary can read it, build it and run it in their own CI.

What a signature does not prove¶

This is the part vendors tend to skip, and auditors should ask about it.

A valid seal proves the record was not changed after it was sealed. It does not prove the test was thorough, or that the declared scope was the right one. Coverage is still a judgment, and the locker makes that judgment reviewable, not unnecessary.

The timestamps are the ones our runner recorded. They are consistent and chained, but they are not qualified timestamps from an independent authority. For most audits that is enough. If you need more, tell us what your auditor expects.

The signer matters. Today, lockers are sealed with an Ed25519 key that Pentr holds, encrypted at rest, and signs with in software. You can generate that key or upload one of your own, but either way we hold it. A sealed locker cannot be altered without verification failing, but we are technically able to sign on your behalf, and we say so in the documentation and in every locker's README. If what you need is independence from us as the signer, the key should live somewhere we cannot reach. Signing with your own HSM or KMS key, and running the agent on premises, are on the Enterprise roadmap.

A finding is still a judgment. Severity and CVSS are opinions. What changes is that the opinion now sits next to the exact request and response it is based on, so a reviewer can disagree with it on evidence.

What changes in practice¶

For the auditor, sampling a finding stops being a matter of reading a paragraph. They open the artifact the finding points to and read the actual exchange. The control mapping (PCI 11.3.1, ASVS V5, ISO A.8.8) is attached when the artifact is captured, not assembled the week before the audit.

For the security team, less time goes into writing and formatting. A retest after remediation is another sealed entry in the same record, not a new paragraph in a revised PDF.

The PDF does not disappear. Most auditors will still want a summary, and people still need something to read on a train. The difference is that the summary becomes an index to the evidence rather than the evidence itself.

If you want to see what this looks like, the locker page walks through a sealed run, and the verifier documentation shows how to check one yourself.