ERT Platform: Transparent Execution with Opaque Implementation

Research Log

1. What Was Built

The ERT host platform progressed from a minimal prototype into a more coherent evaluation system. Public-safe platform capabilities include:

• structured ERT evaluation flow,
• ERC outcome reporting,
• evidence sufficiency distinctions,
• public-safe diagnostic summaries,
• signed evaluation reports,
• offline verification concept,
• signed test-pack manifest concept,
• replay metadata,
• containerized deployment direction,
• public-safe Trace Viewer,
• and API integration testing.

[REDACTED — private platform architecture and evaluator implementation details]

2. Why Signed Reports Matter

Signed reports help establish that a report has not been casually altered after generation.

Public-safe explanation:

A signed report allows the evaluator to produce a result that can later be checked for integrity.

This supports a trust chain where both the evaluation input artifact and the public report can be checked.

Public-safe trust-chain framing:

signed test-pack manifest
→ verified pack provenance
→ ERT evaluation
→ ERC outcome
→ signed public report
→ verification
→ public-safe trace visualization

[REDACTED — operational signing configuration, key-management procedure, and private verification setup]

3. Certification Outcomes

The platform distinguishes between three broad outcome types:

• ERC tier — sufficient evidence and reliable observed behavior under the test conditions.
• No Certification — sufficient evidence was present, but a major observable reliability failure occurred.
• Insufficient Evidence — the evidence was weak, incomplete, degenerate, or not enough to support a reliable outcome.

This distinction is important because weak evidence should not be mistaken for failure, and weak traces should not be over-certified.

4. Public Report Boundary

A major platform goal is to make public reports useful without exposing private internals.

The public report may expose high-level fields such as:

• report version,
• run identifier,
• timestamp or report timing metadata,
• input summary,
• public diagnostic summaries,
• ERC outcome,
• replay metadata,
• and signature information.

The public report should not expose:

• private keys,
• raw private datasets,
• raw internal matrices,
• private calibration data,
• private scoring keys,
• implementation-sensitive internals,
• or protected evaluator logic.

[REDACTED — complete public report schema boundary and forbidden internal-field tests]

5. Verification

The platform includes a verification direction so public reports can be checked after generation.

Public-safe explanation:

Verification helps confirm whether a signed report still matches its original signed content and whether key report fields remain intact.

The exact operational commands, local paths, and configuration details should not be published on the research website.

[REDACTED — local verification commands, file paths, operational endpoints, and deployment-specific configuration]

6. Public-Safe Demo Artifacts

The platform direction includes public-safe example reports that can demonstrate different outcome states without claiming performance for an external model.

Public-safe outcome examples include:

• insufficient evidence,
• no certification due to major observable failure,
• and baseline-or-higher ERC outcome demonstration.

These are demonstration artifacts, not claims about a deployed model or third-party system.

[REDACTED — sample report internals and demonstration pack contents]

7. Trace Viewer

The Trace Viewer is intended as a public-safe inspection interface. It consumes only public report data and does not connect directly to private evaluator internals.

It may show public-safe information such as:

• signature status,
• ERC outcome,
• evidence status,
• diagnostic summaries,
• public geometry or control summaries,
• and replay payload identity.

The viewer is intentionally separated from private evaluator internals.

[REDACTED — internal viewer integration details and private field-blocking implementation]

8. Deployment Direction

The platform can be run locally or through a containerized environment. The public point is not the specific deployment command. The public point is that private key material and sensitive configuration should not be baked into the public runtime image.

Public-safe deployment principle:

deployable evaluation host;
private material supplied separately;
public report surface constrained by design.

[REDACTED — ports, paths, runtime commands, and environment-specific deployment values]

9. What Changed in Direction

This platform stage moved ERT from a conceptual evaluation method toward a demonstrable evaluation workflow.

The major shift was:

from “we can describe reliability evaluation”
to “we can generate, sign, verify, and inspect public-safe evaluation reports.”

10. What Remains Unresolved

Open work includes:

• preparing external demos,
• expanding public-safe test packs,
• improving documentation,
• validating larger report libraries,
• integrating live providers later,
• and strengthening separation between public inspection and private evaluator internals.

Public Boundary

The platform concept, public report boundary, verification principle, and Trace Viewer concept can be discussed publicly. Exact operational commands, keys, private evaluator internals, local configuration paths, private datasets, scoring methods, and implementation-sensitive details should remain private.

[REDACTED — additional private platform notes reserved for controlled disclosure]