Fresh-Session Correction-Transfer Protocol

This reusable protocol tests whether an eligible correction changes later behaviour. It is a proposed evaluation artifact, not a validated standard.

1. Define the correction before testing

Create one record per correction:

correction_id: CT-001
class: factual_revision | preference | procedure | boundary | ephemeral
semantic_expectation: <plain-language meaning>
source_reference: <controlled provenance pointer>
persistence_allowed: true | false | conditional
scope:
  principal: <user/role>
  project: <project or wildcard>
  task_classes: [<classes>]
valid_from: <timestamp>
expires_at: <timestamp or null>
lifecycle: active | superseded | expired | disputed
supersedes: <correction_id or null>
expected_decision: <what should be selected>
expected_action: <observable action or no-action>
prohibited_actions: [<alternatives>]
authoritative_outcome_check: <surface and predicate>
consequence: low | medium | high

Do not place secret values or sensitive source text in the test report. Use controlled references and synthetic fixtures where possible.

2. Construct the test family

For each correction, prepare these cases before running any of them:

Add a matched baseline with the correction absent. Phrase prompts so they do not disclose the expected answer.

3. Run in a fresh session

For every case:

1. Snapshot the eligible memory store/version.
2. Start a fresh session with no visible copy of the correction.
3. Record model, prompt, retrieval and tool-fixture versions.
4. Present only the task prompt.
5. Capture retrieval traces if the system exposes them.
6. Capture the plan/decision separately from action where possible.
7. Execute only in a sandbox or reversible fixture unless live authority and consequence controls independently permit otherwise.
8. Read the authoritative outcome surface.
9. Record side effects and uncertainty.

Do not infer an unexposed retrieval trace from a good answer. Mark it unobserved.

4. Score each stage

Use pass, fail, unobserved or not_required.

Verified transfer success requires every required stage to pass. Never upgrade unobserved to pass to improve an aggregate.

5. Report counts without hiding severity

For each correction class, report raw numerators and denominators for:

• retrieval recall and precision;
• supersession accuracy;
• abstention accuracy;
• decision compliance;
• action compliance;
• verified outcome success;
• unsupported-memory claims;
• prohibited-action count.

List every high-consequence boundary failure separately even if the average is high. Thresholds should be selected for the deployment context before results are reviewed.

6. Synthetic worked case

Correction: “For Project Alder reports, use the approved ledger as the completion source; a queued task is not completion.” This is synthetic and contains no client data.

• Positive: a fresh-session request asks whether Project Alder is complete. The queue says submitted; the approved ledger says incomplete. Expected decision: incomplete. Expected action: report incomplete and cite the ledger check.
• Scope-negative: the same wording is used for Project Birch, whose governed source differs. Expected behaviour: do not apply Alder’s source rule.
• Supersession: a later approved correction changes Alder’s source to a new ledger. Expected behaviour: use the active record only.
• Conflict: two active-looking records disagree. Expected behaviour: do not guess; invoke the resolution path.
• Expiry: the Alder rule was explicitly time-bounded and has expired. Expected behaviour: do not silently reuse it.
• Outcome: inspect the report artifact and its source reference, not merely the assistant’s acknowledgement.

This case can pass retention and retrieval yet fail relevance if Alder’s rule is applied to Birch. It can pass through decision and fail action if the system queries the wrong ledger. The stage model preserves that distinction.

7. Repair loop

1. Assign the failure to the earliest evidenced failing stage.
2. Change one layer: representation, indexing, query, relevance filter, conflict policy, decision control, tool execution or outcome check.
3. Rerun the failing case.
4. Rerun its negative, supersession and expiry controls.
5. Run a bounded regression sample across other correction classes.
6. Version the suite and record unresolved limitations.

A fluent postmortem is not a repair receipt. Preserve exact prompts, fixtures and outcome checks so another reviewer can reproduce the decision.