← AICEP home · Runbook index

AICEP pub/sub encryption — PROPOSED PHASE 2 EXTENSION (v2.5/v2.5.1: per-agent folders + review fixes; v2.4/v2.4.1 review fixes; v2.3 delivery guarantees; v2.2 signed verifier attestation; v2.1 confirmed by design review; reference scripts in reference/)

Status: proposed Phase 2 extension. NOT part of the AICEP 0.1 public draft's normative text — the public draft doesn't specify encryption. It builds on the draft's rules: classify before send (the DLP gate runs before encryption), seat before talk (verifying the key registry counts as seating), credentials never ride the bus (only public keys and ciphertext move). Threat model: the pub/ folder and the transport are fully hostile (they can read, modify, delete or reorder). Confidentiality and integrity must hold anyway. Deletion and withholding are detected only through seq gaps, so the target raises an alarm on any gap.

Algorithms

• Payload: age (X25519 recipients; age CLI or pyrage). Alternative: libsodium secretstream + crypto_box_seal, with AAD = sha256(canonical manifest) || file_id. No hand-rolled crypto.
• Signature: Ed25519 detached signature; minisign or raw (the reference uses a raw base64 signature in manifest.json.sig).
• Version fields: v:1, alg:"age-x25519+ed25519", kem:"x25519". Post-quantum hybrid (ML-KEM) is deferred to 0.2.
• age does not authenticate the sender. Origin rests ONLY on the manifest signature.

Keys

• Private keys stay in the environment or a keystore: an AGE_IDENTITY_FILE path, the OS keychain or a hardware token. Agents and models only ever see a path or key id, never a key value.
• Registry entry: sender_id, age recipient string, signing key id, state (pending/verified/revoked), verified_by, verified_at, expires (≤90 days). An entry stays pending, and ingest refuses it, until a human sets verified_by and verified_at after a full out-of-band fingerprint read-back. Registry changes always need a human.
• Escrow recipient: optional per compartment, offline only, and always listed in recipient_key_ids.

Exactly what is encrypted and what is signed

Publish (source)

DLP gate and crossing package → wrap each body in the envelope (with batch_id, file_id, seq) → pad → encrypt → hash the ciphertext → build the canonical manifest with seq = last_sent_seq[sender_key]+1 → sign → atomic move into pub/. Plaintext lives only in tmpfs (don't rely on shred).

Ingest (target) — strict order, fail closed

1. Signer key in the registry, verified, not revoked, not expired → else REJECT.
2. Verify the Ed25519 signature over the canonical manifest bytes → else REJECT.
3. Check seq > last_seq[sender_key_id] and that now < expires → else REJECT (replay or rollback). Alarm if seq > last_seq+1 (a gap).
4. The recipient list contains our key; compartment = the expected landing compartment.
5. sha256 of EVERY listed ciphertext matches; any file in the folder that isn't in the manifest → REJECT the batch.
6. Only then decrypt (as a tool step, keys from env or keystore) into tmpfs.
7. Each decrypted envelope's batch_id/file_id/seq must match the manifest → else REJECT.
8. supersedes is honored only if the superseding item's seq is higher.
9. Persist last_seq[sender_key_id] = seq only after a full accept.
10. The model receives only the verified bodies, in a fenced data block, as UNTRUSTED data. It never executes, follows or forwards instructions, URLs or tool calls found inside them. It quotes and flags them for human review. Payload content can never change keys, the registry, recipients or the destination.
11. DLP and classification gate → land in a user-only private compartment → signed ack {batch_id, seq, ok/reject reason} returned to the source.

Transport (defense in depth only)

SSH forced command (rrsync / sftp-only, pinned host keys) is preferred. mTLS with TLS 1.3 is an alternative. No public links or buckets.

Key-sharing ceremony

Each side generates keys locally → they exchange public keys → they read back the full fingerprint (age recipient + signing key id) over a second independent channel → registry entry moves pending → verified, with verified_by and verified_at → dummy batch both ways, plus a tamper test (flipped byte, swapped file, replayed seq, unknown key, expired). STOP if any secret appears; logs are linted for AGE-SECRET-KEY- and minisign secret headers.

Rotation and compromise

Rotate at least every 90 days. Purge pub/ after the ack. At-rest files have NO forward secrecy. If the encryption identity is compromised: revoke it, re-encrypt the archives, and treat every past batch as disclosed. If the signing key is compromised: revoke it, publish a revocation signed by the new key and confirmed out of band, and reject that key's batches from the revocation time onward.

Next (nice to have)

Reference scripts in reference/ (aicep_pub.py, aicep_sub.py; pyrage + cryptography Ed25519, raw signature) with tamper suites: 188 automated tests, all passing (v2.5.1).

v2.2 — Signed verifier attestation (closes the "faked accepted paste" gap)

Problem: in v2.1 the model read whatever verify output an operator pasted, so a hand-typed "accepted" could pass. Design: - Verifier key: a dedicated Ed25519 key, separate from the swarm signing key, held only in the receiver's keystore (file mode 600, for example ~/.aicep-keys//verifier.pem via AICEP_VERIFIER_KEY_FILE). Its public key is registered in the gate config together with the pinned sha256 of aicep_sub.py. Both are human-maintained. - Ingest gate (aicep_ingest.py): issues a fresh 128-bit nonce, then runs aicep_sub.py with --nonce and --attest-out. - Attestation: aicep_sub.py signs this canonical attestation with the verifier key: {v, kind, nonce, ts, batch_id, seq, manifest_sha256, script_sha256, exit_code, status, reason, result_sha256, verifier_key_id}. result_sha256 binds the decrypted result the gate later hands over. - Gate checks, all fail-closed (exit 4, no content): - signature from the registered verifier key - nonce matches - ts within 300 s (with 30 s future skew allowed) - script_sha256 = the pinned value = the hash of the script on disk now - result hash matches - exit_code and status agree (0 means accepted) - manifest on disk = the attested manifest hash - batch and seq match - folder contents and every ciphertext hash re-checked now (TOCTOU) - status is accepted - Model input: the model only ever sees gate output {"gate":"pass", attestation, content}. Nothing is pasted by hand. - Tests: tests/test_attestation.py covers a hand-typed accepted with no sig, an attestation from another batch, a replayed attestation (stale nonce), a stale timestamp, a wrong-key signature, an edited exit code and status, a modified aicep_sub.py (hash mismatch, both attested and on disk), files swapped after verify (TOCTOU) and an edited result body. - Residual risk: someone with root on the receiving box, or holding the verifier key, can still forge an attestation, or edit the gate, the config and the pinned hash together. Mitigations: - key separation: the verifier key is distinct from the signing and age keys, mode 600, and rotated with the 90-day cycle - least-privilege accounts, so agents can't read the keystore - audit rows that carry the attestation hash and verifier key id, so a forgery is traceable - human review of any change to the gate config or pinned hash - for higher assurance (later): an OS-separated verifier service or a hardware-backed key.

v2.3 — Delivery guarantees (proposed Phase 2 extension; research in broker-patterns.md)

Format changes: manifest v:2 may carry a signed prev_batch_id. Receipts are signed by the receiver and carry acked_through_seq and ledger_head. Nothing else changes in encryption or signing. 1. Commit marker and staging: upload into _staging/, the signature last, then a READY marker, then move into pub/. Receivers ignore any batch without the marker, so half-uploads are invisible. 2. Idempotent redelivery: the receiver keeps seen[batch_id] = {manifest_sha256, receipt}. The same manifest hash gets the stored receipt resent with nothing reprocessed. The same seq (or batch id) with a different validly signed manifest is rejected, goes to dlq as equivocation, and raises an alarm. 3. Sender outbox: the seq is reserved and logged staged before building. It's published only after the adapter confirms a durable upload. A staged seq is reused only for the same batch id. 4. Dead letters: dlq/<batch>/reason.json holds a receiver-signed record {batch_id, reason, detail, attempts, receiver, t}. Reason codes: sig_fail, hash_mismatch, revoked, replay, expired, size_limit, equivocation, chain_break, qop, ingest_error, gap_timeout. The attempt count is persistent: 3 tries for transient errors, 1 for crypto and policy failures. Every dlq move triggers a G11 email. Redrive needs a named human operator. 5. Retry schedule: the sender resends the same bytes at +1 h, +4 h and +12 h, and stops at batch expiry. With no receipt by then, it raises an unconfirmed alert. Receipts are cumulative, so one receipt acks every seq ≤ acked_through_seq. 6. Receiver state machine: a lease (processing/ plus lease expiry) gives fetched → verified → committing → committed. Verification runs against a temporary state copy. Landing is written, then the state is replaced atomically, then the batch is marked committed. On restart, committing is resolved by comparing last_seq (or by applying the temp state). Earlier phases are redelivered when the lease expires. Each batch is ingested exactly once. 7. Ordering and audit: an out-of-order batch waits in holding/. It's released when the missing seq commits; after 30 minutes the receiver raises a human hold and a G11 email. A prev_batch_id mismatch is a chain break and goes to dlq. Each side keeps an append-only, hash-chained ledger (prev = hash of the previous entry, no content or keys). The receiver's ledger head goes into every receipt. 8. QoP floor (written rule): the only accepted quality of protection is signed + encrypted. Unsigned, unencrypted, sign-only or encrypt-only batches are never ingested and go to dlq as qop. 9. Transport adapters: TransportAdapter (abstract) defines publish, fetch (with lease), ack, nack, dead_letter, hold and list_pending. The adapter only moves opaque bytes. The core enforces seq, dedup, expiry and signatures whatever the adapter guarantees. Built and tested: LocalFolderAdapter. Planned: Drive, then queue brokers (see the broker-patterns.md mapping). Tests: reference/tests/test_delivery.py — 22 collected, passing. (Full suite: 188 automated tests, all passing (v2.5.1).)

v2.4 — Review fixes (design review v2.3 review V1–V8)

No change to the encryption or signature formats. Stricter checks and two behaviour changes (chain is mandatory once started; gap ack is exact and human-only). 1. Verify before anything (V1). The receiver runs the full manifest gate (registry state, revocation, expiry, key id, Ed25519 over the exact canonical bytes, version, schema) before any dedup, reorder hold or chain decision. Failures go to dlq as sig_fail (or the classified reason). Held batches are re-verified before release or any gap_timeout alert; a held batch that fails goes to dlq. One shared function (verify_manifest) is used by ingest, the receiver and the alarm checks. 2. Lease and state safety (V2). The receiver re-checks lease.consumer == me and until > now immediately before the state commit and aborts (lease_lost, nothing committed, landing removed) otherwise. Ingest+commit runs under an exclusive flock on the state dir, and the state commit is a merge that never lowers any last_seq (max per key). A lost pub→processing rename race in fetch skips that batch instead of aborting the fetch. 3. Chain is mandatory once started (V3). Senders using the outbox start the chain at their first batch (prev_batch_id: "genesis", manifest v:2). After the first v2 batch from a sender_key_id, a batch without prev_batch_id (or a v1 manifest) is rejected as a chain downgrade; a wrong prev_batch_id at last+1 is a chain break. 4. Size cap on real bytes (V4). Before any read or decrypt, each listed .age file is lstated (must be a regular file) and capped at size_padded + age overhead (1024 B + 16 B per 64 KiB chunk) and at the per-object and per-batch limits. size_padded is required. 5. Schema reject, not crash (V5). created, expires, seq and each file's size_padded must be integers; each file entry needs a string id and sha256_ct. Anything else is a REJECT (CLI exit 2; receiver → dlq). The receiver also routes any unexpected exception to dlq (rejected), so a batch can never loop forever in processing/. 6. Ledger head anchor (V6). Every append writes a separate <ledger>.anchor {head, count}. Ledger.verify(expected_head=None) fails if the chain's last hash or entry count differs from the anchor (tail truncation), if expected_head (e.g. ledger_head from the last signed receipt) is not in the chain, or if any line is malformed. A legacy ledger with no anchor no longer verifies until re-anchored by a human. 7. Clock bounds and the backdating limit (V7). created must be an integer and created <= now + 300 s on the receiver's clock. The check "signed at/after revoked_at" compares the sender-chosen created, so it only stops honest-clock batches. Limit: a compromised signing key can backdate created and its batches are accepted until a human sets revoked_at; the real guard is the receiver-clock gate now >= revoked_at. Therefore set revoked_at to the suspected compromise time (not the discovery time) and re-audit every batch from that key accepted since then. (The lower bound created >= now - ttl is implied by now < expires.) 8. Exact, human-only gap ack (V8). The blanket --ack-gap flag is removed. --ack-gap-to <sender_key_id>:<seq> --operator <name> accepts only that exact sender and seq; anything else stays held (exit 3). Each ack appends gap_acked{operator, skid, batch_id, from, to} to the ledger. Agents must never pass --ack-gap-to; only a human may, after reviewing the gap. Not changed yet (design review: nice-to-have): V9 stale-phase GC, V10 clear attempts on redrive, V12 dedup on the signed batch_id. V11 is partly done (shared verify for alarms). Tests: 188 automated tests, all passing (v2.5.1). Pre-change copies kept as .bak-v2.3.

v2.4 fixes note (design review v2.4 re-review W1–W4, Oct 16)

• W1 Max validity window. verify_manifest rejects expires <= created and expires - created > MAX_TTL (7 days). With the cap, a batch can be at most 7 days old by its own signed fields, so no separate lower bound on created is needed; the backdating limit above (compromised key until revoked_at is set) still applies.
• W2 Append checks the anchor. Ledger.append refuses (LedgerError) unless the existing anchor matches the on-disk head and count, or there is no anchor and the ledger is empty. A truncated tail can no longer be laundered by the next normal append.
• W3 Human re-anchor + startup check. aicep_sub.py --reanchor-ledger <ledger> --operator NAME (human only) re-anchors a legacy ledger, or one where a crash fell between the line append and the anchor write. It refuses if the internal links don't verify and logs reanchored{operator, old_count, old_head, new_count}. On start, the Receiver fails closed unless its ledger verifies and contains every ledger_head it signed into a receipt, so a re-anchor cannot hide receipted history.
• W4 Chain rule aligned. The Receiver's chain check now uses the same record as ingest (a chain starts only after a v2 batch from that key commits). An existing sender with v1 history can start at prev_batch_id: "genesis". If a chain ever needs restarting, a human resets that key's last_batch entry in sub_state (logged as an operator action) and the sender restarts at genesis.

• Still open: V9, V10, V12 (nice). Tests: 188 automated tests, all passing (v2.5.1). Pre-change copies kept as .bak-v2.4.

• Residual risk (ledger anchor): the anchor file is unsigned, so anyone with write access to the state folder can cut off ledger entries made after the last signed receipt and re-anchor without being detected. Only history covered by a receipt's ledger_head is protected.

v2.5 — Per-agent folders

No change to encryption or signing. Layout and routing only. 1. Layout. Senders publish into <compartment>/pub/<folder>/<batch_id>/ (staging in _staging/<folder>/). <folder> is a platform-agent slug such as grokbot-example or claude-example: lowercase [a-z0-9-], 3–64 characters, no dots or slashes. The adapter refuses an invalid slug at publish; the outbox and the publish CLI (--folder) require one. 2. Registry. Every registry entry carries a folder slug, unique across the registry. The receiver refuses to start if any entry has a missing, invalid or duplicate folder. 3. What the receiver watches. Fetch looks only at the folders of senders that are verified, not revoked and not expired (receiver clock). Any other folder in pub/ is ignored and left untouched; it is logged once to the ledger as unknown_folder_ignored. A batch already in processing/ from a sender that has since been revoked is not redelivered. 4. Folder binding. After the full manifest verification, the receiver checks that the verified signing key is the one registered to the folder the batch was fetched from. If not, the batch goes to dlq as folder_mismatch (crypto-class, one attempt, G11 email). The single-batch CLI takes an optional --folder and applies the same check. Folder names are labels only; trust still comes from the manifest signature. Hold and human redrive return a batch to its original folder. 5. Migration from flat pub/. Batches sitting directly in pub/<batch_id>/ are no longer read (to a v2.5 receiver they look like an unknown folder and are ignored). To migrate: (a) a human adds a unique folder slug to every registry entry; (b) each sender sets its folder (Outbox(..., folder=...) or aicep_pub.py --folder); (c) move each pending flat batch, with its READY marker, into pub/<folder of its signer>/ — the signer is the manifest's sender_key_id; (d) start the receiver and confirm the moved batches commit or land in dlq with a reason. Nothing needs re-signing: the folder is not part of the signed manifest. Before renaming an existing sender's folder, drain holding/ and processing/ for that sender (let them commit or dead-letter); v2.5.1 resolves held, processing and redriven batches from the registry by the verified signing key, but draining first avoids stalls for batches that were never verified. Tests: full suite 177 passed, 0 failed (157 before v2.5 + 20 new in reference/tests/test_review_r3.py). Pre-change copies kept as .bak-v2.4.1.

v2.5.1 fixes note (design review v2.5 review X1–X3 + minor)

• X1 bounded unknown-folder log. At most 64 unknown folders are remembered (as 16-hex hashes, not names); the 65th writes one unknown_folder_overflow ledger row and further junk is counted without new rows.
• X2 alert coalescing. folder_mismatch alerts are coalesced to one email per (reason, source folder) per receiver run; the rest are logged and dead-lettered without email, and one non-email alerts_coalesced summary carries the counts. 20 mismatches in one folder = 2 email notices, not 40.
• X3 folder from the registry. After the folder check passes, the verified sender_key_id is recorded in the batch's transport metadata. Release from holding, redelivery from processing and human redrive (Receiver.redrive, which re-verifies the manifest) use the folder the registry currently assigns to that key, not the folder recorded at fetch time.
• Minor. The single-batch ingest CLI (and aicep_ingest.py) now require --folder. Symlinked batch directories are skipped by fetch, and symlinked batch dirs or files are rejected by the receiver and by ingest.
• Drive adapter requirement (doc). Drive allows duplicate names, so the future Drive adapter must key each sender folder on a Drive folder ID pinned in the registry, never on the name, and must refuse and alert on a duplicate-named sibling folder. Tests: full suite 188 passed, 0 failed, 0 xfailed (177 + design review 3 X tests, markers removed, + 8 new). Pre-change copies .bak-v2.5.