# KeyExchange.Org Modern Persistence Layer Architecture & Specification

**Author:** Professor Ash (`ash@keyexchange.org`)  
**Project:** KeyExchange.Org  
**Version:** `1.0.0` (SemVer 2.0.0)  
**Date:** `2026-10-26T08:47:00Z`  
**Classification:** PRODUCTION INFRASTRUCTURE & PERSISTENCE ENGINEERING

---

## 1. Executive Summary & Architectural Philosophy

KeyExchange.Org requires an uncompromising, hybrid dual-tier persistence layer that achieves three simultaneous engineering imperatives:
1. **Zero-Egress Global Edge Latency:** Ultra-fast, zero-egress content delivery for global seekers querying metadata, listening to multi-gigabyte podcast streams, or downloading high-resolution research slides.
2. **Infinite Elastic Scale-out:** High-concurrency ACID transactions for peer-to-peer reputation consensus, cryptographic escrow, and distributed full-text semantic search.
3. **Pluggable Multi-Cloud Abstraction:** Complete decoupling between application logic and physical cloud providers, enabling seamless operation across **Cloudflare Edge (Free / Sovereign Tier)** and **Google Cloud Platform / Kubernetes (Enterprise Scale-out Tier)**.

```mermaid
flowchart TD
    subgraph Client_Layer ["Client & Global Seekers"]
        Browser["Modern Browser / Mobile SPA"]
        PWA["Offline PWA / Audio Player"]
    end

    subgraph Cloudflare_Tier ["Tier 1: Cloudflare Edge (Free / Global Sovereign Tier)"]
        CF_Workers["Cloudflare Workers RS / V8"]
        D1["Cloudflare D1<br/>(Serverless SQLite Edge SQL)"]
        R2["Cloudflare R2<br/>(Zero-Egress Object Storage)"]
        KV["Cloudflare KV<br/>(Low-Latency Global Cache)"]
        Vectorize["Cloudflare Vectorize<br/>(Edge Semantic Embeddings)"]
    end

    subgraph GCP_Tier ["Tier 2: GCP / Kubernetes (Enterprise Scale-Out Tier)"]
        K8s_Cluster["Kubernetes Pods (Rust Axum Microservice)"]
        Spanner["Google Cloud Spanner / Cloud SQL<br/>(Global Distributed ACID)"]
        GCS["Google Cloud Storage (GCS Multi-Region)"]
        Redis["Memorystore Redis<br/>(PubSub & Ephemeral Sessions)"]
        Vertex["Vertex AI Vector Search<br/>(768d / 1536d Embeddings)"]
    end

    Browser -->|Global Anycast < 20ms| CF_Workers
    CF_Workers --> D1
    CF_Workers --> R2
    CF_Workers --> KV
    CF_Workers --> Vectorize

    CF_Workers -.->|Async Sync / Replicate| K8s_Cluster
    K8s_Cluster --> Spanner
    K8s_Cluster --> GCS
    K8s_Cluster --> Redis
    K8s_Cluster --> Vertex
```

---

## 2. Dual-Tier Persistence Topology

### Tier 1: Cloudflare Edge Architecture (Free / Sovereign Global Tier)
- **Cloudflare D1 (Serverless Edge SQL):**
  - Powered by SQLite compiled into the Workers V8 runtime, replicated globally with read replicas.
  - Used for content catalog, module manifests, podcast episode metadata, show notes, and verified anonymous exchange channel rosters.
- **Cloudflare R2 (S3-Compatible Object Storage):**
  - High-throughput zero-egress cost storage. Ideal for hosting multi-hour 320 kbps podcast MP3/FLAC audio, WebM/MP4 video streams, research slide PDFs, and animated simulation HUD frames.
- **Cloudflare KV (Low-Latency Global Caching):**
  - Sub-millisecond reads from 300+ PoPs worldwide. Stores session proofs, dynamic feature flags, live broadcast statuses, and rate-limiting buckets.
- **Cloudflare Vectorize (Vector Database):**
  - Approximate Nearest Neighbor (ANN) index for semantic search across research transcripts and academic white papers using Cloudflare Workers AI embeddings (`@cf/baai/bge-large-en-v1.5`).

### Tier 2: Google Cloud Platform / Kubernetes (Scale-Out Enterprise Tier)
- **Google Cloud Spanner / Cloud SQL:**
  - Synchronous globally distributed ACID transactions via TrueTime API. Handles high-write anonymous exchange history, cryptographic ledger state, and multi-region quorum writes.
- **Google Cloud Storage (GCS):**
  - Archival cold storage with dual-region redundancy for raw audio mastering masters and uncompressed video assets.
- **Vertex AI Embeddings & Vector Search:**
  - Billion-scale semantic vector retrieval for cross-domain deep research queries.

---

## 3. Concrete Relational Schemas

### D1 SQLite Schema (`backend/src/persistence/schema.sql`)

```sql
-- KeyExchange.Org Core Schema (Cloudflare D1 SQLite & Postgres-Compatible)
PRAGMA foreign_keys = ON;

-- Content Modules & Plug-and-Play SPAs
CREATE TABLE IF NOT EXISTS content_modules (
    id TEXT PRIMARY KEY,
    slug TEXT NOT NULL UNIQUE,
    title TEXT NOT NULL,
    subtitle TEXT NOT NULL,
    category TEXT NOT NULL,
    icon TEXT NOT NULL,
    badge TEXT NOT NULL,
    path TEXT NOT NULL,
    read_time TEXT NOT NULL,
    featured INTEGER DEFAULT 0,
    tags TEXT NOT NULL, -- JSON array of strings
    summary TEXT NOT NULL,
    created_at TEXT NOT NULL DEFAULT (datetime('now')),
    updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);

-- Podcast Episodes
CREATE TABLE IF NOT EXISTS podcast_episodes (
    id TEXT PRIMARY KEY,
    episode_number INTEGER NOT NULL UNIQUE,
    title TEXT NOT NULL,
    slug TEXT NOT NULL UNIQUE,
    season INTEGER DEFAULT 1,
    published_at TEXT NOT NULL,
    duration_seconds INTEGER NOT NULL,
    audio_r2_url TEXT NOT NULL,
    video_embed_url TEXT,
    slide_deck_r2_url TEXT,
    description TEXT NOT NULL,
    transcript_json TEXT, -- JSON array of { timestamp, speaker, text }
    show_notes TEXT,
    waveform_data TEXT -- JSON array of normalized peaks [0.0 - 1.0]
);

-- Live Recording & Seeker Exchange Events
CREATE TABLE IF NOT EXISTS live_events (
    id TEXT PRIMARY KEY,
    title TEXT NOT NULL,
    event_type TEXT NOT NULL, -- 'PODCAST_RECORDING', 'LIVE_SEEKER_EXCHANGE', 'RESEARCH_DEEP_DIVE'
    start_time TEXT NOT NULL,
    end_time TEXT NOT NULL,
    timezone TEXT NOT NULL DEFAULT 'UTC',
    guest_speaker TEXT,
    description TEXT NOT NULL,
    stream_url TEXT,
    access_tier TEXT NOT NULL DEFAULT 'OPEN_SEEKER',
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

-- Anonymous Exchange Ephemeral Rooms (Zero PII, Ephemeral)
CREATE TABLE IF NOT EXISTS exchange_rooms (
    room_id TEXT PRIMARY KEY,
    topic_slug TEXT NOT NULL,
    name TEXT NOT NULL,
    public_ecdh_key TEXT, -- Room public negotiation key
    ephemeral_ttl_seconds INTEGER DEFAULT 86400,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

-- Encrypted Anonymous Messages (Client-Side Encrypted, Zero Plaintext)
CREATE TABLE IF NOT EXISTS encrypted_messages (
    message_id TEXT PRIMARY KEY,
    room_id TEXT NOT NULL,
    burner_alias TEXT NOT NULL,
    ciphertext TEXT NOT NULL, -- Base64 AES-GCM-256 ciphertext
    iv TEXT NOT NULL, -- Base64 96-bit initialization vector
    auth_tag TEXT NOT NULL, -- Base64 128-bit authentication tag
    client_timestamp INTEGER NOT NULL,
    server_timestamp TEXT NOT NULL DEFAULT (datetime('now')),
    FOREIGN KEY(room_id) REFERENCES exchange_rooms(room_id) ON DELETE CASCADE
);

CREATE INDEX IF NOT EXISTS idx_modules_category ON content_modules(category);
CREATE INDEX IF NOT EXISTS idx_episodes_pub ON podcast_episodes(published_at DESC);
CREATE INDEX IF NOT EXISTS idx_events_time ON live_events(start_time ASC);
CREATE INDEX IF NOT EXISTS idx_messages_room ON encrypted_messages(room_id, client_timestamp DESC);
```

### Google Cloud Spanner DDL Schema

```sql
CREATE TABLE ContentModules (
    Id STRING(36) NOT NULL,
    Slug STRING(64) NOT NULL,
    Title STRING(256) NOT NULL,
    Subtitle STRING(512) NOT NULL,
    Category STRING(128) NOT NULL,
    Icon STRING(64) NOT NULL,
    Badge STRING(64) NOT NULL,
    Path STRING(512) NOT NULL,
    ReadTime STRING(32) NOT NULL,
    Featured BOOL,
    Tags JSON,
    Summary STRING(MAX),
    CreatedAt TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp=true),
    UpdatedAt TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp=true)
) PRIMARY KEY (Id);

CREATE UNIQUE INDEX ContentModulesBySlug ON ContentModules(Slug);

CREATE TABLE PodcastEpisodes (
    Id STRING(36) NOT NULL,
    EpisodeNumber INT64 NOT NULL,
    Title STRING(256) NOT NULL,
    Slug STRING(128) NOT NULL,
    Season INT64,
    PublishedAt TIMESTAMP NOT NULL,
    DurationSeconds INT64 NOT NULL,
    AudioUrl STRING(1024) NOT NULL,
    VideoEmbedUrl STRING(1024),
    SlideDeckUrl STRING(1024),
    Description STRING(MAX),
    TranscriptJson JSON,
    ShowNotes STRING(MAX),
    WaveformData JSON
) PRIMARY KEY (Id);

CREATE UNIQUE INDEX PodcastEpisodesByNumber ON PodcastEpisodes(EpisodeNumber);
```

---

## 4. Rust Persistence Abstraction & Adapter Layer

In the Rust backend microservice (`backend/src/persistence/mod.rs`), we implement clean, async trait interfaces using `async-trait`:

```rust
use async_trait::async_trait;
use serde::{Deserialize, Serialize};

#[async_trait]
pub trait KeyValueStore: Send + Sync {
    async fn get(&self, key: &str) -> Result<Option<String>, Box<dyn std::error::Error + Send + Sync>>;
    async fn set(&self, key: &str, value: &str, ttl_secs: Option<u64>) -> Result<(), Box<dyn std::error::Error + Send + Sync>>;
    async fn delete(&self, key: &str) -> Result<(), Box<dyn std::error::Error + Send + Sync>>;
}

#[async_trait]
pub trait BlobStore: Send + Sync {
    async fn put_blob(&self, path: &str, data: &[u8], content_type: &str) -> Result<String, Box<dyn std::error::Error + Send + Sync>>;
    async fn get_blob(&self, path: &str) -> Result<Option<Vec<u8>>, Box<dyn std::error::Error + Send + Sync>>;
    async fn get_presigned_url(&self, path: &str, expiry_secs: u64) -> Result<String, Box<dyn std::error::Error + Send + Sync>>;
}

#[async_trait]
pub trait RelationalStore: Send + Sync {
    async fn execute_query(&self, sql: &str) -> Result<(), Box<dyn std::error::Error + Send + Sync>>;
    async fn query_json(&self, sql: &str) -> Result<serde_json::Value, Box<dyn std::error::Error + Send + Sync>>;
}

#[async_trait]
pub trait VectorStore: Send + Sync {
    async fn search_nearest(&self, embedding: &[f32], top_k: usize) -> Result<Vec<(String, f32)>, Box<dyn std::error::Error + Send + Sync>>;
    async fn upsert_vector(&self, id: &str, embedding: &[f32], metadata: &serde_json::Value) -> Result<(), Box<dyn std::error::Error + Send + Sync>>;
}
```

This guarantees 100% decoupling: the Rust backend can run against local SQLite / in-memory mocks during unit testing, against Cloudflare D1/R2/KV via HTTP REST bindings at the edge, or against Google Cloud Spanner / GCS in Kubernetes without touching application handler code.