Zion Boggan
repos/Oversight/docs/REGISTRY_DEPLOYMENT.md
zionboggan.com ↗
176 lines · markdown
History for this file →
1
# Registry Deployment
2
 
3
This is the public-safe live configuration for the reference Oversight
4
registry. It keeps secrets in `.env`, keeps the registry process off the public
5
host interface, and exposes TLS through Caddy. The Python FastAPI registry and
6
the Rust Axum registry both honor the write-side operator token described here.
7
 
8
## Layout
9
 
10
- `Dockerfile` builds the Python reference registry image.
11
- `docker-compose.yml` runs the registry on loopback and adds a `live` profile
12
  for Caddy.
13
- `Caddyfile` routes the registry, beacon, OCSP-style, and license-style
14
  hostnames to the registry container.
15
- `.env.example` lists every live setting without carrying secret values.
16
 
17
## First Run
18
 
19
```bash
20
cp .env.example .env
21
# Fill OVERSIGHT_DNS_EVENT_SECRET and OVERSIGHT_OPERATOR_TOKEN in .env.
22
# Use high-entropy random values; never commit .env.
23
 
24
docker compose up -d oversight-registry
25
curl http://127.0.0.1:8765/health
26
```
27
 
28
For public TLS, point DNS for the four configured hostnames at the host, then
29
start the live profile:
30
 
31
```bash
32
docker compose --profile live up -d
33
```
34
 
35
## Public Routes
36
 
37
`OVERSIGHT_REGISTRY_DOMAIN` serves the registry metadata, evidence, tlog, and
38
operator routes:
39
 
40
- `GET /health`
41
- `GET /.well-known/oversight-registry`
42
- `GET /evidence/{file_id}`
43
- `GET /tlog/head`
44
- `GET /tlog/proof/{index}`
45
- `GET /tlog/range`
46
- `GET /candidates/semantic`
47
- `POST /register`
48
- `POST /attribute`
49
- `POST /dns_event`
50
 
51
The beacon hostnames route only their beacon families:
52
 
53
- `OVERSIGHT_BEACON_DOMAIN`: `/p/{token}.png`
54
- `OVERSIGHT_OCSP_DOMAIN`: `/r/{token}` and `/ocsp/r/{token}`
55
- `OVERSIGHT_LICENSE_DOMAIN`: `/v/{token}` and `/lic/v/{token}`
56
 
57
Everything else returns `404`.
58
 
59
## Operator Authentication
60
 
61
If `OVERSIGHT_OPERATOR_TOKEN` is set, `POST /register` and `POST /attribute`
62
require either:
63
 
64
```http
65
Authorization: Bearer <token>
66
```
67
 
68
or:
69
 
70
```http
71
X-Oversight-Operator-Token: <token>
72
```
73
 
74
Leaving `OVERSIGHT_OPERATOR_TOKEN` empty keeps the v1 conformance harness and
75
local development behavior unchanged. Do not leave it empty on a public
76
operator deployment. Both reference registry implementations use the same
77
token contract, so live conformance commands work against either backend.
78
 
79
DNS bridge callbacks are separate. Set `OVERSIGHT_DNS_EVENT_SECRET`; the DNS
80
bridge must send either `Authorization: Bearer <secret>` or
81
`X-Oversight-DNS-Secret: <secret>` when posting `/dns_event`.
82
 
83
## Conformance
84
 
85
Local reference check:
86
 
87
```bash
88
python tests/test_registry_conformance.py
89
```
90
 
91
Live check against a token-protected registry:
92
 
93
```bash
94
OVERSIGHT_REGISTRY_URL=https://registry.example.org \
95
OVERSIGHT_OPERATOR_TOKEN=<token> \
96
python tests/test_registry_conformance.py
97
```
98
 
99
The token is read from the environment and sent as a bearer header. Do not put
100
real token values in shell history on shared machines.
101
 
102
## Migrating Python Registry Data To Rust
103
 
104
The Rust Axum registry can import the Python reference registry's SQLite rows
105
without mutating the source database. Run a dry run first:
106
 
107
```bash
108
oversight-registry \
109
  --db /var/lib/oversight/rust-registry.sqlite \
110
  --migrate-from /var/lib/oversight/python-registry.sqlite \
111
  --migrate-dry-run
112
```
113
 
114
The command prints JSON row counts for `manifests`, `beacons`, `watermarks`,
115
`events`, and `corpus`. If the counts are expected, run the same command
116
without `--migrate-dry-run`:
117
 
118
```bash
119
oversight-registry \
120
  --db /var/lib/oversight/rust-registry.sqlite \
121
  --migrate-from /var/lib/oversight/python-registry.sqlite
122
```
123
 
124
The migration copies into the Rust target database after running its schema
125
migrations. It preserves `events.id`, `events.tlog_index`, corpus `metadata`,
126
and the manifest/beacon/watermark relationships that evidence bundles depend
127
on. Validate the copied database before switching traffic:
128
 
129
```bash
130
oversight-registry \
131
  --db /var/lib/oversight/rust-registry.sqlite \
132
  --validate-db
133
```
134
 
135
The validation command prints JSON counts plus integrity failures for orphaned
136
beacons, watermarks, events, corpus rows, identity mismatches, malformed
137
event `extra` JSON, malformed corpus metadata JSON, duplicate or negative
138
tlog indexes, missing event tlog indexes, event tlog indexes outside the
139
on-disk tlog size, event rows whose indexed tlog leaf carries unrelated
140
evidence, malformed or non-contiguous local tlog leaf records, tlog leaf hash
141
mismatches, malformed manifest JSON, invalid manifest signatures, and
142
manifest/file ID divergence. Keep the Python database as a rollback artifact
143
until validation, live conformance, and evidence-bundle checks pass against
144
the Rust service.
145
The Rust `/tlog/range` route also reads through validated tlog records, so
146
malformed or hash-mismatched local leaf data blocks the range response instead
147
of disappearing from monitor output.
148
The Python reference registry uses the same fail-closed local tlog validation
149
for startup recovery and `/tlog/range`; newly appended records include
150
`leaf_data_hex` so exact event bytes can be recomputed by monitors.
151
Both reference registries return the registry v1 error envelope
152
`{"error":{"code":"...","message":"..."}}` for registry failures, and the
153
live conformance harness checks representative envelope codes.
154
`docs/REGISTRY_V1_STABILITY.md` defines the candidate-frozen v1.0 route,
155
field, and error-envelope surface that operators should burn in against.
156
 
157
## Rust Registry Burn-In Checklist
158
 
159
Run this checklist before switching production traffic from the Python
160
reference registry to the Rust Axum registry:
161
 
162
1. Take a cold copy of the Python SQLite database and keep the original
163
   mounted read-only during migration testing.
164
2. Run `--migrate-dry-run` and compare all row counts against the source
165
   database.
166
3. Run the real `--migrate-from` into a fresh Rust database.
167
4. Run `--validate-db` and treat any nonzero field as a deployment blocker.
168
5. Start the Rust registry on loopback with `OVERSIGHT_OPERATOR_TOKEN` and
169
   `OVERSIGHT_DNS_EVENT_SECRET` set.
170
6. Run the live registry v1 conformance harness against the Rust endpoint.
171
7. Fetch `/.well-known/oversight-registry`, `/tlog/head`, `/tlog/range`,
172
   and at least one `/evidence/{file_id}` bundle, then verify the evidence
173
   bundle with an independent client.
174
8. Compare the deployed surface with `docs/REGISTRY_V1_STABILITY.md`.
175
9. Keep the Python database and tlog as rollback artifacts until the Rust
176
   service has completed the operator's burn-in window.