The registry

The registry is a static file tree in Git. GitHub is the source of truth; any static host can serve it.

registry/
  publishers/<id>.json                  who may publish, and their public keys
  entries/<pub>/<name>/<version>.json   one immutable record per published version
  status/<pub>/<name>.json              deprecations, yanks, revocations (mutable)
  v1/catalog.json                       signed catalogue (generated)
  v1/packages/<pub>/<name>.json         one document per capability (generated)
  v1/blobs/<pub>/<name>/*.adcap         small-package mirror (generated)

entries/ and publishers/ are hand-reviewed inputs. v1/ is generated by arcotheon registry build and signed with the registry key.

Reading it efficiently#

Clients do not clone anything. They fetch:

1. catalog.json: one small signed file with a summary of every capability, enough for search and for deciding what to fetch next.
2. packages/<pub>/<name>.json: only for capabilities they care about, checked against the SHA-256 in the catalogue.
3. The archive, from the entry's download list, checked against the entry's SHA-256 and the publisher's signature.

Responses are cacheable: the catalogue changes only when something is published, and blobs are immutable.

Where archives live#

Each version entry lists download URLs in preference order:

Consumers never need the private repository: the catalogue, documents and small archives are served from agdc.dev.

Large resources and storage#

Datasets, indexes and weights are not archives: capabilities reference them by HTTPS URL and SHA-256. The client downloads them in ranged chunks (default 8 MiB) into a .part file, resumes from whatever is already there (also after a crash), fails over between mirrors, re-requests an entitlement grant if it expires mid-download, verifies the whole file's hash before it becomes visible, and caches it by content hash. arcotheon cache verify re-hashes the cache.

For hosting, ObjectStore is the abstraction (blobs/sha256/aa/bb/<sha> keys): a local FileStore, and S3Store, an S3-compatible client with AWS Signature V4 that works with Cloudflare R2. Its SigV4 code matches AWS's published test vector and is tested against an independent server-side verifier. arcotheon storage push <file> --endpoint --bucket uploads a blob (resumable multipart above 64 MiB, signed per part), confirms it with HEAD, optionally re-downloads and re-hashes it (--verify), and prints the resource entry to paste into a manifest. Credentials come from the environment only.

R2 is not operational. On the Sarja Cloudflare account the API answers "Please enable R2 through the Cloudflare Dashboard" (error 10042); enabling it is an account action that needs a plan with billing, so nothing was enabled and no bucket exists. Until then the sample dataset is served as static files from agdc.dev. The S3 client was verified against a local S3-compatible test server, not against a real R2 bucket. Multipart uploads resume after an interruption without re-sending finished parts.

Private and entitled distribution#

Catalogue metadata stays public; the bytes of an entitled package or resource need an authorisation. An entry has access: "entitled" and an entitlement key. The client takes a DownloadAuthorizer; gatewayAuthorizer implements it against an entitlement gateway. The gateway (services/entitlement-gateway, reference code) issues Ed25519-signed grants that live at most ten minutes, are bound to one content hash, support ranged reads, and can be revoked by id or by subject. The required entitlement comes from the server's records, never from the request. See AgentDock integration.

What does not exist: a Sarja entitlement provider. The gateway's Worker adapter ships a deny-everyone provider; the real one must be written against Sarja's account system. The gateway is not deployed.

Immutability, yanking, revocation, deprecation#

• A published (id, version) never changes. The build refuses to rewrite an entry with different content.
• registry yank <id> <version> --reason …: stop choosing it automatically; it can still be pinned exactly.
• registry revoke <id> <version> --reason …: never install it, and refuse to run it where it is installed.
• registry deprecate <id> --message … [--replaced-by <id>]: show a warning everywhere.
• Publisher keys can be marked retired or revoked in publishers/<id>.json; verification fails for packages signed by a revoked key.

These live in status/ files and reach clients with the next catalogue.

Prereleases#

A version with a prerelease suffix is a prerelease. The catalogue reports latest (stable) and, if newer, latestPrerelease. Clients ignore prereleases unless asked (--pre) or given an exact version.

Verifying the registry#

arcotheon registry verify --root registry

checks the catalogue signature, that every document matches its signed hash, and that every mirrored archive matches its entry, parses, and carries a valid signature from an active key of its publisher. Run it before every merge.

Multiple registries#

A client accepts a list of registries and searches them in order, so a company can run a private registry (same format, its own root key) next to the public one.

Keys, rotation and expiry#

• registry/root-keys.json lists the registry root keys (public). Each catalogue carries the current set in rootKeys.
• Rotation: add the new key (registry root add), build with --key <old> --also-sign <new>; clients that trust the old key adopt the new one. Then revoke the old key (registry root revoke) and sign with the new key only. Clients that adopted the new key keep working; a client that never saw it needs an SDK with the new pin.
• Publisher keys: registry publisher add-key | retire-key | revoke-key <id> …. A retired key still verifies old packages but cannot publish; a revoked key invalidates what it signed. registry publisher suspend <id> --reason … blocks every version of a publisher. registry publisher verify-identity records who verified a publisher's identity (required before trusted).
• Expiry: each catalogue expires after 60 days by default (--ttl-days). Re-sign with registry build --refresh before then; an online client refuses an expired catalogue.
• Signing without holding the private key: --signer-cmd "<command>" --key-id <fingerprint>.

Limits of version 0.1#

• No production root key exists yet (see security); the sample registry uses a development key.
• A client must be updated if the root is rotated away from every key it knows.