Community database protocol: ogc-db-proto #43

Manually merged

SkyfaR merged 1 commit from db-proto into main 2026-10-07 07:31:32 +02:00

Conversation 0 Commits 1 Files changed 20 +2771 -9

SkyfaR commented 2026-10-06 23:35:10 +02:00

Owner

Copy link

First piece of the community database (CompressDB), per the design: the crate crates/ogc-db-proto, which the app, the intake service and the aggregator (repo LevelXStudios/ogc-db) will share. Nothing in the app uses it yet.

• Keys: store game keys (steam:, epic:, gog:, amazon:; numeric IDs without leading zeros, so one game has one key), builds, content platform, method keys as text (btrfs-zstd:N, wof:*; the kernel's own level maps to btrfs-zstd:3).
• Values: ratios in ‰ (20–1000), size bucket round(4·log2(MiB)) computed exactly in integers, percentages rounded to 5, the rules for poorly compressing extensions.
• Payload: the deny_unknown_fields allowlist, cross-field rules, a canonical writer (sorted keys, no whitespace, pad last, exactly 1024 bytes).
• Proof of work: stamp, weekly HMAC-derived salt, bounded solver, replay set keyed on the stamp, ISO weeks.
• Validator: size, schema, bounds, byte-for-byte canonical form, gen blocklist, injected app-ID set; each rejection is one code of the fixed enum.

Tests: golden bytes of both examples of the design, determinism across machines, proptest leak tests, every validator bound positive and negative, proof-of-work known-answer vectors checked independently in Python.

The root Cargo.toml becomes a workspace (the root package is unchanged). It excludes target/, because cargo refuses crates unpacked below a workspace (the Arch package's cargo package tree, accesskit-c in GTK's build). CI's check job formats, lints and tests the whole workspace; the Windows jobs are unchanged. ARCHITECTURE.md has a section on the crate.

First piece of the community database (CompressDB), per the design: the crate `crates/ogc-db-proto`, which the app, the intake service and the aggregator (repo `LevelXStudios/ogc-db`) will share. Nothing in the app uses it yet. - **Keys:** store game keys (`steam:`, `epic:`, `gog:`, `amazon:`; numeric IDs without leading zeros, so one game has one key), builds, content platform, method keys as text (`btrfs-zstd:N`, `wof:*`; the kernel's own level maps to `btrfs-zstd:3`). - **Values:** ratios in ‰ (20–1000), size bucket `round(4·log2(MiB))` computed exactly in integers, percentages rounded to 5, the rules for poorly compressing extensions. - **Payload:** the `deny_unknown_fields` allowlist, cross-field rules, a canonical writer (sorted keys, no whitespace, pad last, exactly 1024 bytes). - **Proof of work:** stamp, weekly HMAC-derived salt, bounded solver, replay set keyed on the stamp, ISO weeks. - **Validator:** size, schema, bounds, byte-for-byte canonical form, `gen` blocklist, injected app-ID set; each rejection is one code of the fixed enum. Tests: golden bytes of both examples of the design, determinism across machines, proptest leak tests, every validator bound positive and negative, proof-of-work known-answer vectors checked independently in Python. The root `Cargo.toml` becomes a workspace (the root package is unchanged). It excludes `target/`, because cargo refuses crates unpacked below a workspace (the Arch package's `cargo package` tree, accesskit-c in GTK's build). CI's check job formats, lints and tests the whole workspace; the Windows jobs are unchanged. ARCHITECTURE.md has a section on the crate.

SkyfaR added 1 commit 2026-10-06 23:35:10 +02:00

Add ogc-db-proto, the community database's contribution protocol ... c936287435

A workspace member with the payload of the community database: game and
method keys, coarse values, the canonical 1024-byte body, the proof of work
with its weekly salt, and the validator the intake service will share.
Nothing in the app uses it yet. The workspace excludes target/, where the
Arch package and GTK's build unpack crates of their own.

SkyfaR referenced this pull request 2026-10-07 04:01:50 +02:00

Community database in the app: shipped data, predictions, ogc db, opt-in download #47

SkyfaR referenced this pull request from a commit 2026-10-07 07:31:32 +02:00

Merge pull request 'Community database protocol: ogc-db-proto' (#43) from db-proto into main

SkyfaR manually merged commit 7e4eedbd4a into main 2026-10-07 07:31:32 +02:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

LevelXStudios/OpenGameCompressor!43

No description provided.