LevelXStudios/OpenGameCompressor
4
2
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions

Windows port, part 2: WOF engine, Windows platform and commands (W2) #31

Manually merged
SkyfaR merged 68 commits from windows-w2 into main 2026-10-05 16:05:00 +02:00
Conversation 0 Commits 68 Files changed 233 +16334 -1130
SkyfaR commented 2026-10-04 20:00:14 +02:00
Owner
Copy link

Windows port, phase W2. It builds on #29 (W0 probe, W1 groundwork) and adds the engine, the platform layer and the commands. Plan: windows-port-plan.md, section 10, W2 row.

Engine (src/backend/, src/compress/wof_*, src/estimate/wof*, src/skipcache.rs)

  • A backend interface with btrfs (unchanged, moved) and WOF. The codec is threaded through state, the timer, history and the CLI. Linux files stay byte-identical; Windows writes wof:xpress8k.
  • Volume admission. Only local NTFS with an answering WOF driver is admitted. Refused, with a reason: ReFS, FAT, exFAT, network drives, Windows' folders, drive and profile roots, Xbox folders and OneDrive. "Needs admin" is reported as a state, not as a refusal.
  • Per-file step:
    • Skip rules: reparse, encrypted, cloud and offline (never opened), WIM, LZNT1, sparse, hard links, read-only, in use.
    • A 16-chunk sample with a 3 % threshold, and a skip cache.
    • Every result is read back; error 344 means "not beneficial". No markers are written.
  • Switching. A file stored with another algorithm is switched only when it pays against its current allocation, when the user named the algorithm, or with --force. So CompactGUI's LZX files do not grow as XPRESS8K. A decision not to switch is remembered.
  • Room. 1 GiB once per run, plus each in-flight file's estimated need. A file that waits for room honours cancel, the stop and the pause. A file that does not fit by itself is left alone, and the run goes on.
  • Analysis gives all four algorithms from one read (XPRESS model exact, LZX ×1.13 with a 72 % floor, both from the real-PC runs). CompressSummary gains exact, guessed_files and guessed_bytes.

Platform (src/platform/windows/, src/launchers/, src/pause.rs)

  • Steam is found through the registry: SteamPath normalised, libraryfolders.vdf, manifests. The installdir check is stricter on both systems, as the plan says.
  • Own-folder refusals for Windows' system, profile, Program Files, Xbox, OneDrive and network places, compared in raw and canonical form.
  • Xbox and Game Pass games are listed as not compressible and cached per drive.
  • Running-game detection:
    • a handle-free image path, with Denied and Gone told apart by error code;
    • per-game program index and session filter for protected processes;
    • junction-moved games (Windows only);
    • Steam tools excluded by appid;
    • RunningAppID only chooses which folder is searched first, and never pauses on its own.
  • Power through GetSystemPowerStatus.
  • Console control (Ctrl-C, close, logoff, shutdown) with the two-stage semantics. check.sh allows only reviewed DLLs in ogc.exe, so user32 cannot slip in.
  • Saves by rename retry briefly on sharing violations on Windows (platform::replace).

Commands (src/commands/windows/, src/commands/wof*)

  • ogc analyze, compress, decompress and auto run on Windows. dedupe, migrate and timer are hidden there.
  • Compress and auto are generic over the engine and the ledger, and tested against a fake. The tests cover cancel, a full drive, DirectStorage, needs-admin, Steam busy, files in use or without room, and a named algorithm.
  • A game with files left for later is not recorded as done; an earlier record is marked outdated. The game is never recorded with an algorithm its files do not have.
  • Decompress refuses a drive before walking it, and takes one file at a time on a hard disk.

Packaging (scripts/windows/dist.sh)

  • A portable opengamecompressor-<version>-windows-x86_64.zip. libunwind is linked in, so it needs no extra DLL; an import allow-list enforces this.
  • The test package ogc-wintest-<version>.zip: real-WOF tests, run-tests.ps1 and a German guide. Checked under Wine and PowerShell 7.
  • Published for the next real-PC run as wintest-1 in the measurement repo, built from 04ffeb4.

Tests

  • Linux: fmt, both clippy runs, OGC_TEST_REQUIRE_BTRFS=1 cargo test, and the Broadway suite.
  • Windows: check.sh, and wine-test.sh with lib 336 and bin 72 passing plus the smoke run.
  • Each piece had a review with adversarial verification. The verified findings are fixed in this branch.

Not verified yet

  • Real WOF: the OGC_TEST_REQUIRE_NTFS end-to-end test, the room rules, and WOF's disk-full behaviour mid-file.
  • Real process detection against anti-cheat, RunningAppID (H18), and logoff/shutdown delivery.
  • PowerShell 5.1.
  • The translations are not checked by native speakers.

The friend's PC run with wintest-1 covers the first part of these. Its report decides what follows before merging.

Windows port, phase W2. It builds on #29 (W0 probe, W1 groundwork) and adds the engine, the platform layer and the commands. Plan: windows-port-plan.md, section 10, W2 row. ## Engine (`src/backend/`, `src/compress/wof_*`, `src/estimate/wof*`, `src/skipcache.rs`) - A backend interface with btrfs (unchanged, moved) and WOF. The codec is threaded through state, the timer, history and the CLI. Linux files stay byte-identical; Windows writes `wof:xpress8k`. - Volume admission. Only local NTFS with an answering WOF driver is admitted. Refused, with a reason: ReFS, FAT, exFAT, network drives, Windows' folders, drive and profile roots, Xbox folders and OneDrive. "Needs admin" is reported as a state, not as a refusal. - Per-file step: - Skip rules: reparse, encrypted, cloud and offline (never opened), WIM, LZNT1, sparse, hard links, read-only, in use. - A 16-chunk sample with a 3 % threshold, and a skip cache. - Every result is read back; error 344 means "not beneficial". No markers are written. - Switching. A file stored with another algorithm is switched only when it pays against its current allocation, when the user named the algorithm, or with `--force`. So CompactGUI's LZX files do not grow as XPRESS8K. A decision not to switch is remembered. - Room. 1 GiB once per run, plus each in-flight file's estimated need. A file that waits for room honours cancel, the stop and the pause. A file that does not fit by itself is left alone, and the run goes on. - Analysis gives all four algorithms from one read (XPRESS model exact, LZX ×1.13 with a 72 % floor, both from the real-PC runs). `CompressSummary` gains `exact`, `guessed_files` and `guessed_bytes`. ## Platform (`src/platform/windows/`, `src/launchers/`, `src/pause.rs`) - Steam is found through the registry: `SteamPath` normalised, `libraryfolders.vdf`, manifests. The `installdir` check is stricter on both systems, as the plan says. - Own-folder refusals for Windows' system, profile, Program Files, Xbox, OneDrive and network places, compared in raw and canonical form. - Xbox and Game Pass games are listed as not compressible and cached per drive. - Running-game detection: - a handle-free image path, with Denied and Gone told apart by error code; - per-game program index and session filter for protected processes; - junction-moved games (Windows only); - Steam tools excluded by appid; - `RunningAppID` only chooses which folder is searched first, and never pauses on its own. - Power through `GetSystemPowerStatus`. - Console control (Ctrl-C, close, logoff, shutdown) with the two-stage semantics. `check.sh` allows only reviewed DLLs in `ogc.exe`, so user32 cannot slip in. - Saves by rename retry briefly on sharing violations on Windows (`platform::replace`). ## Commands (`src/commands/windows/`, `src/commands/wof*`) - `ogc analyze`, `compress`, `decompress` and `auto` run on Windows. `dedupe`, `migrate` and `timer` are hidden there. - Compress and auto are generic over the engine and the ledger, and tested against a fake. The tests cover cancel, a full drive, DirectStorage, needs-admin, Steam busy, files in use or without room, and a named algorithm. - A game with files left for later is not recorded as done; an earlier record is marked outdated. The game is never recorded with an algorithm its files do not have. - Decompress refuses a drive before walking it, and takes one file at a time on a hard disk. ## Packaging (`scripts/windows/dist.sh`) - A portable `opengamecompressor-<version>-windows-x86_64.zip`. libunwind is linked in, so it needs no extra DLL; an import allow-list enforces this. - The test package `ogc-wintest-<version>.zip`: real-WOF tests, `run-tests.ps1` and a German guide. Checked under Wine and PowerShell 7. - Published for the next real-PC run as `wintest-1` in the measurement repo, built from 04ffeb4. ## Tests - Linux: fmt, both clippy runs, `OGC_TEST_REQUIRE_BTRFS=1 cargo test`, and the Broadway suite. - Windows: `check.sh`, and `wine-test.sh` with lib 336 and bin 72 passing plus the smoke run. - Each piece had a review with adversarial verification. The verified findings are fixed in this branch. ## Not verified yet - Real WOF: the `OGC_TEST_REQUIRE_NTFS` end-to-end test, the room rules, and WOF's disk-full behaviour mid-file. - Real process detection against anti-cheat, `RunningAppID` (H18), and logoff/shutdown delivery. - PowerShell 5.1. - The translations are not checked by native speakers. The friend's PC run with `wintest-1` covers the first part of these. Its report decides what follows before merging. 
The platform half of W2. Steam for Windows is found by the registry
(HKCU SteamPath, then the machine's InstallPath), written the Windows
way from the lowercase forward-slash form a real PC stores, and read
as on Linux from there; library paths are canonical without the \\?\
prefix. The installdir check refuses Windows' path forms on both
systems.

Own folders get Windows' refusal rules (system folders, Program Files
itself, profiles and the user's folders, the Xbox app's folders,
OneDrive, network drives, drive tops) as plain text rules tested
everywhere. Xbox app games are listed as warnings that they cannot
be compressed.

Windows gets running (Toolhelp snapshot, program paths, start times),
power (GetSystemPowerStatus), file identity and the scan's reparse
rules, and a console control handler with the two stages of the
signal handling. pause now builds on Windows; runstatus waits for the
engine's compress::Progress.

The Wine smoke lists a made-up Steam installation and Xbox game from
the test prefix's registry.
The engines now sit behind src/backend/: Backend::{Btrfs, Wof} names what each
writes (zstd 3 and XPRESS8K by default), its chunk size, whether it needs a
sync, and why a folder cannot be compressed (Backend::for_path, Refusal).
btrfs.rs moved to backend/btrfs.rs and compress.rs's per-file step to
compress/btrfs_file.rs, both unchanged; compress.rs keeps what every engine
reports and the parallel walk. pause::Pause, decompress's and analyze's reports
are shared files now.

The WOF engine is built and tested on every system through a Layer, which
backend/wof/sys/ implements with Windows' calls and backend/wof/fake.rs with a
stand-in:
- backend/wof.rs holds Microsoft's numbers alone: the algorithm numbers, the
  8-byte WOF_FILE_COMPRESSION_INFO_V1 for WofSetFileDataLocation (the FSCTL's
  20 bytes where wofutil.dll is missing), the read-back of
  FSCTL_GET_EXTERNAL_BACKING (provider at byte 4, algorithm at byte 12), the
  Win32 codes, and the skip rules.
- Volume admission: local NTFS with an answering WOF driver only; ReFS, FAT,
  exFAT, network drives, Windows' and the Xbox app's folders, drive and profile
  roots and OneDrive are refused with a reason. An effective-access check tells
  folders that need administrator rights.
- Per file (compress/wof_file.rs): too small, cloud and offline files are left
  alone without being opened; files are opened for reading, shared with readers
  only, so an open writer means in use; reparse points, encrypted, WIM-backed,
  LZNT1, sparse, hard-linked and read-only files are left alone; the skip cache
  and a 16-chunk sample decide incompressible files; room is kept as plan R5
  says; every call is read back and only the algorithm asked for counts; error
  344 is "not beneficial" and remembered. Sizes after a run are exact.
- The skip cache (skipcache.rs) is SQLite in the cache folder, keyed by volume
  serial, file ID, size, LastWriteTime and algorithm.
- Analysis (compress/wof_analyze.rs) reads each file's samples once for all four
  algorithms with Windows' XPRESS Huffman encoder; LZX is that model at 32 KiB
  with run 1's correction (saving x1.13, at least 72 % of the model).
- Decompress (compress/wof_undo.rs) deletes the external backing of each WOF
  file and reads it back.
On Windows, compress, decompress and analyze offer this engine with the API of
Linux' modules; fs_info reads the volume.

The codec is threaded through state::Entry, auto's TimerConfig, Status and
tasks, and history::Detail; games.tsv, prefixes.tsv, analysis.tsv, the history
and the timer's unit still write a zstd level bare, Windows writes
wof:xpress8k, and a line of a codec the system's engine does not write is left
out. --level takes either form and --algorithm a WOF algorithm (hidden on
Linux); ogc list names a WOF game's algorithm. Messages in all 13 languages.
# Conflicts:
#	Cargo.toml
#	docs/ARCHITECTURE.md
#	src/lib.rs
Game detection on Windows missed or misread several cases:

- A game moved to another drive and linked back into steamapps\common
  runs from where it lies; the watch now also looks in each Steam game's
  folder that common does not hold by its real path.
- A handle whose rights an anti-cheat driver stripped read as an exited
  process. Failures now map to Denied (ERROR_ACCESS_DENIED) or Gone in a
  pure, tested function, and the program comes first from
  NtQuerySystemInformation(SystemProcessIdInformation), which needs no
  handle; QueryFullProcessImageNameW is the fallback.
- Processes Windows tells nothing about count only in the user's
  session, and the programs are indexed per game folder without a cap,
  leaving Proton's prefixes out.
- The own-lineage walk stops at a parent younger than its child, so a
  reused PID cannot hide a game.
- Wallpaper Engine and SteamVR (pause::STEAM_TOOLS) are no games: their
  folders are left out by app ID and their programs by name.
- processes_using_folders applies the same rule to hidden processes,
  retries module snapshots on ERROR_BAD_LENGTH and skips them where the
  program already matches.
- Steam's RunningAppID is read as an extra signal: the named game's
  folder is looked in first, and work pauses while Steam names an app
  even when no process is found.
Places::of folded the environment, registry and home as written while
the folder asked about is canonical, so an 8.3 %TEMP% or a profile moved
with a junction slipped through. Each place is now kept both as written
and canonical (through an injected canonicalize), and a folder that
holds one of the user's own folders or OneDrive is refused as a user
folder.
Every load_all looked at .GamingRoot, XboxGames and
ModifiableWindowsApps on every fixed drive, waking external disks on each
app refresh and timer run. What a drive holds is now kept for the
process's lifetime and read again after launchers::rescan, which the
app's explicit reload calls. Warnings are still built per load, in the
current language.
The look at the processes is a field of Lookout, so the tests of
Steam's RunningAppID make it up instead of finding the sleepers other
tests run at the same time; a new test checks that the process of the
game Steam names is waited for.
State, exclusions, own folders and the history all save by renaming a
new file over the old one. On Windows an antivirus, the indexer or a
sync tool may hold either file briefly, and the rename fails with
ERROR_ACCESS_DENIED, ERROR_SHARING_VIOLATION or ERROR_LOCK_VIOLATION.
platform::replace now tries it again up to 10 times with a 20-200 ms
backoff there; Linux renames once, as before. The retry policy is a pure
function tested with an injected rename.
Compressing reserved max(1 GiB, 0.8 x file) for every file in flight, so
with eight threads small files needed about 8 GiB free, and a shortfall
caused only by other files in flight stopped the run as "drive full".

Both directions now share wof_room: SPARE_BYTES once, plus 0.8 x the file
(compressing) or what the file grows by (decompressing) for each file in
progress. A file that does not fit beside the files in progress waits
until one of them is done; only a file that does not fit by itself stops
the run. The fake WOF can delay its calls and counts how many ran at once,
and tests run four threads with room for two or three files.
Any WOF file stored with another algorithm was switched to the target,
judged only against its logical size. A CompactGUI game in LZX compressed
with the default XPRESS8K therefore grew, while the analysis, which
clamped each estimate to the current allocation, reported no change.

Now such a file is switched only when forced, or when its quick sample
estimates the target to take at least 3 % less than the file takes now
(its allocation). Otherwise it is left alone as SkipReason::OtherAlgorithm,
counted per algorithm in left_alone, and not remembered in the skip cache,
since the answer depends on how the file is stored. Without a model it
stays. The analysis applies the same rule and reports a switched file
with its estimate. New text in all 13 languages; WofAlgo derives Ord so
the reason can key the summary.
The summary is exact, yet files NTFS was never asked about (too small to
open, cloud and offline files, files a writer held, files not started and
failed files) count as their length rounded up to whole clusters. Such a
file is now a step of its own, and guessed_files and guessed_bytes say how
many and how much of both totals is guessed. The field docs say what is
exact: the saving, since a guessed file counts the same before and after.
An OGC_TEST_REQUIRE_NTFS-gated test runs compress_dir_with(RealWof) on a
temporary folder with a read-only file, a hard link, a sparse file and a
file a writer holds, checks left_alone, the guessed sizes, each file's
backing and stored_after against GetCompressedFileSizeW, the skip cache
with real file IDs, and then decompresses the folder and compares every
file's content. The real-WOF test of the calls now also asserts that NTFS
shows neither LZNT1 nor the compressed attribute on a WOF file, which the
skip rules rely on; the VM run settles that.

Fake tests cover a WOF file over LZNT1 (left alone, even forced; returned
from WOF by decompressing) and a link in a game folder, which never reaches
Layer::open. SkipReason::NameSurrogate is documented as defence in depth.
The architecture notes now say how room is kept once per run, when a file stored with another algorithm is switched, which sizes of a WOF summary are guessed, that links never reach the engine, and what the NTFS-gated tests check.
Clippy for Windows refuses fields assigned after Default::default().
The scan asked platform::file_id and platform::is_plain_file in turn,
which on Windows opened every file for its link count and a reparse
point a second time for its tag. One platform::plain_file_id now tells
both from one metadata-only handle (Linux: the regular file's device and
inode, as before). The decision is tested with a FnOnce open, so a
second open cannot happen, and a file whose data is elsewhere is not
opened at all.
Windows does not deliver CTRL_LOGOFF_EVENT and CTRL_SHUTDOWN_EVENT to a
console process that has loaded user32 or gdi32. console_control.rs
now names the constraint, and windows-check builds ogc.exe and fails
where its import table names user32.dll, gdi32.dll or shell32.dll.
runstatus needed only compress::Progress and running, which Windows has
now: its run.json lies in %LOCALAPPDATA%\OpenGameCompressor\run. The
test of a stale status file runs the test program itself as the child
(platform::testing::sleeper) instead of sleep, which Windows lacks.
The command line is a crate of its own, which the library's cfg(test)
items do not reach. The stand-in is public but hidden from the docs.
The commands share with Linux what both do alike: the work list, the
excluded games, the lock, the progress, the history, the record of
compressed games (now for any codec) and the summary, which on Windows
adds the exact sizes before and after and the files WOF left alone.
decompress is one module for both; its checks before rewriting a folder
come from preflight on Linux and from their Windows stand-ins.

What is Windows' own is in commands/wof.rs, built and tested everywhere
with the engine on the stand-in WOF: a folder Backend::for_path refuses
is told why, one that needs administrator rights is told so and never
elevated, Steam's work and programs in the folder hold it back as on
Linux. A game keeps its algorithm, else gets XPRESS8K; files at once
follow the drive (one on a hard disk) up to --jobs. Games that bring
DirectStorage are warned of, and auto leaves them alone. analyze shows
what each of the four algorithms would save, the default marked.

auto runs once from the command line with Linux' guards: the lock, a
running game and the battery. dedupe, migrate and timer stay unsupported
on Windows. Linux' output does not change.
Where Windows' engine means something else by a command or an option
(what ogc is, compress and decompress, --level, --force, --jobs,
--include-new), main.rs gives clap that system's English text and the
help looks up the key with -wof first. The btrfs-only options are hidden
there. Linux' help, man pages and completions are unchanged.
Against the made-up Steam installation, both must name WOF, exit with 1
and leave the game's file and games.tsv untouched. README and
ARCHITECTURE describe the Windows commands.
# Conflicts:
#	docs/ARCHITECTURE.md
run-tests.ps1 runs the real-WOF tests with OGC_TEST_REQUIRE_NTFS=1 in a
folder of their own on the Steam library's drive, then takes one Steam
game through ogc list, analyze, compress, Steam's verification and
decompress, and writes every line into a timestamped report without the
profile folder, user name or computer name. Its tests run with
PowerShell 7 in Microsoft's container (ps-test.sh), with a fake ogc and
test program, and reject syntax Windows PowerShell 5.1 lacks.
dist.sh builds the release ogc.exe and the library's test program for
x86_64-pc-windows-gnullvm with +crt-static, which links llvm-mingw's
libunwind in instead of importing libunwind.dll, fails when a program
imports a DLL Windows does not ship, and zips
opengamecompressor-<version>-windows-x86_64.zip (stripped ogc.exe,
licence) and ogc-wintest-<version>.zip, each with a .sha256.
dist/wine-check.sh unpacks both under Wine without WINEPATH, starts
ogc.exe on a made-up Steam installation and checks that run-tests.ps1's
filters pick exactly the real-WOF tests.
On Windows a reader holding the file for a moment made the rename fail,
and with it the app's request to stop a run. Linux renames once, as before.
The check denied user32, gdi32 and shell32 by name, but many other system
libraries load them too. ogc.exe may now import only the libraries it
imports today; any new one fails the check until it has been reviewed.
A profile moved with a junction to D:\me made D: a folder of profiles, so
every folder on D: was refused as one. The profile itself is still
refused.
Steam's word that it runs an app paused work for as long as the value
stayed set, with no process behind it and no limit; a value left over
held every run back. What the value holds is not measured yet (H18), so
it now only tells which game's folder is searched first, and only a
game's process pauses work. The Steam game text goes with it.
On Linux the watch passed these folders to the check of exe, cwd and
memory maps too, which changed how games are found there. Linux tells a
moved game by Steam's mark and looks in the libraries alone again, as
before; Windows, which knows a game by its program only, keeps them.
The other API sets (shell, user) may stand for libraries that load
user32. The architecture names the allowed imports and that run.json and
the stop request are saved through platform::replace.
The drives' devices covered volumes with a letter only, so a library on
a volume mounted on a folder had no Win32 path from the system's list of
processes. Such a volume now stands for the first folder it is mounted
on.
ProcessIdToSessionId opens the process, which an anti-cheat driver that
hides a game's program refuses too, so the game it was meant for was
never counted. The sessions now come from the system's list of
processes; one Windows does not tell counts as the user's, and the
user's sessions are the console's and this process's unless that is
session 0.
each_file_until hands the work a Halt that tells it to stop waiting
(the run cancelled or stopped, or a game running) and counts a file it
gave up as not started. A file given up for a game is tried again once
the game has exited. each_file is the same walk for work that never
gives up, so the btrfs engine and Linux decompression are unchanged.
The writer was dropped after the first run, so the second run on a
real NTFS volume compressed held.pak and the end-to-end test failed.
It now stays open until the second run is done, which must leave it
alone as in use again, and is dropped before decompressing. The
read-only file's permission is restored by a drop guard, so a failed
assertion still leaves a temporary folder that can be removed.
A file waiting for the room the files in progress keep now waits with
its handle closed, so Steam and the game may write to it, and asks the
run's Halt before each wait and after it: cancelled, stopped or paused,
it gives up and is not started (and once a game has exited, it is tried
again). WOF was called for such files once the files in progress were
done, even after Cancel or while a game ran.

A file that does not fit by itself beside the spare 1 GiB is left alone
(SkipReason::NoRoom, a new string in all 13 languages) and the run goes
on; in decompression it fails alone. Only a volume without the spare
1 GiB, or WOF's ERROR_DISK_FULL, stops the run as full now. One big
file stopped every other file and folder on the drive before.

The room a file needs is its quick sample's estimate and an eighth of
its length (H12: WOF needs about the compressed size), at most four
fifths of its length, which stays the need without a sample.
Every compress run sampled every file stored with another algorithm
(CompactGUI's LZX: 58,033 files on the friend's PC) through WOF only to
leave it alone again. A file the quick sample leaves as it is is now
remembered in the skip cache's new not_switched table, keyed like the
skip cache by volume serial, file ID, size, LastWriteTime and the
algorithm asked for, and also by the algorithm it is stored with, as
switching keeps LastWriteTime. Stored otherwise since, it is sampled
again; without an estimate nothing is remembered. The fake WOF notes
its reads, and a test shows a second run does not read the LZX file.
WofOptions::switch_algorithm switches a file stored with another WOF algorithm without the test whether that saves space; plain files keep their test. ogc compress sets it when --algorithm or --level names the algorithm, so that an LZX game compressed with --algorithm xpress4k gets xpress4k instead of being left LZX and recorded as xpress4k. Without a name, and in ogc auto, the engine's rule stays.
What ogc compress and ogc auto do with a folder on Windows, from the checks to the record, is now windows/folder.rs, generic over the engine (Wof) and over what a run reads and writes beside it (windows/ledger.rs: the launcher's view of a game, the exclusions, the history, games.tsv). It is built for the tests on every system and runs there against the stand-in WOF and a ledger in memory: a cancelled run and a full drive record nothing, the rest of a full drive is left alone, auto leaves DirectStorage games, folders that need administrator rights and games Steam works on for later, and a compressed game is recorded with wof:xpress8k together with the apps sharing its folder.

A game is recorded with what its files are stored with: an LZX game that a run with XPRESS8K left as it was stays recorded as LZX, and the run says so. A game with files left in use is not recorded, so that a later run comes back to them; the run says the files are left for the next one.
wof::check refused a drive without WOF, FAT, exFAT, a network drive or Windows' own folders only for compressing, while the engine refuses them for decompressing too. ogc decompress therefore opened every file of such a folder to measure the room it needed, and noted a failure in the history for each game there. It is now a skipped line before anything is walked.

Decompressing takes as many files at once as the drive takes (commands::checks::jobs: --jobs on Linux, one on a hard disk on Windows), as compressing does.
On Windows the help no longer offers what works only on Linux: dedupe, migrate and timer are hidden (they still parse and say so), Getting started names no timer, and list, the games, --all and --dir name Steam and the own folders instead of Heroic, Lutris, Faugus and Bottles. --level/--algorithm say that files stored with another algorithm are switched to the one named, and --force that it switches them whatever that saves. decompress says that auto-recompress leaves the game alone, and the text for a folder that needs administrator rights fits decompressing too. All in 13 languages; Linux' help is unchanged.
# Conflicts:
#	src/compress/wof_file.rs
room_needed capped the room a file keeps at four fifths of its length, so a
large file whose sample saved only a few percent could take the drive past
the spare room while WOF wrote it. With a usable sample the need is now the
estimate and an eighth of the length, at most the length; without a sample
four fifths of a file below 1 MiB and all of a larger one.
The engine's tests never set switch_algorithm, and the commands' tests run
the engine without a cache, so a resolution that consulted the not_switched
cache before the switch guard would have passed every test.
State::record_unfinished keeps an existing entry with the given level and
no build, so that auto::outdated sees it as outdated and the automatic run
takes the game up again. A game without a build of its own loses its entry,
and a game without an entry stays without. The games.tsv format is the same,
and nothing on Linux calls it.
Leave games with files in use or without room for a later run on Windows
All checks were successful
CI / Windows, command line (clippy with tests) (pull_request) Successful in 40s
Details
CI / Windows, command line (tests under Wine) (pull_request) Successful in 1m29s
Details
CI / Format, lint and test (pull_request) Successful in 5m33s
Details
CI / Bundle programs, command line and helper (pull_request) Successful in 2m13s
Details
CI / Bundle programs, desktop app (pull_request) Successful in 2m57s
Details
CI / Arch package (pull_request) Successful in 3m5s
Details
CI / Bundle for any distribution (pull_request) Successful in 40s
Details
VM test / Migrations in a VM (pull_request) Successful in 17m53s
Details
04ffeb49e9
 
A file too large for the room left on the drive was left alone as NoRoom,
which the commands did not count as unfinished, so a game whose large files
stayed plain on a nearly full drive was recorded as compressed with the
run's algorithm. compress and auto now treat NoRoom like files in use: the
game is not recorded as done, compress ends with 1 and auto's run is
incomplete, and the history notes the folder as failed with the reason.

An earlier record no longer survives such a run unchanged: it is rewritten
with what the files are stored with and no build, so that ogc auto comes
back to the game instead of keeping a record that names an algorithm most
of its files may lack. The message says truthfully what takes the files up,
in all 13 languages.
The real-WOF folder test read every file back through the engine's own
handle, which shares reading only. Windows refuses it with error 32 while
the test's writer still holds held.pak, so the test failed on the first
real PC (wintest-1). The read-back now opens each file with Rust's default
sharing; every assertion stays.
run-tests.ps1 passed --algorithm=xpress8k on every run, which also switches
files stored with another algorithm: on the first real PC it turned a
game's CompactGUI LZX into XPRESS8K, taking more room. The option now goes
to ogc only when -Verfahren names a method.

After the first ogc analyze, a game with more than 5 % stored compressed
already gets a warning: the test would mostly measure the old compression,
and the final decompress undoes it too. Enter picks another game, w goes on.

The report shows control characters as text ([U+0000 x938]): the raw NULs
of a zero-filled manifest quoted by ogc list made the whole report binary
to grep and similar tools, which then showed nothing of it. An empty
stderr line is recorded empty instead of as RemoteException.
ANLEITUNG.txt says why a game compressed with CompactGUI or compact is a
poor test, how to see it in Explorer, and what the script's new warning
means. The architecture notes describe the default method, the warning
and the visible control characters of the report.
wintest-1: run-tests.ps1 named --algorithm=xpress8k, which switched every
LZX file CompactGUI had made in Age of Empires II, and the game grew from
3.5 to 3.9 GiB. A named algorithm now switches a file stored with another
one only where its quick sample estimates it to take no more than now;
--force still switches all of them, and without a name the 3 % rule
stays. The skip cache keeps the files not switched by each rule in a
table of its own, so that what one rule found never answers for the
other; the existing not_switched table keeps its rows for the 3 % rule.
The summary line for files left stored with another WOF algorithm now
names that algorithm, the one asked for, and that "ogc compress --force"
switches them anyway. A run whose exact sizes end above where they began
says on a line of its own how much more the files take, next to the
free space it used. The btrfs engine reports neither, so Linux prints
as before.
The WOF analysis counts the size of the files stored with each
algorithm, and analyze names under its table every game holding files
stored with another algorithm than the one analysed: wintest-1's Age of
Empires II, 77 % LZX from CompactGUI, showed 28.6 MiB to save with
XPRESS8K and nothing to say why so little.
A damaged app manifest of zero bytes made the VDF parser report a
missing value for a key of about a thousand NUL characters, which
ogc list printed raw in its table and escaped in its JSON. Keys and
values that parse errors quote (the VDF key, installdir, depot and
manifest ids) now have control and direction characters escaped and
are cut after 40 characters.
On a real PC an app manifest held only zero bytes, as a crash or a
full drive can leave it. Steam's text files that are empty or hold
only zero bytes are now reported as damaged, in the user's language,
before they are parsed. Reading them moves into library/read.rs.
The first line of compress and decompress read "0.0%  0 B / 0 B
(0 / 0 files)" while the folder was still scanned, on Linux as on
Windows. The thread that follows the work now shows nothing until
the scan has set the total of files or bytes.
The doc comments of --level and --force on Windows are what English help shows (help::tests::english_help_is_the_doc_comments, run under Wine); they now say what cli-help.ftl says.
Keep a 1 % margin when switching to a named WOF algorithm, and name it in the hint
Some checks failed
CI / Windows, command line (clippy with tests) (pull_request) Successful in 39s
Details
CI / Windows, command line (tests under Wine) (pull_request) Successful in 1m26s
Details
CI / Format, lint and test (pull_request) Failing after 2m47s
Details
CI / Arch package (pull_request) Has been skipped
Details
CI / Bundle programs, command line and helper (pull_request) Has been skipped
Details
CI / Bundle programs, desktop app (pull_request) Has been skipped
Details
CI / Bundle for any distribution (pull_request) Has been skipped
Details
VM test / Migrations in a VM (pull_request) Successful in 17m28s
Details
d99722f719
 
A named algorithm switched files whose sample estimated them at exactly
their current size, so a slightly low estimate could grow them. The hint
for files left with another algorithm now gives the full command with
--algorithm, since a plain --force keeps a game's recorded algorithm.
The Windows test report notes the share found compressed, the docs say
that ogc list goes by ogc's records only, and the PowerShell test of an
empty stderr line no longer depends on how stdout and stderr interleave.
Size the room of the waiting test by the estimate the engine keeps room for
Some checks failed
CI / Format, lint and test (pull_request) Failing after 2m47s
Details
CI / Windows, command line (clippy with tests) (pull_request) Successful in 40s
Details
CI / Windows, command line (tests under Wine) (pull_request) Successful in 1m30s
Details
CI / Arch package (pull_request) Has been skipped
Details
CI / Bundle programs, command line and helper (pull_request) Has been skipped
Details
CI / Bundle programs, desktop app (pull_request) Has been skipped
Details
CI / Bundle for any distribution (pull_request) Has been skipped
Details
VM test / Migrations in a VM (pull_request) Successful in 17m38s
Details
64dcaec2ed
 
Since the room a file keeps follows its sample's estimate, five MiB
beside the spare held six of the test's text files, not three, and CI
saw four at once. The test now gives room for three by the estimate.
Wait until the test's child runs sleep before taking its start
All checks were successful
CI / Windows, command line (clippy with tests) (pull_request) Successful in 39s
Details
CI / Windows, command line (tests under Wine) (pull_request) Successful in 1m26s
Details
CI / Format, lint and test (pull_request) Successful in 5m30s
Details
CI / Bundle programs, command line and helper (pull_request) Successful in 2m13s
Details
CI / Arch package (pull_request) Successful in 3m6s
Details
CI / Bundle programs, desktop app (pull_request) Successful in 3m8s
Details
CI / Bundle for any distribution (pull_request) Successful in 40s
Details
VM test / Migrations in a VM (pull_request) Successful in 17m52s
Details
296baad1e0
 
Right after the fork the child still has the test program's name, so a
Running taken then no longer matched once the child had exec'd sleep,
and the test failed now and then in CI.
SkyfaR manually merged commit 7bc36dfc20 into main 2026-10-05 16:05:00 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
LevelXStudios/OpenGameCompressor!31
No description provided.