Integrity and safe restores

SQZ is built so that a restore either gives you back exactly what you stored, or stops and tells you why. It never overwrites your files and never leaves half-written output.

Every layer is checked#

sqz t runs all of these without writing anything.

Nothing is overwritten#

• sqz c writes the new archive under a temporary name and publishes it only when it is complete. It never replaces an existing file.
• Extraction writes each file under a temporary name and gives it its real name only after its check passes. Existing files are never overwritten.
• sqz zip reads the ZIP back and checks every entry before keeping it, and never replaces an existing file.

Publishing under the real name uses a hard link, which is why the destination drive must support hard links. On one that does not, extraction fails safely instead of falling back to something weaker.

Checks before writing#

Extraction looks at the destination before writing anything: existing entries, names that would merge on a drive that ignores letter case, names Windows cannot create, and free space. If any would be a problem, it stops with one error naming up to five paths, and writes nothing. See the full list.

Safe with hostile archives#

• Symlinks are restored last and never followed. A symlink whose target is absolute, or would climb out of the output folder, is skipped.
• Every folder between the output folder and an entry must be a real folder, not a symlink, when the entry is created.
• setuid, setgid and sticky bits are never granted.
• SQZ never reserves memory based on a size an archive claims before checking it, so a few crafted bytes cannot force huge allocations.
• Transform outputs are capped while they are written, and their declared sizes are validated.
• Damaged JPEG, PNG, ZIP, gzip and PDF entries are refused cleanly: the JPEG decoder checks the lengths in its header before reserving memory (a crafted entry could once ask for about 4 GB), and the decoder that rebuilds deflate streams refuses damaged block counts and "last block" flags instead of running out of memory or looping. These cases were found by fuzzing and are kept as regression tests. Good entries restore byte for byte as before.

How this is tested#

• Crash tests: an append torn at every byte, or zero-filled, always leaves the previous snapshot readable, and the next append recovers. A damaged block stops extraction without leaving partial or temporary files.
• Fuzzing: six fuzz targets (whole archives, the index, block codecs, the SQCM decoder, transforms and chunk layouts) run with AddressSanitizer on both the Rust and the C++ code, for a minute per target on every change and 20 minutes weekly.
• An independent reader: a separate Python reader written only from the format specification is checked against every frozen test archive and every single-byte corruption of them.
• Round trips: every benchmark run restores its output and compares it byte for byte; any mismatch fails the run.
• Backups: fault injection flips, truncates, removes and swaps every kind of object, and interrupts backups, prune and copy at each stage.