Storage, backup and cleanup API
scripts/storage_ops.py provides bounded Python 3.12+ standard-library operations. scripts/patchcairn_storage.py provides human-readable capabilities and selected-root metadata assessment. The CLI has no mutation command. Run with an existing Python, for example python -I -S -B scripts/patchcairn_storage.py capabilities.
The trusted coordinator owns actual current-human authorization, session state, independent reviews and the workflow engine's exact action/plan/scope gate. These low-level methods have the caller's OS privileges. Neither a digest nor an imported approval field nor a standalone quotation authenticates consent. They are not an OS sandbox. Keep the selected scope quiet and exclusively controlled.
On this pageOn this page
Exact Python interface
Import StorageService, StorageRefusal, plan_sha256, render_storage and storage_capabilities from storage_ops. The sibling scoped_fs module provides checked native filesystem primitives; there is no weaker portable fallback. Every operation below uses keyword-only arguments.
StorageService(*, output_root, quiet_workspace, protected_roots=None) requires quiet_workspace=True and an existing, positively owned and owner-private directory strictly below the executing toolkit's script-derived work/. The coordinator establishes that ownership and creates a dedicated namespace with private_workspace.ensure_private_directory(..., create=True). Existing broad directories are refused; this service never changes their ownership, mode or ACL. The helper does not create or discover a new external destination. All generated plans, metadata, archives, manifests, reports and receipts belong in private work artifacts. Roots are explicit absolute directory paths, limited to 32, with no overlapping, aliased or whole-volume roots.
| Method | Result and important inputs |
|---|---|
assess_storage(*, roots, limits=None) | Metadata inventory with exact root identities, stable entry records, deterministic tree/extension/size buckets, visible protected exclusions and logical sizes. No ordinary file content reads. |
find_duplicates(*, inventory, content_read_authorized, limits=None) | Requires an explicit additional content-read decision. Returns stable identity/size/SHA-256 groups and a logical-byte opportunity; reclaimable_bytes stays unknown and removal_authorized stays false. |
plan_backup(*, inventory, selected, destination, content_read_authorized, required_recovery=True, exclusions=None) | Exact selected file keys, bounded identity/content hashes, visible unbacked/protected entries and destination. selected is a nonempty unique list such as r001/documents/example.txt; optional exclusions maps exact skipped keys to bounded reasons. |
check_preconditions(*, plan, expected_plan_sha256) | Read-only native freshness snapshot for the fixed plan kind, source contents, exact outputs and required recovery before begin_operation. Does not consume the plan or authorize mutation. |
create_backup(*, plan, expected_plan_sha256) | Exclusive immutable ZIP, external immutable manifest and durable attempt/step/outcome evidence. Returns exact artifact paths, native records and hashes. |
verify_backup(*, receipt) | Reopens exact archive/manifest records, verifies their hashes, exact entry list, regular-file type, CRC, member SHA-256, expanded-byte caps and safe destination mapping. Returns verified manifest and bounded in-memory archive bytes for internal use. Do not put returned archive bytes in a chat/report. |
verify_outcome(*, plan, expected_plan_sha256, receipt) | Read-only actual postconditions for all five fixed recipes. Requires byte-exact persisted outcome and rechecks archive, restored content, quarantine membership/source absence, restored originals or exact purged absence. Returns typed verification facts and exact outcome artifact record/hash. A missing outcome stays held/unknown; no receipt reconstruction or retry occurs. |
plan_restore(*, receipt, destination) | Verifies backup and plans a new owned-work destination outside all original source roots. Existing destinations cause a hold. No original-file overwrite mode exists. |
restore_backup(*, plan, expected_plan_sha256) | Creates that exact new destination, restores under root IDs, verifies file bytes from disk, and records immutable progress/outcome receipts. Explicit skipped files remain visible. |
plan_cleanup(*, inventory, selected, quarantine_destination, content_read_authorized, recovery_required=True, recovery_backup=None, recovery_verification=None, protected=None) | Exact selected inventory/identity/hash/scope. Required recovery defaults to true and requires a verified backup covering every target plus a completed restoration-probe receipt. The immutable restoration outcome and every restored probe file are re-read and bound to the exact archive and manifest hashes before quarantine/purge. protected is an exact list of protected inventory keys. This service's output artifacts remain ineligible host-cleanup targets. |
quarantine(*, plan, expected_plan_sha256) | Requires checked native same-volume, no-overwrite moves. Uses numbered quarantine names, durable immutable progress and no copy/delete fallback. It normally frees no disk space. |
plan_quarantine_restore(*, receipt) | Checks exact quarantine inventory and content, including unknown-file holds; plans restoration to the original selected roots. |
restore_quarantine(*, plan, expected_plan_sha256) | Checked same-volume no-overwrite moves to exact original paths and original root identities. Existing files cause a hold. |
plan_purge(*, receipt) | Creates a distinct irreversible plan after checking exact quarantine inventory/content and required recovery. It explicitly requires separate owner approval and loss acknowledgment through the coordinator's workflow. |
purge_quarantine(*, plan, expected_plan_sha256) | Removes only exact individually hashed quarantine files using the checked native primitive. It has no approved=True or quotation shortcut. The coordinator must first obtain the separate actual irreversible approval and pass the engine gate. Empty quarantine directories and audit receipts remain. |
Every mutating plan is bound using plan_sha256(plan) and its exact kind. That hash detects changed plans and does not prove authorization. Full selected inventory freshness is checked before backup/quarantine, including unknown added entries. Source identity and content are checked again at the native read/move boundary. Original selected root device/inode/mode remain bound for quarantine restoration. Quarantine restore/purge also require byte-exact immutable original outcome and attempt records, the original complete plan hash, exact source/recovery bindings and its numbered target mapping; imported receipt JSON alone is insufficient. Existing outputs are never overwritten. An immutable attempt record contains the exact complete plan and consumes it even if a later step fails. Numbered quarantine mappings can therefore be reconstructed from that original ordered plan even if a native move completes before its step receipt is written. Do not automatically retry; inspect attempt, step and outcome evidence and register a separately reviewed recovery action. Failure to write a later receipt can leave completed files or a partial operation, so an absent outcome is unknown, never proof of no change.
Private output admission
The service checks actual output-root privacy at construction and again before selected-scope operations and artifact writes. Existing ordinary repository/work parents may remain shared; the dedicated output namespace must be private. POSIX admission requires effective-user ownership and actual 0700 directories; newly created artifact files must be 0600. Windows admission requires the current token user to own the dedicated directory, a protected inheritable DACL with access limited to that user, SYSTEM and Administrators, and persistent ACL support. New nested output directories receive that private descriptor at creation. Checked native file creation inherits only those allowed entries, and actual file ACLs are checked afterward; copies are never first written into a broadly readable output namespace. Existing backup/manifest/probe/receipt artifacts are rechecked before being read as recovery evidence.
Same-volume Windows quarantine rename preserves the source file DACL. The selected file must therefore already pass ensure_private_file before hashing/moving, and the moved file is checked again. A broad or unknown source DACL causes a hold without permission changes, ownership takeover, or a copy/delete fallback. POSIX ordinary 0644 source permissions are preserved: the private 0700 quarantine namespace protects the moved contents by directory traversal, so no new 0600 requirement or chmod is imposed on those source files.
A filesystem that cannot demonstrate these guarantees is unsupported for private storage operations. In particular, WSL on a mounted Windows drive can report 0777 even after 0700 was requested. That is an actual capability hold, not permission to weaken the check, remount the filesystem, repair existing permissions or silently choose an external output path. Positive private-copy fixtures skip on such a filesystem; the real refusal test must still pass. Quiet-workspace and same-user/administrator limits remain as documented; these checks do not create an OS sandbox.
Scope and limits
The optional explicit protected_roots list supplies known custom-profile or retained-recovery boundaries without discovering them. Standard browser channel and multiword names are also protected, including Microsoft Edge and Google Chrome beta/development variants. Protected browser/profile/session/chat, password/credential/keychain/private-key, common authentication-file and system names are pruned before descending. Their protected boundary names remain visible as exclusions, and their children/content stay unread. Symlinks, reparses, special files, hardlinks, alias identities, case ambiguity, inaccessible scope and unsupported primitives cause categorical holds. A protected name policy is an additional boundary and cannot establish that all other files are nonsecret; scope and selected contents still need owner review. Ordinary file hashing requires a distinct explicit content-read decision. Byte equality alone never establishes whether a file can be removed.
Compiled service caps are 10,000 metadata entries, depth 48, 16 MiB per ordinary file read and 48 MiB total selected content payload per hashing/backup phase. Verification can re-read those same bounded contents; these are payload caps, not an exact whole-operation I/O estimate. Metadata logical-size totals do not consume that read budget; metadata traversal remains bounded by entry/depth/root caps, and selected content budgets are checked before any hashing. Callers may lower caps with limits keys max_entries, max_depth, max_file_bytes and max_total_bytes. They may not raise them. The archive output cap is 64 MiB. Larger content-read or archive tasks are an explicit hold requiring a narrower reviewed scope or a separately reviewed streaming design; no full-computer scan is provided. Payload buffers are bounded; total process memory, compression ratio and duration remain unknown.
Archive entries use fixed payload/000001 names. Restore paths come only from the verified manifest, with strict relative-name, protected-boundary, duplicate/case and byte caps. The ZIP contains only selected regular-file contents; original ACLs, extended attributes, ownership, timestamps, sparse allocation, executable semantics and empty directory preservation are not promised. This is file-content recovery, not a system image. ZIP compression is not encryption. Output must remain under the exact owned work directory and outside every selected source root; external drives, cloud destinations and encryption are unsupported here and must remain held.
Metadata totals are logical sizes. Physical allocation, actual reclaimable disk space, duplicate removability, free-space gain after purge and compression duration stay unknown unless separately measured. Quarantine moves usually free zero bytes. Backup creation consumes additional space; its receipt reports measured archive plus manifest bytes and explicitly leaves additional control-record size unknown. Creating a backup can need more free space before cleanup. The retained restoration probe writes another copy of the selected file contents, so the combined backup/probe/cleanup job may consume more space than it frees; combined physical net gain remains unknown. A ZIP retained in work can share the source drive and is local file-content recovery, not an off-device disaster-recovery guarantee. A required recovery backup remains outside source cleanup scope; this API does not release recovery retention or delete its own evidence/backup output.
Engine integration boundary
The coordinator registers the exact serialized storage plan as owned engine evidence, registers a concrete action with the correct kind (backup_create, backup_restore, cleanup_quarantine or permanent_purge), binds source/output scope and required recovery, obtains independent compatibility_data and evidence_security reviews and records the current owner's actual decision. It then registers fresh precondition evidence and calls WorkflowEngine.begin_operation for that exact action and plan. Only a successful durable begin receipt permits the fixed corresponding method above. Every result needs owned outcome evidence and finish_operation; failure/partial/unknown stays held and is reconciled before resuming. Pausing blocks new operations. A stored plan, mode name or schedule does not bypass that gate. Permanent purge requires its own actual irreversible-loss decision; a prior quarantine approval does not cover it.
The reusable library intentionally cannot authenticate a caller's claimed human authority. Direct Python calls execute with that process's permissions. The integrated coordinator is responsible for applying the engine gate; the read-only storage CLI supplies no alternate mutation path.
Actual test limits
Synthetic fixtures live only below work/upgrade_v3/storage; new positive fixtures and native request namespaces use the private-workspace helper, without adopting or repairing older broad fixture namespaces. Native privacy delta checks exercise unsafe-root refusal before selected content access, actual private ZIP/probe/receipt ACLs, and Windows broad-source quarantine refusal. Earlier functional results remain explicitly bound to their prior logic revisions. Test commands are python -I -S -B tests/test_storage_ops.py on a supported local Python and the same command on an explicitly selected supported Linux environment. Native adapter/platform results and capability skips must be reported as observed; mocked protocols do not certify the host filesystem. In particular a Linux renameat2 symbol does not prove a mounted filesystem supports no-overwrite moves. A runtime capability refusal is preserved with no fallback. Tests do not authorize a live assessment, backup or cleanup.
Developed by Michael. Private session records belong locally under work/; this public guide contains no live computer records.