Hugging Face's logo

andyshu
/
opensysone

Text Classification
opensysone
decision-scoring
lora
experimental
Model card Files
xet
 Community
opensysone / source /HUGGINGFACE.md
andyshu's picture
andyshu
Back up verified OpenSysOne training snapshot and pinned source
9f7eb0d verified
preview code
|
Raw History Blame
5.61 kB

Hugging Face backup

The user requested backups in andyshu/opensysone. The existing model repository is private; retain its visibility and license: unknown metadata. Authentication uses the existing Hugging Face credential store. Never copy credentials into source, logs, manifests or artifacts.

Publication verified on 17 September at 02:54 UTC. Source, documentation, provenance and all four selected/resumable checkpoint pairs are saved in the private repository. The publisher exited 0 after checking every remote size and hash, then publishing and checking CURRENT_SNAPSHOT.json and the model card. The first verified payload commit is 2d5c26ab7929b8c858aeea88842781fdbf79cd54, with pointer commit b213728f9acc5e009bc96704db341913d582be4c and source 93602e6. hf-snapshot-publication.json records the verification. The live CURRENT_SNAPSHOT.json is authoritative when documentation/source is refreshed.

The replacement credential authenticates as andyshu with write access and is stored in the existing login store; token values are excluded from artifacts. The earlier HTTP 403/read-only attempt remains as historical evidence in hf-initial-artifacts-publication.json; hf-write-auth-verified.json records that authentication was repaired.

The final watcher has no inherited HF_TOKEN/HUGGING_FACE_HUB_TOKEN override; it reads the updated stored login when publication begins. Commands in a shell that retains the old read-only environment token must unset those overrides.

Initial snapshot

Immutable local staging: /home/andy/ai/opensysone/exports/20260917T022835Z-hf-backup. Remote payload: snapshots/20260917T022835Z-hf-backup/. The payload contains 1,066,618,246 bytes of checkpoints and provenance, excluding the manifest/checksum index. All four selected/resumable pairs were verified on CPU against stable source hashes, model/data provenance and validation evidence.

Candidate Selected step Resumable step
GX10 4B 2,500 2,598
Spark A 4B 2,500 2,604
Spark B 2B 2,000 6,000
Spark B 4B refinement pilot branch 0, inherited GX10 2,500 8

These are uncalibrated snapshots. The fourth backup is the completed pilot; the longer refinement campaign runs independently. The remote CURRENT_SNAPSHOT.json records the verified payload and committed source snapshot. source/ contains browsable code, documentation and small evidence. Source archives preserve both the current documentation revision and exact 4a60423 training revision. Base weights and raw datasets are excluded; pinned revisions and dataset reconstruction requirements are in the backup manifest and model card.

Upload completion is recorded in results/20260917-fleet-progress/hf-*-publication.json. Do not infer success from staging alone. Remote verification checks every file's size and LFS SHA-256 or Git blob hash, plus the small manifest bytes.

For a later explicit snapshot refresh, use the existing stored login:

env -u HF_TOKEN -u HUGGING_FACE_HUB_TOKEN \
  ~/ai/envs/opensysone/bin/python scripts/publish_hf_snapshot.py \
  --artifacts /home/andy/ai/opensysone/exports/20260917T022835Z-hf-backup \
  --repo-id andyshu/opensysone \
  --output /home/andy/ai/opensysone/runs/NEW-UNUSED-hf-snapshot-publish

Use a fresh output directory for each attempt and commit source/docs first. This creates a separate immutable publication stage, archives committed HEAD plus each recorded training revision, and publishes sources/<commit>/source.tar.gz with per-file manifests. It checks all remote files before setting CURRENT_SNAPSHOT.json and the model card together. It refuses to replace an existing final release. The attempt has a 30-minute timeout and its own state.json/exit_code; failures record only the error type, without credentials or signed URLs. No retry has run with the known read-only token.

Automatic final publication

scripts/publish_hf_final.py runs independently of training and the fleet controller. It waits for a complete, API-ready fleet with exit 0, checks the frozen selection, completed test/holdout metrics, correctness evidence and matching model/deployment/ probe hashes, then creates an immutable export under ~/ai/opensysone/exports. It uploads to final/<fleet-campaign>/, verifies the remote payload, and only then updates FINAL_MODEL.json and the model card together. It never selects or evaluates a model itself. Failure of this backup does not change the fleet's deployment state.

The watcher stops by 2026-09-17 18:46:10 UTC, 30 minutes after the original delivery deadline. Uploads have three bounded attempts. Training still stops by 16:00 UTC and the final model's deadline remains 18:16:10 UTC.

The current watcher path and process are recorded in HANDOVER.md. Inspect its state.json, exit_code and adjacent log. Before stopping, verify /proc/<pid>/cmdline against state.json.command, then send SIGTERM to that exact watcher only. Resume with a fresh output directory; an existing immutable export is hash-checked and reused:

env -u HF_TOKEN -u HUGGING_FACE_HUB_TOKEN \
  ~/ai/envs/opensysone/bin/python scripts/publish_hf_final.py \
  --campaign /home/andy/ai/opensysone/runs/20260916T194403396250Z-fleet \
  --repo-id andyshu/opensysone \
  --output /home/andy/ai/opensysone/runs/NEW-UNUSED-hf-final-watch --watch

The command stays in the foreground; detach with a recorded process/log if leaving it unattended. Never start a second watcher without checking the recorded one.