# Changelog — SolidPrivacy Scrub This changelog records meaningful product, architecture, workflow and recognizer changes for the Scrub legal document scrubber. Conventions: - Keep this file explicit and human-readable. - Group changes by project phase / version. - Record what changed, why it changed, and what is intentionally not changed. - Do not use this as a substitute for tests; every recognizer hardening step should also add or update regression tests. --- ## Current development rule From v10 onward, recognizer work follows this order: 1. Add or update synthetic regression cases. 2. Add or update tests. 3. Change recognizer / scanner logic. 4. Verify GitHub Actions tests are green. 5. Let GitHub sync to Hugging Face automatically. 6. Test the app in Hugging Face. For UI/UX-only work, prefer pure helper modules and tests before touching Streamlit UI flow. --- ## v13.3 — Deterministic reinsert helper closeout Status: implemented; awaiting coordinator verification of Actions/sync. Purpose: - Verify and close out the v13.3 deterministic reinsert helper workpackage as far as possible from the connector environment. - Record that the helper remains pure, local and deterministic. - Preserve the boundary that no UI, AI-output flow, cloud processing or export/download behavior change was added. Files added or changed in the full v13.3 helper line: - `scrub_key_reinsert.py` - `tests/test_scrub_key_reinsert.py` - `WORKPACKAGES.md` - `CHANGELOG.md` - `handover/workpackages/20260607_1745_v13_3_reinsert_helper.md` - `handover/workpackages/20260607_1815_v13_3_reinsert_helper.md` - `handover/workpackages/20260607_1830_v13_3_reinsert_helper_closeout.md` Main changes: - Added `detect_placeholders(text)` for conservative placeholder-token detection. - Added `build_reinsert_mapping(scrub_key)` to build a deterministic placeholder-to-original mapping from included Scrub Key items. - Added `reinsert_from_scrub_key(text, scrub_key)` to return reinserted text and an audit summary. - Reused existing `validate_scrub_key(...)` validation. - Invalid Scrub Keys return validation issues and do not modify the input text. - Duplicate placeholder entries are detected and excluded from reinsertion to avoid ambiguity. - Excluded Scrub Key items are ignored even if malformed/imported data contains them. - Audit output includes item count, active item count, excluded item count, replacement count, placeholders not found, unknown placeholders, duplicate placeholders and validation issues. - Audit output explicitly records local/no-AI/no-cloud behavior through `local_only=True`, `ai_processing=False` and `cloud_processing=False`. Testing and verification: - Added `tests/test_scrub_key_reinsert.py`. - Tests cover valid reinsert, multiple placeholders, repeated placeholders, missing placeholders, unknown placeholders, invalid Scrub Key validation issues, duplicate placeholder detection, excluded rows not being reinserted, synthetic values only, input immutability and no-AI/no-cloud flags. - Local targeted validation recorded by the implementation worker on the available/reconstructed subset: - `PYTHONPATH=. pytest -q tests/test_scrub_key.py tests/test_scrub_key_import.py tests/test_scrub_key_reinsert.py` → 25 passed. - GitHub Actions lookup for `07d33cad10dacbcd634125f82ab234388f84729b` returned `workflow_runs: []` through the connector. - GitHub Actions status is therefore not visible to this closeout worker. - Hugging Face sync status is not visible to this closeout worker. - App verification is not applicable because this is a helper-only package. Intentionally not changed: - No code files changed in WP7B closeout. - No UI added. - No direct edit to `presidio_streamlit.py`. - No direct edit to `fix_streamlit_nested_expanders.py`. - No edit to `scrub_key.py`. - No edit to `scrub_key_import.py`. - No AI calls. - No cloud processing. - No automatic document rehydration. - No change to TXT, CSV, DOCX or PDF export behavior. - No change to Scrub Key export behavior. - No change to Scrub Key import UI behavior. - No secrets, tokens or real personal data. Outcome: - v13.3 deterministic reinsert helper is implemented. - Formal external Actions/sync verification remains pending coordinator verification. - A separate v13.3 reinsert UI workpackage should only start after verification. --- ## v13.2 — Scrub Key import/reload UI app verification closeout Status: completed, app-verified and closed. Purpose: - Administratively close the v13.2 Scrub Key import/reload UI after app verification. - Record that the implemented import/reload flow works in the Hugging Face app. - Preserve the boundary that this phase is import/reload only and does not add AI-output reinsert. Files added or changed: - `WORKPACKAGES.md` - `CHANGELOG.md` - `handover/workpackages/20260607_1730_v13_2_scrub_key_import_ui_app_closeout.md` Technical evidence already recorded: - Tests #89 green — commit `83353e4`. - Tests #90 green — commit `4a1ef55`. - Sync to Hugging Face Space #104 green — commit `4a1ef55`. - Tests #91 green — commit `4d8bfe9`. - Sync to Hugging Face Space #105 green — commit `4d8bfe9`. - Tests #92 green — commit `ff8321f`. - Sync to Hugging Face Space #106 green — commit `ff8321f`. App verification: - Confirmed by coordinator/user. - `Scrub Key laden` works. - Scrub Key import/reload UI is visible. - Upload/paste import flow works. - Pseudonymization/reversibility warning is visible. - Existing `Download Scrub Key (.json)` remains visible. - Existing TXT, CSV, DOCX and PDF downloads remain available. Closeout notes: - GitHub Actions tests were green based on coordinator evidence. - GitHub to Hugging Face sync was green based on coordinator evidence. - Import/reload remains local and uses the existing helper logic. - The key remains pseudonymization/reversible and must be protected. - No AI-output reinsert behavior was added. - No automatic document rehydration was added. - No export/download behavior was intentionally changed. Intentionally not changed in this app-verification closeout: - No code files changed. - No tests changed. - No direct edit to `fix_streamlit_nested_expanders.py`. - No direct edit to `presidio_streamlit.py`. - No edit to `scrub_key.py`. - No edit to `scrub_key_import.py`. - No edit to `tests/*`. - No AI-output reinsert behavior. - No automatic document rehydration. - No change to TXT, CSV, DOCX or PDF export/download behavior. - No cloud processing. - No secrets, tokens or real personal data. Outcome: - v13.2 Scrub Key import/reload UI is completed, app-verified and closed. --- ## Earlier completed work - v13.2 Scrub Key import/reload UI integration. - v13.2 Scrub Key import/reload helper and tests. - v13.1 Scrub Key JSON export UI closeout. - v12.6 Export sanity checks closeout. - v13.0 Scrub Key specification and pure model. - v12.5 Final review summary. - v12.4 Review guidance text. - Project governance setup. - v12.3 Review table simplification. - v12.2 Review focus filters. - v12.1 Review table status model. - v11.2 Dutch recognizer integration tests. - v11.1 Legal reference recognizer hardening. - v10 Regression test layer. - v9.1 UI polish and baseline stabilization. - v9 Dutch Legal UI Layer. --- ## Planned later phase — v13 and beyond Possible directions: - Deterministic reinsert UI after verification. - AI-output reinsert. - Further recognizer expansion by legal domain.