Spaces:
Running
Running
| # Changelog — SolidPrivacy Scrub | |
| This changelog records meaningful product, architecture, workflow and recognizer changes for the Scrub legal document scrubber. | |
| Conventions: | |
| - Keep this file explicit and human-readable. | |
| - Group changes by project phase / version. | |
| - Record what changed, why it changed, and what is intentionally not changed. | |
| - Do not use this as a substitute for tests; every recognizer hardening step should also add or update regression tests. | |
| --- | |
| ## Current development rule | |
| From v10 onward, recognizer work follows this order: | |
| 1. Add or update synthetic regression cases. | |
| 2. Add or update tests. | |
| 3. Change recognizer / scanner logic. | |
| 4. Verify GitHub Actions tests are green. | |
| 5. Let GitHub sync to Hugging Face automatically. | |
| 6. Test the app in Hugging Face. | |
| For UI/UX-only work, prefer pure helper modules and tests before touching Streamlit UI flow. | |
| --- | |
| ## WP13 — v13.7 TXT reinsert upload/download UI | |
| Status: implemented; awaiting GitHub Actions, Hugging Face sync and app verification. | |
| Purpose: | |
| - Add controlled TXT upload/download support inside `Originele waarden terugzetten`. | |
| - Keep the existing pasted-text reinsert flow available as fallback. | |
| - Reuse the existing deterministic local TXT helper. | |
| - Keep DOCX upload reinsert UI, PDF reinsert, AI and cloud behavior out of scope. | |
| Files added or changed: | |
| - Changed `fix_streamlit_nested_expanders.py`. | |
| - Changed `tests/test_two_mode_ui_patch.py`. | |
| - Changed `WORKPACKAGES.md`. | |
| - Changed `CHANGELOG.md`. | |
| - Added `tests/test_txt_reinsert_ui_patch.py`. | |
| - Added `handover/workpackages/20260608_0000_v13_7_txt_reinsert_upload_download_ui.md`. | |
| Main change: | |
| - Added section `TXT-bestand terugzetten` inside the `Originele waarden terugzetten` mode. | |
| - Added TXT upload label `Upload een TXT-bestand met placeholders`. | |
| - Added action button `Zet TXT-bestand lokaal terug`. | |
| - Added output label `Herstelde TXT-tekst`. | |
| - Added download label `Download hersteld TXT-bestand (.txt)`. | |
| - The UI calls: | |
| - `reinsert_txt_bytes(content, scrub_key, encoding="utf-8")`. | |
| - TXT reinsert requires a loaded Scrub Key before running. | |
| - The restored TXT result shows text output and an audit summary. | |
| - The existing pasted-text reinsert flow remains available. | |
| - The existing anonymization workflow remains in `Anonimiseren`. | |
| Safety/warnings: | |
| - The TXT reinsert UI warns that restored output may contain personal/confidential data again. | |
| - The UI states that reinsert is local-only and uses no AI/cloud processing. | |
| - The Scrub Key warning remains present and keeps the pseudonymization/reversibility context. | |
| Tests added/updated: | |
| - Added `tests/test_txt_reinsert_ui_patch.py`. | |
| - Updated `tests/test_two_mode_ui_patch.py`. | |
| The tests check: | |
| - `reinsert_txt_bytes` is imported and used; | |
| - TXT UI labels are present; | |
| - TXT upload accepts `.txt` only; | |
| - TXT reinsert requires a loaded Scrub Key; | |
| - TXT reinsert is injected only in the `Originele waarden terugzetten` mode; | |
| - pasted-text reinsert labels remain present; | |
| - `Scrub Key laden` remains present; | |
| - `Anonimiseren` remains present; | |
| - existing anonymization/export markers remain present; | |
| - existing Scrub Key export remains present; | |
| - no DOCX upload reinsert UI is added; | |
| - no PDF reinsert is added; | |
| - no AI/cloud behavior is added; | |
| - `apply_replacements_to_text` is not altered; | |
| - existing scrubbed download markers are not rewired. | |
| Validation: | |
| - Local clone/test run could not be performed in the container because outbound GitHub DNS was unavailable: | |
| - `Could not resolve host: github.com`. | |
| - GitHub Actions: awaiting verification for WP13 commits. | |
| - Hugging Face sync: awaiting verification for WP13 commits. | |
| - App verification: required because UI behavior changed. | |
| Intentionally not changed: | |
| - `presidio_streamlit.py` was not directly edited. | |
| - No DOCX upload reinsert UI added. | |
| - No PDF reinsert added. | |
| - No AI calls added. | |
| - No cloud processing added. | |
| - No automatic document rehydration beyond TXT local reinsert added. | |
| - No existing TXT, CSV, DOCX or PDF scrubbed export/download behavior intentionally changed. | |
| - No Scrub Key JSON export behavior intentionally changed. | |
| - No Scrub Key import/reload behavior intentionally changed except reusing the loaded key for TXT reinsert. | |
| - No Scrub Key storage, secrets, tokens or real personal data added. | |
| Outcome: | |
| - WP13 is implemented and awaits GitHub Actions, Hugging Face sync and app verification. | |
| - Next recommended workpackage is `WP13-CLOSEOUT — v13.7 TXT reinsert upload/download UI app verification closeout`. | |
| --- | |
| ## WP12B — v13.6 Two-mode UI app verification closeout | |
| Status: completed and app-verified after Actions/sync verification. | |
| Purpose: | |
| - Administratively close WP12, WP12-FIX and WP12-FIX2 after successful technical verification and app verification. | |
| - Record that v13.6 two-mode UI is now complete. | |
| - Confirm that no further closeout blocker remains before WP13. | |
| Implementation sequence closed: | |
| - WP12 introduced the two-mode UI skeleton. | |
| - WP12-FIX cleaned up content separation. | |
| - WP12-FIX2 fixed the indentation/runtime error. | |
| - WP12B records successful Actions/sync and app verification. | |
| Technical verification evidence: | |
| ```text | |
| Tests #155 green — commit b27d115 | |
| Sync to Hugging Face Space #169 green — commit b27d115 | |
| Tests #156 green — commit 0e357bb | |
| Sync to Hugging Face Space #170 green — commit 0e357bb | |
| Tests #157 green — commit 268234d | |
| Sync to Hugging Face Space #171 green — commit 268234d | |
| ``` | |
| Latest verified WP12-FIX2 commit: | |
| ```text | |
| 268234d9d1aeb9c82658c4c30702f51cfdd58c96 | |
| ``` | |
| Outcome: | |
| - v13.6 two-mode UI is closed as completed and app-verified. | |
| - Next recommended workpackage is `WP13 — v13.7 TXT reinsert upload/download UI`. | |
| --- | |
| ## WP12-FIX2 — v13.6 Two-mode indentation/runtime hotfix | |
| Status: completed and app-verified through WP12B closeout. | |
| Outcome: | |
| - Generated indentation around `Scrub Key laden` and the local reinsert UI block was corrected. | |
| - GitHub Actions and Hugging Face sync were later verified green by the coordinator. | |
| - App verification confirmed the script execution error is gone. | |
| --- | |
| ## WP12-FIX — v13.6 Two-mode UI content separation cleanup | |
| Status: completed through WP12B closeout after WP12-FIX2 runtime hotfix. | |
| Outcome: | |
| - WP12-FIX separated the intended content paths conceptually. | |
| - WP12-FIX2 corrected the runtime indentation issue. | |
| - WP12B confirmed app verification. | |
| --- | |
| ## WP12 — v13.6 Two-mode UI skeleton and tab separation | |
| Status: completed and app-verified through WP12B closeout. | |
| Outcome: | |
| - WP12 created the first visible mode skeleton. | |
| - WP12-FIX improved actual content separation. | |
| - WP12-FIX2 fixed the runtime indentation error. | |
| - WP12B closed v13.6 after Actions/sync and app verification. | |
| --- | |
| ## WP11 — v13.5 Two-mode reinsert UI planning | |
| Status: completed; planning/specification-only workpackage. | |
| Outcome: | |
| - WP11 planning is complete. | |
| - Next recommended implementation workpackage was `WP12 — v13.6 Two-mode UI skeleton and tab separation`. | |
| --- | |
| ## WP10B — v13.4 TXT/DOCX reinsert foundation verification and closeout | |
| Status: completed; Actions/sync not visible through connector, coordinator verification required. | |
| Outcome: | |
| - WP10B closeout is complete. | |
| - Coordinator should verify Actions/sync externally before marking WP10 formally closed. | |
| --- | |
| ## WP10 — v13.4 TXT/DOCX reinsert foundation helper and tests | |
| Status: implemented; awaiting coordinator verification of Actions/sync. | |
| Outcome: | |
| - WP10 helper/test foundation is implemented. | |
| - WP10 awaits coordinator verification of GitHub Actions and Hugging Face sync. | |
| --- | |
| ## WP9 — AI-output / document reinsert workflow UX and architecture review | |
| Status: completed; review-only workpackage. | |
| Outcome: | |
| - WP9 is complete. | |
| - Product direction for reinsert is documented before implementation. | |
| --- | |
| ## v13.3 — Deterministic reinsert UI app verification closeout | |
| Status: completed and app-verified after Actions/sync verification. | |
| Outcome: | |
| - v13.3 deterministic local reinsert UI is completed, app-verified and formally closed. | |
| --- | |
| ## Earlier completed work | |
| - v13.6 two-mode UI implementation and app verification closeout. | |
| - v13.3 deterministic reinsert UI implementation. | |
| - v13.3 deterministic reinsert UI planning. | |
| - v13.3 deterministic reinsert helper verification reconciliation. | |
| - v13.2 Scrub Key import/reload UI app verification closeout. | |
| - v13.2 Scrub Key import/reload UI integration. | |
| - v13.2 Scrub Key import/reload helper and tests. | |
| - v13.1 Scrub Key JSON export UI closeout. | |
| - v12.6 Export sanity checks closeout. | |
| - v13.0 Scrub Key specification and pure model. | |
| - v12.5 Final review summary. | |
| - v12.4 Review guidance text. | |
| - Project governance setup. | |
| - v12.3 Review table simplification. | |
| - v12.2 Review focus filters. | |
| - v12.1 Review table status model. | |
| - v11.2 Dutch recognizer integration tests. | |
| - v11.1 Legal reference recognizer hardening. | |
| - v10 Regression test layer. | |
| - v9.1 UI polish and baseline stabilization. | |
| - v9 Dutch Legal UI Layer. | |
| --- | |
| ## Planned later phase — v13 and beyond | |
| Possible directions: | |
| - WP13-CLOSEOUT app verification closeout. | |
| - WP14 — v13.8 DOCX reinsert upload/download UI. | |
| - PDF text extraction research only after separate reliability review. | |
| - Further recognizer expansion by legal domain. | |