Spaces:
Running
Running
solidprivacy-nl commited on
Commit ·
008d64e
1
Parent(s): 1f5f201
Record v13.5 two-mode UI planning
Browse files- CHANGELOG.md +101 -119
CHANGELOG.md
CHANGED
|
@@ -26,60 +26,94 @@ For UI/UX-only work, prefer pure helper modules and tests before touching Stream
|
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
-
##
|
| 30 |
|
| 31 |
-
Status: completed;
|
| 32 |
|
| 33 |
Purpose:
|
| 34 |
|
| 35 |
-
-
|
| 36 |
-
-
|
| 37 |
-
-
|
|
|
|
|
|
|
| 38 |
|
| 39 |
Files added or changed:
|
| 40 |
|
|
|
|
| 41 |
- Changed `WORKPACKAGES.md`.
|
| 42 |
- Changed `CHANGELOG.md`.
|
| 43 |
-
- Added `handover/workpackages/
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
-
|
| 48 |
-
-
|
| 49 |
-
-
|
| 50 |
-
-
|
| 51 |
-
-
|
| 52 |
-
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
-
|
| 61 |
-
|
| 62 |
-
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
-
|
| 69 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
Intentionally not changed:
|
| 72 |
|
| 73 |
-
- No code
|
| 74 |
-
- No
|
| 75 |
-
- No
|
| 76 |
- No edit to `scrub_key_document_reinsert.py`.
|
| 77 |
-
- No edit to `tests/test_scrub_key_document_reinsert.py`.
|
| 78 |
- No edit to `scrub_key_reinsert.py`.
|
| 79 |
- No edit to `scrub_key.py`.
|
| 80 |
- No edit to `scrub_key_import.py`.
|
| 81 |
-
- No edit to `
|
| 82 |
-
- No
|
| 83 |
- No PDF reinsert added.
|
| 84 |
- No AI calls added.
|
| 85 |
- No cloud processing added.
|
|
@@ -89,6 +123,32 @@ Intentionally not changed:
|
|
| 89 |
|
| 90 |
Outcome:
|
| 91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
- WP10B closeout is complete.
|
| 93 |
- Coordinator should verify Actions/sync externally before marking WP10 formally closed.
|
| 94 |
- Next recommended workpackage remains WP11 — v13.5 Two-mode reinsert UI planning.
|
|
@@ -106,85 +166,6 @@ Purpose:
|
|
| 106 |
- Reuse the existing deterministic Scrub Key reinsert logic.
|
| 107 |
- Keep PDF, UI, AI calls and cloud processing out of scope.
|
| 108 |
|
| 109 |
-
Files added or changed:
|
| 110 |
-
|
| 111 |
-
- Added `scrub_key_document_reinsert.py`.
|
| 112 |
-
- Added `tests/test_scrub_key_document_reinsert.py`.
|
| 113 |
-
- Changed `WORKPACKAGES.md`.
|
| 114 |
-
- Changed `CHANGELOG.md`.
|
| 115 |
-
- Added `handover/workpackages/20260608_0000_v13_4_txt_docx_reinsert_foundation.md`.
|
| 116 |
-
|
| 117 |
-
Main helper behavior:
|
| 118 |
-
|
| 119 |
-
- Added `reinsert_text_document(text, scrub_key)` for plain text document-level reinsert.
|
| 120 |
-
- Added `reinsert_txt_bytes(content, scrub_key, encoding="utf-8")` for TXT bytes input/output.
|
| 121 |
-
- Added `reinsert_docx_bytes(content, scrub_key)` for DOCX main-document text-node reinsert.
|
| 122 |
-
- Reuses `reinsert_from_scrub_key(...)` from `scrub_key_reinsert.py`.
|
| 123 |
-
- Returns restored content and audit summary.
|
| 124 |
-
- Reports `document_type`, `replacement_count`, `item_count`, `active_item_count`, `excluded_item_count`, `placeholders_not_found`, `unknown_placeholders`, `duplicate_placeholders`, `validation_issues`, `local_only`, `ai_processing` and `cloud_processing`.
|
| 125 |
-
- Remains deterministic and side-effect free.
|
| 126 |
-
- Does not mutate the input Scrub Key.
|
| 127 |
-
|
| 128 |
-
TXT behavior:
|
| 129 |
-
|
| 130 |
-
- Accepts plain text through `reinsert_text_document(...)`.
|
| 131 |
-
- Accepts bytes through `reinsert_txt_bytes(...)`.
|
| 132 |
-
- Decodes bytes strictly with UTF-8 by default.
|
| 133 |
-
- Returns restored text and restored bytes.
|
| 134 |
-
- Reports decode/type issues as validation issues instead of silently changing input.
|
| 135 |
-
|
| 136 |
-
DOCX behavior and limitations:
|
| 137 |
-
|
| 138 |
-
- Uses Python standard library only: `zipfile`, `BytesIO` and `xml.etree.ElementTree`.
|
| 139 |
-
- Processes only `word/document.xml` text nodes.
|
| 140 |
-
- Supports normal body paragraphs and tables in `word/document.xml`.
|
| 141 |
-
- Returns restored DOCX bytes.
|
| 142 |
-
- Leaves the original uploaded bytes untouched.
|
| 143 |
-
- Does not restore placeholders split across multiple Word runs/text nodes.
|
| 144 |
-
- Does not process headers, footers, comments, tracked changes or metadata.
|
| 145 |
-
- Does not claim perfect formatting preservation.
|
| 146 |
-
- Records limitations in returned `limitations` and `unsupported_parts` fields.
|
| 147 |
-
|
| 148 |
-
Tests added:
|
| 149 |
-
|
| 150 |
-
- TXT text reinsert with one placeholder.
|
| 151 |
-
- TXT bytes reinsert with multiple placeholders.
|
| 152 |
-
- TXT unknown placeholder remains unchanged and is reported.
|
| 153 |
-
- Invalid Scrub Key returns validation issues.
|
| 154 |
-
- DOCX reinsert with one placeholder in a paragraph.
|
| 155 |
-
- DOCX reinsert with multiple placeholders.
|
| 156 |
-
- DOCX table text replacement in `word/document.xml`.
|
| 157 |
-
- DOCX output remains a valid DOCX package.
|
| 158 |
-
- DOCX paragraph text is restored correctly.
|
| 159 |
-
- DOCX helper returns audit summary.
|
| 160 |
-
- DOCX unsupported areas / limitations are documented.
|
| 161 |
-
- Helper does not mutate input Scrub Key.
|
| 162 |
-
- No AI/cloud behavior.
|
| 163 |
-
- Synthetic values only.
|
| 164 |
-
|
| 165 |
-
Testing and validation:
|
| 166 |
-
|
| 167 |
-
- Local reconstructed targeted validation:
|
| 168 |
-
- `PYTHONPATH=. pytest -q tests/test_scrub_key.py` → 6 passed.
|
| 169 |
-
- `PYTHONPATH=. pytest -q tests/test_scrub_key_reinsert.py` → 12 passed.
|
| 170 |
-
- `PYTHONPATH=. pytest -q tests/test_scrub_key_document_reinsert.py` → 14 passed.
|
| 171 |
-
- Local reconstructed full available subset:
|
| 172 |
-
- `PYTHONPATH=. pytest -q` → 32 passed.
|
| 173 |
-
- Repository clone via container was not possible because outbound GitHub DNS was unavailable, so validation was performed on reconstructed files from GitHub-fetched content plus the new helper/tests.
|
| 174 |
-
|
| 175 |
-
Intentionally not changed:
|
| 176 |
-
|
| 177 |
-
- No UI files changed.
|
| 178 |
-
- No edit to `fix_streamlit_nested_expanders.py`.
|
| 179 |
-
- No edit to `presidio_streamlit.py`.
|
| 180 |
-
- No PDF reinsert implementation added.
|
| 181 |
-
- No AI calls added.
|
| 182 |
-
- No cloud processing added.
|
| 183 |
-
- No automatic app document rehydration added.
|
| 184 |
-
- No existing TXT, CSV, DOCX or PDF scrubbed export/download behavior changed.
|
| 185 |
-
- No Scrub Key JSON export/import behavior changed.
|
| 186 |
-
- No secrets, tokens or real personal data stored.
|
| 187 |
-
|
| 188 |
Outcome:
|
| 189 |
|
| 190 |
- WP10 helper/test foundation is implemented.
|
|
@@ -288,7 +269,8 @@ Outcome:
|
|
| 288 |
Possible directions:
|
| 289 |
|
| 290 |
- Coordinator verification of WP10 Actions/sync.
|
| 291 |
-
- Two-mode UI
|
| 292 |
-
- TXT
|
|
|
|
| 293 |
- PDF text extraction research only after separate reliability review.
|
| 294 |
- Further recognizer expansion by legal domain.
|
|
|
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
+
## WP11 — v13.5 Two-mode reinsert UI planning
|
| 30 |
|
| 31 |
+
Status: completed; planning/specification-only workpackage.
|
| 32 |
|
| 33 |
Purpose:
|
| 34 |
|
| 35 |
+
- Plan the future two-mode UI before changing Streamlit UI code.
|
| 36 |
+
- Clearly separate `Anonimiseren` from `Originele waarden terugzetten`.
|
| 37 |
+
- Decide where pasted-text, TXT and DOCX reinsert should fit.
|
| 38 |
+
- Compare current single-scroll workflow, tabs and landing-card options.
|
| 39 |
+
- Define the next safe UI implementation workpackage.
|
| 40 |
|
| 41 |
Files added or changed:
|
| 42 |
|
| 43 |
+
- Added `TWO_MODE_UI_SPEC.md`.
|
| 44 |
- Changed `WORKPACKAGES.md`.
|
| 45 |
- Changed `CHANGELOG.md`.
|
| 46 |
+
- Added `handover/workpackages/20260608_0000_v13_5_two_mode_ui_planning.md`.
|
| 47 |
+
|
| 48 |
+
Main recommendation:
|
| 49 |
+
|
| 50 |
+
- Move Scrub toward a two-mode interface:
|
| 51 |
+
- `Anonimiseren`;
|
| 52 |
+
- `Originele waarden terugzetten`.
|
| 53 |
+
- Use Streamlit tabs or two clear mode panels as the first implementation step.
|
| 54 |
+
- Treat a landing choice with two large cards/buttons as the better long-term product direction, but not the next patch-based implementation step.
|
| 55 |
+
- Keep pasted-text reinsert available as the simplest and safest fallback.
|
| 56 |
+
- Add TXT upload/download reinsert UI after the mode skeleton is verified.
|
| 57 |
+
- Add DOCX upload/download reinsert UI later using the WP10 helper.
|
| 58 |
+
- Keep PDF reinsert excluded until a separate reliability review.
|
| 59 |
+
|
| 60 |
+
Options compared:
|
| 61 |
+
|
| 62 |
+
1. Current single-scroll workflow:
|
| 63 |
+
- lowest short-term cost;
|
| 64 |
+
- too much cognitive load;
|
| 65 |
+
- weak separation between scrubbed and restored privacy states.
|
| 66 |
+
2. Streamlit tabs:
|
| 67 |
+
- recommended first step;
|
| 68 |
+
- clear mode separation;
|
| 69 |
+
- lower implementation risk in current app structure.
|
| 70 |
+
3. Landing choice with two large cards/buttons:
|
| 71 |
+
- best mature product UX;
|
| 72 |
+
- higher refactor risk;
|
| 73 |
+
- better later after the patch-based UI has been simplified.
|
| 74 |
+
|
| 75 |
+
Specified user journeys:
|
| 76 |
+
|
| 77 |
+
- `Anonimiseren`:
|
| 78 |
+
- upload/paste source text or document;
|
| 79 |
+
- review detected replacements;
|
| 80 |
+
- download currently supported scrubbed TXT/DOCX/PDF outputs;
|
| 81 |
+
- optionally download Scrub Key JSON;
|
| 82 |
+
- show warning that Scrub Key is reversible/pseudonymization.
|
| 83 |
+
- `Originele waarden terugzetten`:
|
| 84 |
+
- load/paste Scrub Key;
|
| 85 |
+
- choose paste text, upload TXT or upload DOCX;
|
| 86 |
+
- validate key locally;
|
| 87 |
+
- reinsert original values locally;
|
| 88 |
+
- show audit summary;
|
| 89 |
+
- warn restored output may contain sensitive/confidential data again;
|
| 90 |
+
- download restored TXT or DOCX where supported.
|
| 91 |
+
|
| 92 |
+
Recommended implementation sequence:
|
| 93 |
+
|
| 94 |
+
1. `WP12 — v13.6 Two-mode UI skeleton and tab separation`.
|
| 95 |
+
2. `WP13 — v13.7 TXT reinsert upload/download UI`.
|
| 96 |
+
3. `WP14 — v13.8 DOCX reinsert upload/download UI`.
|
| 97 |
+
4. `WP15 — PDF text extraction reliability review only`.
|
| 98 |
+
|
| 99 |
+
Validation:
|
| 100 |
+
|
| 101 |
+
- Tests: not applicable; planning/specification-only workpackage.
|
| 102 |
+
- App verification: not applicable; no UI behavior changed.
|
| 103 |
+
- GitHub Actions: not required for planning-only documentation change.
|
| 104 |
+
- Hugging Face sync: not required for planning-only documentation change.
|
| 105 |
|
| 106 |
Intentionally not changed:
|
| 107 |
|
| 108 |
+
- No UI code changed.
|
| 109 |
+
- No edit to `fix_streamlit_nested_expanders.py`.
|
| 110 |
+
- No edit to `presidio_streamlit.py`.
|
| 111 |
- No edit to `scrub_key_document_reinsert.py`.
|
|
|
|
| 112 |
- No edit to `scrub_key_reinsert.py`.
|
| 113 |
- No edit to `scrub_key.py`.
|
| 114 |
- No edit to `scrub_key_import.py`.
|
| 115 |
+
- No edit to `tests/*`.
|
| 116 |
+
- No TXT/DOCX reinsert UI added.
|
| 117 |
- No PDF reinsert added.
|
| 118 |
- No AI calls added.
|
| 119 |
- No cloud processing added.
|
|
|
|
| 123 |
|
| 124 |
Outcome:
|
| 125 |
|
| 126 |
+
- WP11 planning is complete.
|
| 127 |
+
- Next recommended implementation workpackage is `WP12 — v13.6 Two-mode UI skeleton and tab separation`.
|
| 128 |
+
|
| 129 |
+
---
|
| 130 |
+
|
| 131 |
+
## WP10B — v13.4 TXT/DOCX reinsert foundation verification and closeout
|
| 132 |
+
|
| 133 |
+
Status: completed; Actions/sync not visible through connector, coordinator verification required.
|
| 134 |
+
|
| 135 |
+
Purpose:
|
| 136 |
+
|
| 137 |
+
- Verify and formally close, or pending-close, WP10.
|
| 138 |
+
- Check GitHub Actions and Hugging Face sync visibility for implementation commit `eb0c1ed2397ec1a4dc256d6e7e615ac4c026c0ee`.
|
| 139 |
+
- Update control files without changing code.
|
| 140 |
+
|
| 141 |
+
Verification result:
|
| 142 |
+
|
| 143 |
+
- Commit metadata was visible for `eb0c1ed2397ec1a4dc256d6e7e615ac4c026c0ee`.
|
| 144 |
+
- GitHub combined commit status returned an empty status list.
|
| 145 |
+
- GitHub workflow-runs query for the commit returned no visible workflow runs.
|
| 146 |
+
- GitHub Actions: not visible through connector.
|
| 147 |
+
- Hugging Face sync: not visible through connector.
|
| 148 |
+
- App verification: not applicable; WP10 was helper/test-only and added no UI behavior.
|
| 149 |
+
|
| 150 |
+
Outcome:
|
| 151 |
+
|
| 152 |
- WP10B closeout is complete.
|
| 153 |
- Coordinator should verify Actions/sync externally before marking WP10 formally closed.
|
| 154 |
- Next recommended workpackage remains WP11 — v13.5 Two-mode reinsert UI planning.
|
|
|
|
| 166 |
- Reuse the existing deterministic Scrub Key reinsert logic.
|
| 167 |
- Keep PDF, UI, AI calls and cloud processing out of scope.
|
| 168 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 169 |
Outcome:
|
| 170 |
|
| 171 |
- WP10 helper/test foundation is implemented.
|
|
|
|
| 269 |
Possible directions:
|
| 270 |
|
| 271 |
- Coordinator verification of WP10 Actions/sync.
|
| 272 |
+
- Two-mode UI skeleton and tab separation.
|
| 273 |
+
- TXT reinsert upload/download UI.
|
| 274 |
+
- DOCX reinsert upload/download UI.
|
| 275 |
- PDF text extraction research only after separate reliability review.
|
| 276 |
- Further recognizer expansion by legal domain.
|