Hugging Face's logo

Spaces:
solidprivacy
/
scrub
Sleeping

App Files Community
scrub / WORKPACKAGES.md
solidprivacy-nl
Record v13.5 two-mode UI planning outcome
1f5f201
preview code
|
Raw History Blame
13.8 kB

SolidPrivacy Scrub — Workpackages

This file translates ROADMAP.md into executable workpackages.

Use:

  • PROJECT_PROMPT.md for full worker instructions and operating rules.
  • PROJECT_PROMPT_SHORT.md for the compact ChatGPT Project Instructions version.
  • ROADMAP.md for product direction and phase order.
  • WORKPACKAGES.md for immediate execution planning and parallelization.
  • CHANGELOG.md for implementation history.

Mandatory worker start sequence

Every worker must start by reading, in order:

  1. PROJECT_PROMPT.md
  2. ROADMAP.md
  3. WORKPACKAGES.md
  4. CHANGELOG.md

If the active repository is not solidprivacy-nl/scrub, stop and report the mismatch.

Every worker must end with a handover summary and write that summary to:

handover/workpackages/

Filename format:

handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md

Current execution principle

Avoid parallel edits to the same Streamlit UI patch area.

Parallel work is safe for:

  • pure helper modules;
  • tests;
  • specifications;
  • documentation;
  • non-UI architecture work.

Parallel work is risky for:

  • presidio_streamlit.py;
  • fix_streamlit_nested_expanders.py;
  • export/download UI blocks;
  • shared replacement table flow.

UI integration should therefore happen sequentially.

Completed prerequisite

WP0 — v12.3 stabilization check

Status: completed by user verification.

Evidence:

  • GitHub Actions tests green.
  • GitHub to Hugging Face sync green.
  • App reloaded successfully.
  • pandas Index truth-value error gone.
  • Simplified review table working.
  • Technical details available in separate expander.

Completed UI workpackages

WP1 — v12.4 Review guidance text

Status: completed.

Outcome:

  • Review workflow guidance is visible.
  • Export semantics were not changed.

WP2 — v12.5 Final review summary

Status: completed and formally closed after verification.

Implemented files:

  • review_summary.py
  • tests/test_review_summary.py
  • tests/test_review_summary_ui_patch.py
  • fix_streamlit_nested_expanders.py

Outcome:

  • The app shows Eindcontrole vóór download before downloads.
  • The summary is advisory and does not change export/download semantics.
  • v12.5 is complete.

WP3 — v12.6 Export sanity checks

Status: completed and formally closed.

Implemented files:

  • export_sanity.py
  • tests/test_export_sanity.py
  • tests/test_export_sanity_ui_patch.py
  • fix_streamlit_nested_expanders.py

Outcome:

  • The app shows advisory Extra exportcontrole before downloads.
  • Downloads are not blocked.
  • TXT, CSV, DOCX and PDF export behavior is not changed.
  • v12 Review UX line is complete from WP1 through WP3.

Completed strategic / helper workpackages

WP4 — v13.0 Scrub Key specification and pure model

Status: completed.

Implemented files:

  • SCRUB_KEY_SPEC.md
  • scrub_key.py
  • tests/test_scrub_key.py

Outcome:

  • v13.0 model/spec is complete.

WP4B / WP4B-FIX — v13.1 Scrub Key JSON export UI and mapping hotfix

Status: completed and app-verified.

Implemented files:

  • fix_streamlit_nested_expanders.py
  • tests/test_scrub_key_ui_patch.py

Outcome:

  • v13.1 Scrub Key JSON export is complete and app-verified.

WP5 — v13.2 Scrub Key import/reload helper and tests

Status: completed.

Implemented files:

  • scrub_key_import.py
  • tests/test_scrub_key_import.py

Outcome:

  • v13.2 helper layer was ready for UI integration and is now part of the completed v13.2 import/reload flow.

WP6 — v13.2 Scrub Key import/reload UI integration

Status: completed and app-verified.

Implemented files:

  • fix_streamlit_nested_expanders.py
  • tests/test_scrub_key_import_ui_patch.py

Outcome:

  • v13.2 Scrub Key import/reload UI is completed, app-verified and closed.

WP7A / WP7B / WP7B-FINAL — v13.3 Deterministic reinsert helper

Status: completed and formally closed after Actions/sync verification.

Implemented files:

  • scrub_key_reinsert.py
  • tests/test_scrub_key_reinsert.py

Outcome:

  • Added deterministic local reinsert_from_scrub_key(text, scrub_key).
  • Helper reports audit fields and no-AI/no-cloud status.
  • v13.3 deterministic reinsert helper is completed and formally closed.

WP8 — v13.3 Deterministic reinsert UI planning

Status: implemented; reinsert UI implementation completed in WP8B and app-verified in WP8C.

Implemented files:

  • REINSERT_UI_SPEC.md
  • WORKPACKAGES.md
  • CHANGELOG.md
  • handover/workpackages/20260607_1900_v13_3_reinsert_ui_planning.md

Outcome:

  • v13.3 deterministic reinsert UI was planned before implementation.

WP8B / WP8C — v13.3 Deterministic reinsert UI implementation and app verification closeout

Status: completed and app-verified after Actions/sync verification.

Implemented files:

  • fix_streamlit_nested_expanders.py
  • tests/test_scrub_key_reinsert_ui_patch.py
  • tests/test_scrub_key_ui_patch.py
  • WORKPACKAGES.md
  • CHANGELOG.md
  • handover/workpackages/20260607_1915_v13_3_reinsert_ui_implementation.md
  • handover/workpackages/20260607_1930_v13_3_reinsert_ui_app_closeout.md

Outcome:

  • v13.3 deterministic local reinsert UI is completed, app-verified and formally closed.
  • No AI calls, cloud processing, automatic document rehydration or DOCX/PDF reinsert were added.
  • Existing scrubbed export/download behavior was not changed.

WP9 — AI-output / document reinsert workflow UX and architecture review

Status: completed; review-only workpackage.

Added files:

  • AI_OUTPUT_REINSERT_WORKFLOW_REVIEW.md
  • handover/workpackages/20260608_0000_ai_output_reinsert_workflow_review.md

Changed files:

  • WORKPACKAGES.md
  • CHANGELOG.md

Outcome:

  • Pasted-text reinsert remains a safe baseline and fallback, but is not sufficient as the final legal-document workflow.
  • Recommended UX direction is a two-mode interface:
    • Anonimiseren;
    • Originele waarden terugzetten.
  • Recommended implementation sequence is phased:
    • keep pasted text;
    • add TXT upload/download reinsert;
    • add DOCX reinsert helper and tests;
    • add DOCX reinsert UI;
    • investigate PDF text extraction only;
    • consider PDF output only after reliability review.
  • Recommended architecture remains local-only, deterministic and helper-first.

Validation:

  • Tests: not applicable; planning/review-only workpackage.
  • App verification: not applicable; no UI behavior changed.

WP10 — v13.4 TXT/DOCX reinsert foundation helper and tests

Status: implemented; awaiting coordinator verification of Actions/sync.

Latest implementation commit checked by WP10B:

eb0c1ed2397ec1a4dc256d6e7e615ac4c026c0ee

Added files:

  • scrub_key_document_reinsert.py
  • tests/test_scrub_key_document_reinsert.py
  • handover/workpackages/20260608_0000_v13_4_txt_docx_reinsert_foundation.md

Changed files:

  • WORKPACKAGES.md
  • CHANGELOG.md

Implemented helper behavior:

  • Added reinsert_text_document(text, scrub_key) for plain text document-level reinsert.
  • Added reinsert_txt_bytes(content, scrub_key, encoding="utf-8") for TXT bytes input/output.
  • Added reinsert_docx_bytes(content, scrub_key) for DOCX main-document text-node reinsert.
  • Reuses existing deterministic reinsert_from_scrub_key(...) logic.
  • Returns restored text / restored bytes plus audit summary fields.
  • Reports document_type, local_only=True, ai_processing=False and cloud_processing=False.
  • Keeps TXT/DOCX reinsert pure and side-effect free.
  • Does not mutate the input Scrub Key.

DOCX foundation limitations:

  • Processes only word/document.xml text nodes.
  • Supports normal body paragraphs and tables in word/document.xml.
  • Does not restore placeholders split across multiple Word runs/text nodes.
  • Does not process headers, footers, comments, tracked changes or metadata.
  • Does not claim perfect formatting preservation.

Validation status:

  • Local reconstructed targeted validation:
    • PYTHONPATH=. pytest -q tests/test_scrub_key.py → 6 passed.
    • PYTHONPATH=. pytest -q tests/test_scrub_key_reinsert.py → 12 passed.
    • PYTHONPATH=. pytest -q tests/test_scrub_key_document_reinsert.py → 14 passed.
  • Local reconstructed full available subset:
    • PYTHONPATH=. pytest -q → 32 passed.
  • Repository clone via container was not possible because outbound GitHub DNS was unavailable, so validation was performed on reconstructed files from GitHub-fetched content plus the new helper/tests.

WP10B verification / closeout status:

  • Commit metadata visible for eb0c1ed2397ec1a4dc256d6e7e615ac4c026c0ee.
  • GitHub combined commit status returned an empty status list.
  • GitHub workflow-runs query for the commit returned no visible workflow runs.
  • GitHub Actions: not visible through connector.
  • Hugging Face sync: not visible through connector.
  • App verification: not applicable; helper-only package and no UI behavior changed.

Boundaries preserved:

  • No UI files changed.
  • No edit to fix_streamlit_nested_expanders.py.
  • No edit to presidio_streamlit.py.
  • No PDF reinsert implementation added.
  • No AI calls added.
  • No cloud processing added.
  • No automatic app document rehydration added.
  • Existing TXT, CSV, DOCX and PDF scrubbed export/download behavior was not changed.
  • Scrub Key JSON export/import behavior was not changed.
  • Synthetic test data only.

WP10B — v13.4 TXT/DOCX reinsert foundation verification and closeout

Status: completed; Actions/sync not visible through connector, coordinator verification required.

Added files:

  • handover/workpackages/20260608_0000_v13_4_txt_docx_reinsert_foundation_closeout.md

Changed files:

  • WORKPACKAGES.md
  • CHANGELOG.md

Outcome:

  • WP10 closeout was performed without changing code.
  • Actions/sync could not be verified through connector-visible status data.
  • WP10 remains implemented and awaits coordinator verification of Actions/sync.
  • App verification is not applicable because WP10 added helper/test logic only and no UI behavior.

WP11 — v13.5 Two-mode reinsert UI planning

Status: completed; planning/specification-only workpackage.

Added files:

  • TWO_MODE_UI_SPEC.md
  • handover/workpackages/20260608_0000_v13_5_two_mode_ui_planning.md

Changed files:

  • WORKPACKAGES.md
  • CHANGELOG.md

Outcome:

  • Recommended moving Scrub to a two-mode interface:
    • Anonimiseren;
    • Originele waarden terugzetten.
  • Compared the current single-scroll workflow, Streamlit tabs, and a landing choice with two large cards/buttons.
  • Recommended Streamlit tabs or two clear mode panels as the first implementation step.
  • Recommended landing cards as the longer-term mature product direction, not the next patch-based step.
  • Specified the ideal user journey for Anonimiseren.
  • Specified the ideal user journey for Originele waarden terugzetten.
  • Planned TXT/DOCX reinsert UI phasing:
    • keep pasted text available;
    • add TXT upload/download UI after the mode skeleton is verified;
    • add DOCX upload/download UI later using WP10 helper;
    • keep PDF reinsert excluded until separate reliability review.
  • Reconfirmed local-only design, no AI calls, no cloud processing, no silent overwrite of review rows, no Scrub Key storage, and no export/download semantics changes.

Validation:

  • Tests: not applicable; planning/specification-only workpackage.
  • App verification: not applicable; no UI behavior changed.

Boundaries preserved:

  • No UI code changed.
  • No edit to fix_streamlit_nested_expanders.py.
  • No edit to presidio_streamlit.py.
  • No edit to scrub_key_document_reinsert.py.
  • No edit to scrub_key_reinsert.py.
  • No edit to scrub_key.py.
  • No edit to scrub_key_import.py.
  • No edit to tests/*.
  • No TXT/DOCX reinsert UI added.
  • No PDF reinsert added.
  • No AI/cloud behavior added.
  • No export/download behavior changed.

Active / next recommended workpackage

WP12 — v13.6 Two-mode UI skeleton and tab separation

Status: recommended next implementation workpackage; not started here.

Goal:

  • Implement the first two-mode UI structure with minimal risk.
  • Use Streamlit tabs or two clear mode panels:
    • Anonimiseren;
    • Originele waarden terugzetten.
  • Keep the existing anonymization workflow working.
  • Keep pasted-text reinsert available.
  • Keep Scrub Key export/import behavior unchanged.
  • Keep existing scrubbed TXT/CSV/DOCX/PDF downloads unchanged.

Recommended scope:

  • UI skeleton/navigation only.
  • Move or present current anonymization workflow under Anonimiseren where feasible.
  • Move or present current Scrub Key load + pasted-text reinsert flow under Originele waarden terugzetten where feasible.
  • Add/adjust patch-level UI tests.
  • Ask for app verification because UI behavior changes.
  • Do not add TXT upload reinsert UI yet.
  • Do not add DOCX upload reinsert UI yet.
  • Do not add PDF reinsert.
  • Do not add AI/cloud behavior.
  • Do not change export/download semantics.

Recommended later workpackages:

WP13 — v13.7 TXT reinsert upload/download UI
WP14 — v13.8 DOCX reinsert upload/download UI
WP15 — PDF text extraction reliability review only

Recommended execution order

  1. Coordinator may still verify WP10 GitHub Actions and Hugging Face sync, because connector-visible status data was unavailable in WP10B.
  2. Start WP12 as the first UI implementation step.
  3. After WP12 is app-verified, implement TXT reinsert upload/download UI.
  4. After TXT UI is verified, implement DOCX reinsert upload/download UI.
  5. Keep PDF full reinsert out of scope until a separate reliability review.
  6. Keep AI/cloud behavior out unless explicitly approved.
  7. Preserve export/download and Scrub Key import/export semantics.