Upload folder using huggingface_hub (part 4)

.gitattributes

@@ -448,3 +448,20 @@ helpers/mcp_server.py.dox.md filter=lfs diff=lfs merge=lfs -text
 helpers/media_artifacts.py filter=lfs diff=lfs merge=lfs -text
 helpers/media_artifacts.py.dox.md filter=lfs diff=lfs merge=lfs -text
 helpers/message_queue.py filter=lfs diff=lfs merge=lfs -text
+helpers/message_queue.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/messages.py filter=lfs diff=lfs merge=lfs -text
+helpers/messages.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/microsoft_tunnel.py filter=lfs diff=lfs merge=lfs -text
+helpers/microsoft_tunnel.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/migration.py filter=lfs diff=lfs merge=lfs -text
+helpers/migration.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/modules.py filter=lfs diff=lfs merge=lfs -text
+helpers/modules.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/network.py filter=lfs diff=lfs merge=lfs -text
+helpers/network.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/notification.py filter=lfs diff=lfs merge=lfs -text
+helpers/notification.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/parallel_tools.py filter=lfs diff=lfs merge=lfs -text
+helpers/parallel_tools.py.dox.md filter=lfs diff=lfs merge=lfs -text
+helpers/performance.py filter=lfs diff=lfs merge=lfs -text
+helpers/performance.py.dox.md filter=lfs diff=lfs merge=lfs -text

helpers/message_queue.py.dox.md

@@ -1,53 +1,3 @@
-# message_queue.py DOX
-
-## Purpose
-
-- Own the `message_queue.py` helper module.
-- This module stores and drains queued user messages for a context.
-- Keep this file-level DOX profile synchronized with `message_queue.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `message_queue.py` owns the runtime implementation.
-- `message_queue.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Top-level functions:
-- `get_queue(context: 'AgentContext') -> list`: Get current queue from context.data.
-- `_get_next_seq(context: 'AgentContext') -> int`: Get next sequence number.
-- `_sync_output(context: 'AgentContext')`: Sync queue to output_data for frontend polling.
-- `add(context: 'AgentContext', text: str, attachments: list[str] | None=..., item_id: str | None=...) -> dict`: Add message to queue. Attachments should be filenames, will be converted to full paths.
-- `remove(context: 'AgentContext', item_id: str | None=...) -> int`: Remove item(s). If item_id is None, clears all. Returns remaining count.
-- `pop_first(context: 'AgentContext') -> dict | None`: Remove and return first item.
-- `pop_item(context: 'AgentContext', item_id: str) -> dict | None`: Remove and return specific item.
-- `has_queue(context: 'AgentContext') -> bool`: Check if queue has items.
-- `log_user_message(context: 'AgentContext', message: str, attachment_paths: list[str], message_id: str | None=..., source: str=...)`: Log user message to console and UI. Used by message API and queue processing.
-- `send_message(context: 'AgentContext', item: dict, source: str=...)`: Send a single queued message (log + communicate).
-- `send_next(context: 'AgentContext') -> bool`: Send next queued message. Returns True if sent, False if queue empty.
-- `send_all_aggregated(context: 'AgentContext') -> int`: Aggregate and send all queued messages as one. Returns count of items sent.
-- Notable constants/configuration names: `QUEUE_KEY`, `QUEUE_SEQ_KEY`, `UPLOAD_FOLDER`.
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Observed side-effect areas: filesystem reads, filesystem deletion, settings/state persistence.
-- Imported dependency areas include: `helpers`, `helpers.print_style`, `os`, `typing`, `uuid`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `context.set_data`, `get_queue`, `context.set_output_data`, `_sync_output`, `queue.pop`, `context.log.log`, `log_user_message`, `context.communicate`, `pop_first`, `has_queue`, `join`, `context.get_data`, `att.startswith`, `_get_next_seq`, `uuid.uuid4`, `UserMessage`, `send_message`, `guids.generate_id`, `os.path.basename`, `PrintStyle`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- No direct test reference was found by name search; choose the nearest behavioral test or perform a focused smoke check.
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:8872cd81aa6c11ea3961f8dc1f085f7518576404dbfb3ad8bc7de3d8dbb58130
+size 3680

helpers/messages.py

@@ -1,75 +1,3 @@
-# from . import files
-
-import json
-
-
-def truncate_text(agent, output, threshold=1000):
-    threshold = int(threshold)
-    if not threshold or len(output) <= threshold:
-        return output
-
-    # Adjust the file path as needed
-    placeholder = agent.read_prompt(
-        "fw.msg_truncated.md", length=(len(output) - threshold)
-    )
-    # placeholder = files.read_file("./prompts/default/fw.msg_truncated.md", length=(len(output) - threshold))
-
-    start_len = (threshold - len(placeholder)) // 2
-    end_len = threshold - len(placeholder) - start_len
-
-    truncated_output = output[:start_len] + placeholder + output[-end_len:]
-    return truncated_output
-
-
-def truncate_dict_by_ratio(agent, data: dict|list|str, threshold_chars: int, truncate_to: int):
-    threshold_chars = int(threshold_chars)
-    truncate_to = int(truncate_to)
-    
-    def process_item(item):
-        if isinstance(item, dict):
-            truncated_dict = {}
-            cumulative_size = 0
-
-            for key, value in item.items():
-                processed_value = process_item(value)
-                serialized_value = json.dumps(processed_value, ensure_ascii=False)
-                size = len(serialized_value)
-
-                if cumulative_size + size > threshold_chars:
-                    truncated_dict[key] = truncate_text(
-                        agent, serialized_value, truncate_to
-                    )
-                else:
-                    cumulative_size += size
-                    truncated_dict[key] = processed_value
-
-            return truncated_dict
-
-        elif isinstance(item, list):
-            truncated_list = []
-            cumulative_size = 0
-
-            for value in item:
-                processed_value = process_item(value)
-                serialized_value = json.dumps(processed_value, ensure_ascii=False)
-                size = len(serialized_value)
-
-                if cumulative_size + size > threshold_chars:
-                    truncated_list.append(
-                        truncate_text(agent, serialized_value, truncate_to)
-                    )
-                else:
-                    cumulative_size += size
-                    truncated_list.append(processed_value)
-
-            return truncated_list
-
-        elif isinstance(item, str):
-            if len(item) > threshold_chars:
-                return truncate_text(agent, item, truncate_to)
-            return item
-
-        else:
-            return item
-
-    return process_item(data)
+version https://git-lfs.github.com/spec/v1
+oid sha256:85b47588d6dc155be0101dd047a27d00292e513ad2c83369dfda1ea514ee439b
+size 2472

helpers/messages.py.dox.md

@@ -1,50 +1,3 @@
-# messages.py DOX
-
-## Purpose
-
-- Own the `messages.py` helper module.
-- This module truncates message payloads for display and storage safety.
-- Keep this file-level DOX profile synchronized with `messages.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `messages.py` owns the runtime implementation.
-- `messages.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Top-level functions:
-- `truncate_text(agent, output, threshold=...)`
-- `truncate_dict_by_ratio(agent, data: dict | list | str, threshold_chars: int, truncate_to: int)`
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Observed side-effect areas: filesystem reads, filesystem writes, settings/state persistence.
-- Imported dependency areas include: `json`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `agent.read_prompt`, `process_item`, `json.dumps`, `truncate_text`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- Related tests observed by source search:
-  - `tests/email_parser_test.py`
-  - `tests/test_browser_agent_regressions.py`
-  - `tests/test_chat_compaction.py`
-  - `tests/test_document_query_fallback.py`
-  - `tests/test_download_toast_regressions.py`
-  - `tests/test_fasta2a_client.py`
-  - `tests/test_mcp_handler_multimodal.py`
-  - `tests/test_oauth_codex.py`
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a37e1ed033fccfe1fe484143cb246cafb0278cbc01dafb88a5f7246ddce2db4
+size 2190

helpers/microsoft_tunnel.py

@@ -1,144 +1,3 @@
-import getpass
-import hashlib
-import os
-import socket
-
-from flaredantic import MicrosoftConfig, MicrosoftTunnel, NotifyEvent
-
-try:
-    from flaredantic.core.exceptions import MicrosoftTunnelError
-except Exception:  # pragma: no cover - keeps tests independent from package internals
-    MicrosoftTunnelError = RuntimeError
-
-from helpers import files
-from helpers.tunnel_common import FlaredanticTunnelHelper
-
-
-MICROSOFT_TUNNEL_ID_ENV_KEYS = (
-    "A0_MICROSOFT_DEV_TUNNEL_ID",
-    "MICROSOFT_DEV_TUNNEL_ID",
-)
-MICROSOFT_TUNNEL_TIMEOUT = 120
-
-
-def default_microsoft_tunnel_id():
-    for env_key in MICROSOFT_TUNNEL_ID_ENV_KEYS:
-        configured = (os.environ.get(env_key) or "").strip()
-        if configured:
-            return configured
-
-    seed = "|".join([
-        getpass.getuser(),
-        socket.gethostname(),
-        files.get_abs_path("usr"),
-    ])
-    digest = hashlib.sha256(seed.encode("utf-8")).hexdigest()[:10]
-    return f"agent-zero-{digest}"
-
-
-class AgentZeroMicrosoftTunnel(MicrosoftTunnel):
-    def notify(self, event, message, data=None):
-        try:
-            return super().notify(event, message, data)
-        except AttributeError:
-            self.agent_zero_notifications.append({
-                "event": event.value if hasattr(event, "value") else event,
-                "message": message,
-                "data": data,
-            })
-            return None
-
-    @property
-    def agent_zero_notifications(self):
-        if not hasattr(self, "_agent_zero_notifications"):
-            self._agent_zero_notifications = []
-        return self._agent_zero_notifications
-
-    def _notify_progress(self, message, data=None):
-        self.notify(NotifyEvent.INFO, message, data)
-
-    def _ensure_logged_in(self):
-        parent = getattr(super(), "_ensure_logged_in", None)
-        if callable(parent):
-            parent()
-        self._notify_progress(
-            "Microsoft Dev Tunnels login confirmed. Preparing your tunnel..."
-        )
-
-    def _ensure_tunnel(self):
-        tunnel_id = self.config.tunnel_id
-        port = str(self.config.port)
-
-        self._notify_progress(
-            f"Checking Microsoft Dev Tunnel `{tunnel_id}`...",
-            {"tunnel_id": tunnel_id},
-        )
-        show = self._run_cmd(["show", tunnel_id])
-        if show.returncode != 0:
-            self._notify_progress(
-                f"Creating Microsoft Dev Tunnel `{tunnel_id}`...",
-                {"tunnel_id": tunnel_id},
-            )
-            create = self._run_cmd(["create", tunnel_id])
-            if create.returncode != 0:
-                raise MicrosoftTunnelError(f"Failed to create tunnel: {create.stdout}")
-        else:
-            self._notify_progress(
-                f"Microsoft Dev Tunnel `{tunnel_id}` already exists. Checking port {port}...",
-                {"tunnel_id": tunnel_id, "port": port},
-            )
-
-        self._notify_progress(
-            f"Checking Microsoft Dev Tunnel port {port}...",
-            {"tunnel_id": tunnel_id, "port": port},
-        )
-        port_show = self._run_cmd(["port", "show", tunnel_id, "-p", port])
-        if port_show.returncode != 0:
-            self._notify_progress(
-                f"Creating Microsoft Dev Tunnel port {port}...",
-                {"tunnel_id": tunnel_id, "port": port},
-            )
-            port_create = self._run_cmd([
-                "port",
-                "create",
-                tunnel_id,
-                "-p",
-                port,
-                "--protocol",
-                "http",
-            ])
-            if port_create.returncode != 0:
-                raise MicrosoftTunnelError(
-                    f"Failed to create port: {port_create.stdout}"
-                )
-
-        self._notify_progress(
-            "Microsoft Dev Tunnel setup is ready. Starting the secure host..."
-        )
-
-
-class MicrosoftDevTunnel(FlaredanticTunnelHelper):
-    label = "Microsoft Dev Tunnels"
-
-    def build_tunnel(self):
-        config = MicrosoftConfig(
-            port=self.port,
-            verbose=True,
-            timeout=MICROSOFT_TUNNEL_TIMEOUT,
-            tunnel_id=default_microsoft_tunnel_id(),
-        )
-        return AgentZeroMicrosoftTunnel(config)
-
-    def start(self):
-        try:
-            return super().start()
-        except Exception as e:
-            if "Timeout waiting for Microsoft Dev Tunnels URL" not in str(e):
-                raise
-            tunnel_id = default_microsoft_tunnel_id()
-            raise RuntimeError(
-                "Microsoft Dev Tunnels did not return a URL. Agent Zero uses "
-                f"the tunnel id `{tunnel_id}` to avoid flaredantic's global "
-                "`flaredantic` tunnel-id collision. If this still fails, set "
-                "`A0_MICROSOFT_DEV_TUNNEL_ID` to a fresh unique value and try again."
-            ) from e
+version https://git-lfs.github.com/spec/v1
+oid sha256:7da436b3e30d6d203584c0b38311d2fb262dff93cf346eee7b53103984e6fa0d
+size 4824

helpers/microsoft_tunnel.py.dox.md

@@ -1,50 +1,3 @@
-# microsoft_tunnel.py DOX
-
-## Purpose
-
-- Own the `microsoft_tunnel.py` helper module.
-- This module implements Microsoft Dev Tunnel provider behavior.
-- Keep this file-level DOX profile synchronized with `microsoft_tunnel.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `microsoft_tunnel.py` owns the runtime implementation.
-- `microsoft_tunnel.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Classes:
-- `AgentZeroMicrosoftTunnel` (`MicrosoftTunnel`)
-  - `notify(self, event, message, data=...)`
-  - `agent_zero_notifications(self)`
-- `MicrosoftDevTunnel` (`FlaredanticTunnelHelper`)
-  - `build_tunnel(self)`
-  - `start(self)`
-- Top-level functions:
-- `default_microsoft_tunnel_id()`
-- Notable constants/configuration names: `MICROSOFT_TUNNEL_ID_ENV_KEYS`, `MICROSOFT_TUNNEL_TIMEOUT`.
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Observed side-effect areas: filesystem reads, network calls, settings/state persistence, tunnel state.
-- Imported dependency areas include: `flaredantic`, `getpass`, `hashlib`, `helpers`, `helpers.tunnel_common`, `os`, `socket`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `join`, `strip`, `hashlib.sha256.hexdigest`, `self.notify`, `callable`, `self._notify_progress`, `self._run_cmd`, `MicrosoftConfig`, `AgentZeroMicrosoftTunnel`, `getpass.getuser`, `socket.gethostname`, `files.get_abs_path`, `super.notify`, `parent`, `super.start`, `hashlib.sha256`, `MicrosoftTunnelError`, `default_microsoft_tunnel_id`, `RuntimeError`, `seed.encode`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- Related tests observed by source search:
-  - `tests/test_tunnel_remote_link.py`
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c9da2c11e435e32d7e35482a386bebc99892fed6bdf124e9b1ae52d25749acd
+size 2561

helpers/migration.py

@@ -1,139 +1,3 @@
-import os
-import json
-from helpers import files
-from helpers import subagents, extension
-from helpers import yaml as yaml_helper
-from helpers.print_style import PrintStyle
-
-
-def startup_migration() -> None:
-    migrate_user_data()
-    convert_agents_json_yaml()
-
-    extension.call_extensions_sync("startup_migration", None)
-
-
-
-def migrate_user_data() -> None:
-    """
-    Migrate user data from /tmp and other locations to /usr.
-    """
-
-    # --- Migrate Directories -------------------------------------------------------
-    # Move directories from tmp/ or other source locations to usr/
-    
-    _move_dir("tmp/chats", "usr/chats")
-    _move_dir("tmp/scheduler", "usr/scheduler", overwrite=True)
-    _move_dir("tmp/uploads", "usr/uploads")
-    _move_dir("tmp/upload", "usr/upload")
-    _move_dir("tmp/downloads", "usr/downloads")
-    _move_dir("tmp/email", "usr/email")
-    _move_dir("knowledge/custom", "usr/knowledge", overwrite=True)
-
-    # --- Migrate Files -------------------------------------------------------------
-    # Move specific configuration files to usr/
-    
-    _move_file("tmp/settings.json", "usr/settings.json")
-    _move_file("tmp/secrets.env", "usr/secrets.env")
-    _move_file(".env", "usr/.env", overwrite=True)
-
-    # --- Special Migration Cases ---------------------------------------------------
-    
-    # Migrate Memory
-    _migrate_memory()
-
-    # Flatten default directories (knowledge/default -> knowledge/, etc.)
-    # We use _merge_dir_contents because we want to move the *contents* of default/ 
-    # into the parent directory, not move the default directory itself.
-    _merge_dir_contents("knowledge/default", "knowledge")
-
-    # --- Cleanup -------------------------------------------------------------------
-    
-    # Remove obsolete directories after migration
-    _cleanup_obsolete()
-
-
-def convert_agents_json_yaml() -> None:
-    for root in subagents.get_agents_roots():
-        rel_root = files.deabsolute_path(root)
-        for subdir in files.get_subdirectories(rel_root):
-            agent_yaml = os.path.join(rel_root, subdir, "agent.yaml")
-            if files.exists(agent_yaml):
-                continue
-
-            agent_json = os.path.join(rel_root, subdir, "agent.json")
-            if not files.exists(agent_json):
-                continue
-
-            try:
-                agent_obj = json.loads(files.read_file(agent_json))
-                files.write_file(agent_yaml, yaml_helper.dumps(agent_obj))
-            except Exception as e:
-                PrintStyle.error(f"Failed to convert {agent_json} to YAML", e)
-                continue
-
-# --- Helper Functions ----------------------------------------------------------
-
-def _move_dir(src: str, dst: str, overwrite: bool = False) -> None:
-    """
-    Move a directory from src to dst if src exists and dst does not.
-    """
-    if files.exists(src) and (not files.exists(dst) or overwrite):
-        PrintStyle().print(f"Migrating {src} to {dst}...")
-        if overwrite and files.exists(dst):
-            files.delete_dir(dst)
-        files.move_dir(src, dst)
-
-def _move_file(src: str, dst: str, overwrite: bool = False) -> None:
-    """
-    Move a file from src to dst if src exists and dst does not.
-    """
-    if files.exists(src) and (not files.exists(dst) or overwrite):
-        PrintStyle().print(f"Migrating {src} to {dst}...")
-        files.move_file(src, dst)
-
-def _migrate_memory(base_path: str = "memory") -> None:
-    """
-    Migrate memory subdirectories.
-    """
-    subdirs = files.get_subdirectories(base_path)
-    for subdir in subdirs:
-        if subdir == "embeddings":
-            # Special case: Embeddings
-            _move_dir("memory/embeddings", "tmp/memory/embeddings")
-        else:
-            # Move other memory items to usr/memory
-            dst = f"usr/memory/{subdir}"
-            _move_dir(f"memory/{subdir}", dst)
-
-def _merge_dir_contents(src_parent: str, dst_parent: str) -> None:
-    """
-    Moves all items from src_parent to dst_parent.
-    Useful for flattening structures like 'knowledge/default/*' -> 'knowledge/*'.
-    """
-    if not files.exists(src_parent):
-        return
-
-    entries = files.list_files(src_parent)
-    for entry in entries:
-        src = f"{src_parent}/{entry}"
-        dst = f"{dst_parent}/{entry}"
-        abs_src = files.get_abs_path(src)
-        if os.path.isdir(abs_src):
-            _move_dir(src, dst)
-        elif os.path.isfile(abs_src):
-            _move_file(src, dst)
-
-def _cleanup_obsolete() -> None:
-    """
-    Remove directories that are no longer needed.
-    """
-    to_remove = [
-        "knowledge/default",
-        "memory",
-        "logs",
-    ]
-    for path in to_remove:
-        if files.exists(path):
-            PrintStyle().print(f"Removing {path}...")
-            files.delete_dir(path)
+version https://git-lfs.github.com/spec/v1
+oid sha256:32fe773b85f182cf48e2e27d272782460dfe5acb29bc67cad6da0d2c85905e89
+size 4798

helpers/migration.py.dox.md

@@ -1,54 +1,3 @@
-# migration.py DOX
-
-## Purpose
-
-- Own the `migration.py` helper module.
-- This module migrates legacy user data and runtime layout at startup.
-- Keep this file-level DOX profile synchronized with `migration.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `migration.py` owns the runtime implementation.
-- `migration.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Top-level functions:
-- `startup_migration() -> None`
-- `migrate_user_data() -> None`: Migrate user data from /tmp and other locations to /usr.
-- `convert_agents_json_yaml() -> None`
-- `_move_dir(src: str, dst: str, overwrite: bool=...) -> None`: Move a directory from src to dst if src exists and dst does not.
-- `_move_file(src: str, dst: str, overwrite: bool=...) -> None`: Move a file from src to dst if src exists and dst does not.
-- `_migrate_memory(base_path: str=...) -> None`: Migrate memory subdirectories.
-- `_merge_dir_contents(src_parent: str, dst_parent: str) -> None`: Moves all items from src_parent to dst_parent.
-- `_cleanup_obsolete() -> None`: Remove directories that are no longer needed.
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Startup cleanup removes obsolete `memory`, `knowledge/default`, and legacy `logs` directories; repeated runs are safe.
-- Observed side-effect areas: filesystem reads, filesystem writes, filesystem deletion, settings/state persistence, secret handling, scheduler state.
-- Imported dependency areas include: `helpers`, `helpers.print_style`, `json`, `os`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `migrate_user_data`, `convert_agents_json_yaml`, `extension.call_extensions_sync`, `_move_dir`, `_move_file`, `_migrate_memory`, `_merge_dir_contents`, `_cleanup_obsolete`, `subagents.get_agents_roots`, `files.get_subdirectories`, `files.list_files`, `files.deabsolute_path`, `files.exists`, `files.move_dir`, `files.move_file`, `files.get_abs_path`, `os.path.isdir`, `os.path.join`, `files.delete_dir`, `os.path.isfile`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- Related tests observed by source search:
-  - `tests/test_migration_cleanup.py`
-  - `tests/test_browser_agent_regressions.py`
-  - `tests/test_office_canvas_setup.py`
-  - `tests/test_office_document_store.py`
-  - `tests/test_speech_plugin_split.py`
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a111e1b05cb3490e7e6156ea4957ea7cf4a48565464eb61329724fec1e1094a
+size 3194

helpers/modules.py

@@ -1,97 +1,3 @@
-
-import re, os, importlib, importlib.util, inspect, sys
-from types import ModuleType
-from typing import Any, Type, TypeVar
-from helpers.files import get_abs_path
-from fnmatch import fnmatch
-
-
-T = TypeVar("T")  # Define a generic type variable
-
-
-def import_module(file_path: str) -> ModuleType:
-    # Handle file paths with periods in the name using importlib.util
-    abs_path = get_abs_path(file_path)
-    module_name = os.path.basename(abs_path).replace(".py", "")
-
-    # Create the module spec and load the module
-    spec = importlib.util.spec_from_file_location(module_name, abs_path)
-    if spec is None or spec.loader is None:
-        raise ImportError(f"Could not load module from {abs_path}")
-
-    module = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(module)
-    return module
-
-
-def load_classes_from_folder(
-    folder: str, name_pattern: str, base_class: Type[T], one_per_file: bool = True
-) -> list[Type[T]]:
-    classes = []
-    abs_folder = get_abs_path(folder)
-
-    # Get all .py files in the folder that match the pattern, sorted alphabetically
-    py_files = sorted(
-        [
-            file_name
-            for file_name in os.listdir(abs_folder)
-            if fnmatch(file_name, name_pattern) and file_name.endswith(".py")
-        ]
-    )
-
-    # Iterate through the sorted list of files
-    for file_name in py_files:
-        file_path = os.path.join(abs_folder, file_name)
-        # Use the new import_module function
-        module = import_module(file_path)
-
-        # Get all classes in the module
-        class_list = inspect.getmembers(module, inspect.isclass)
-
-        # Filter for classes that are subclasses of the given base_class
-        # iterate backwards to skip imported superclasses
-        for cls in reversed(class_list):
-            if cls[1] is not base_class and issubclass(cls[1], base_class):
-                classes.append(cls[1])
-                if one_per_file:
-                    break
-
-    return classes
-
-
-def load_classes_from_file(
-    file: str, base_class: type[T], one_per_file: bool = True
-) -> list[type[T]]:
-    classes = []
-    # Use the new import_module function
-    module = import_module(file)
-
-    # Get all classes in the module
-    class_list = inspect.getmembers(module, inspect.isclass)
-
-    # Filter for classes that are subclasses of the given base_class
-    # iterate backwards to skip imported superclasses
-    for cls in reversed(class_list):
-        if cls[1] is not base_class and issubclass(cls[1], base_class):
-            classes.append(cls[1])
-            if one_per_file:
-                break
-
-    return classes
-
-
-def purge_namespace(namespace: str):
-    to_delete = [
-        name
-        for name in sys.modules
-        if name == namespace or name.startswith(namespace + ".")
-    ]
-
-    # delete deepest first just to be tidy
-    to_delete.sort(key=lambda n: n.count("."), reverse=True)
-
-    for name in to_delete:
-        del sys.modules[name]
-
-    importlib.invalidate_caches()
-    return to_delete
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c70c8a4e53bde7172a8e5a3e38ae91c0bd3b086c013f0c7c622724f0b59e4cc
+size 3010

helpers/modules.py.dox.md

@@ -1,53 +1,3 @@
-# modules.py DOX
-
-## Purpose
-
-- Own the `modules.py` helper module.
-- This module imports modules and classes dynamically with namespace cache control.
-- Keep this file-level DOX profile synchronized with `modules.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `modules.py` owns the runtime implementation.
-- `modules.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Top-level functions:
-- `import_module(file_path: str) -> ModuleType`
-- `load_classes_from_folder(folder: str, name_pattern: str, base_class: Type[T], one_per_file: bool=...) -> list[Type[T]]`
-- `load_classes_from_file(file: str, base_class: type[T], one_per_file: bool=...) -> list[type[T]]`
-- `purge_namespace(namespace: str)`
-- Notable constants/configuration names: `T`.
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Observed side-effect areas: filesystem reads, filesystem deletion.
-- Imported dependency areas include: `fnmatch`, `helpers.files`, `importlib`, `importlib.util`, `inspect`, `os`, `re`, `sys`, `types`, `typing`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `TypeVar`, `get_abs_path`, `os.path.basename.replace`, `importlib.util.spec_from_file_location`, `importlib.util.module_from_spec`, `spec.loader.exec_module`, `import_module`, `inspect.getmembers`, `to_delete.sort`, `importlib.invalidate_caches`, `ImportError`, `os.path.join`, `os.path.basename`, `issubclass`, `os.listdir`, `name.startswith`, `n.count`, `fnmatch`, `file_name.endswith`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- Related tests observed by source search:
-  - `tests/test_a0_connector_prompt_gating.py`
-  - `tests/test_browser_agent_regressions.py`
-  - `tests/test_docker_release_plan.py`
-  - `tests/test_error_retry_plugin.py`
-  - `tests/test_file_tree_visualize.py`
-  - `tests/test_git_version_label.py`
-  - `tests/test_mcp_handler_multimodal.py`
-  - `tests/test_memory_quality.py`
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:10b2bb916631cb41b19aee53a407783a4d5650b2d33e07d73b780d4be61ebb10
+size 2809

helpers/network.py

@@ -1,202 +1,3 @@
-from __future__ import annotations
-
-from dataclasses import dataclass
-import ipaddress
-import os
-import socket
-import struct
-from urllib.parse import urljoin, urlparse
-
-import requests
-
-
-SAFE_HTTP_SCHEMES = frozenset({"http", "https"})
-DEFAULT_FETCH_TIMEOUT = (3.05, 10.0)
-DEFAULT_HTTP_USER_AGENT = "@mixedbread-ai/unstructured"
-
-
-@dataclass(frozen=True)
-class HttpFetchResult:
-    url: str
-    content: bytes
-    content_type: str | None
-    encoding: str | None
-
-
-class UnsafeUrlError(ValueError):
-    """Raised when a remote URL resolves to a non-public destination."""
-
-
-def _build_request_headers() -> dict[str, str]:
-    user_agent = (
-        os.getenv("USER_AGENT")
-        or os.getenv("user_agent")
-        or DEFAULT_HTTP_USER_AGENT
-    ).strip()
-    return {"User-Agent": user_agent or DEFAULT_HTTP_USER_AGENT}
-
-
-def _normalize_content_type(content_type: str | None) -> str | None:
-    if not content_type:
-        return None
-    return content_type.split(";", 1)[0].strip().lower() or None
-
-
-def resolve_host_ips(hostname: str) -> tuple[ipaddress._BaseAddress, ...]:
-    try:
-        results = socket.getaddrinfo(
-            hostname,
-            None,
-            family=socket.AF_UNSPEC,
-            type=socket.SOCK_STREAM,
-        )
-    except socket.gaierror as exc:
-        raise UnsafeUrlError(f"Unable to resolve hostname '{hostname}'") from exc
-
-    ips: list[ipaddress._BaseAddress] = []
-    seen: set[str] = set()
-    for _family, _type, _proto, _canonname, sockaddr in results:
-        address = sockaddr[0]
-        if "%" in address:
-            address = address.split("%", 1)[0]
-        ip = ipaddress.ip_address(address)
-        key = ip.compressed
-        if key in seen:
-            continue
-        seen.add(key)
-        ips.append(ip)
-
-    if not ips:
-        raise UnsafeUrlError(f"Hostname '{hostname}' did not resolve to an IP address")
-
-    return tuple(ips)
-
-
-def validate_public_http_url(url: str) -> tuple[ipaddress._BaseAddress, ...]:
-    parsed = urlparse(url)
-
-    if parsed.scheme not in SAFE_HTTP_SCHEMES:
-        raise UnsafeUrlError("Only http:// and https:// URLs are supported")
-    if not parsed.hostname:
-        raise UnsafeUrlError("URL hostname is required")
-    if parsed.username or parsed.password:
-        raise UnsafeUrlError("URLs with embedded credentials are not allowed")
-
-    hostname = parsed.hostname.rstrip(".").lower()
-    if hostname == "localhost" or hostname.endswith(".localhost"):
-        raise UnsafeUrlError(f"Blocked local hostname '{hostname}'")
-
-    ips = resolve_host_ips(hostname)
-    blocked = [str(ip) for ip in ips if not ip.is_global]
-    if blocked:
-        raise UnsafeUrlError(
-            f"Blocked non-public address resolution for '{hostname}': {', '.join(blocked)}"
-        )
-
-    return ips
-
-
-def fetch_public_http_resource(
-    url: str,
-    *,
-    max_bytes: int,
-    max_redirects: int = 5,
-    timeout: tuple[float, float] = DEFAULT_FETCH_TIMEOUT,
-) -> HttpFetchResult:
-    current_url = url
-    session = requests.Session()
-    session.trust_env = False
-
-    for redirect_count in range(max_redirects + 1):
-        validate_public_http_url(current_url)
-
-        try:
-            with session.get(
-                current_url,
-                stream=True,
-                allow_redirects=False,
-                headers=_build_request_headers(),
-                timeout=timeout,
-            ) as response:
-                if 300 <= response.status_code < 400:
-                    location = response.headers.get("Location")
-                    if not location:
-                        raise ValueError(
-                            f"Remote URL redirect is missing a Location header: {current_url}"
-                        )
-                    if redirect_count >= max_redirects:
-                        raise ValueError(
-                            f"Remote URL exceeded redirect limit ({max_redirects}): {url}"
-                        )
-                    current_url = urljoin(current_url, location)
-                    continue
-
-                if response.status_code >= 400:
-                    raise ValueError(
-                        f"Remote URL returned HTTP {response.status_code}: {current_url}"
-                    )
-
-                content_length = response.headers.get("Content-Length")
-                if content_length:
-                    try:
-                        declared_length = int(content_length)
-                    except ValueError:
-                        declared_length = None
-                    if declared_length is not None and declared_length > max_bytes:
-                        raise ValueError(
-                            f"Remote document exceeds max size {max_bytes} bytes: {current_url}"
-                        )
-
-                body = bytearray()
-                for chunk in response.iter_content(chunk_size=64 * 1024):
-                    if not chunk:
-                        continue
-                    body.extend(chunk)
-                    if len(body) > max_bytes:
-                        raise ValueError(
-                            f"Remote document exceeds max size {max_bytes} bytes: {current_url}"
-                        )
-
-                return HttpFetchResult(
-                    url=current_url,
-                    content=bytes(body),
-                    content_type=_normalize_content_type(
-                        response.headers.get("Content-Type")
-                    ),
-                    encoding=response.encoding,
-                )
-        except requests.RequestException as exc:
-            raise ValueError(
-                f"Remote document fetch failed for {current_url}: {exc}"
-            ) from exc
-
-    raise ValueError(f"Remote URL exceeded redirect limit ({max_redirects}): {url}")
-
-
-def is_loopback_address(address: str) -> bool:
-    """Check whether *address* resolves to a loopback interface."""
-    _checkers = {
-        socket.AF_INET: lambda x: (
-            struct.unpack("!I", socket.inet_aton(x))[0] >> (32 - 8)
-        ) == 127,
-        socket.AF_INET6: lambda x: x == "::1",
-    }
-    try:
-        socket.inet_pton(socket.AF_INET6, address)
-        return _checkers[socket.AF_INET6](address)
-    except socket.error:
-        pass
-    try:
-        socket.inet_pton(socket.AF_INET, address)
-        return _checkers[socket.AF_INET](address)
-    except socket.error:
-        pass
-    for family in (socket.AF_INET, socket.AF_INET6):
-        try:
-            r = socket.getaddrinfo(address, None, family, socket.SOCK_STREAM)
-        except socket.gaierror:
-            return False
-        for fam, _, _, _, sockaddr in r:
-            if not _checkers[fam](sockaddr[0]):
-                return False
-    return True
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6083e43f235ecd1470957f3fc33999150bd018bf2b4944a46dcb8a74474b0a1
+size 6715

helpers/network.py.dox.md

@@ -1,55 +1,3 @@
-# network.py DOX
-
-## Purpose
-
-- Own the `network.py` helper module.
-- This module validates public URLs and fetches remote resources safely.
-- Keep this file-level DOX profile synchronized with `network.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `network.py` owns the runtime implementation.
-- `network.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Classes:
-- `HttpFetchResult` (no explicit base class)
-- `UnsafeUrlError` (`ValueError`)
-- Top-level functions:
-- `_build_request_headers() -> dict[str, str]`
-- `_normalize_content_type(content_type: str | None) -> str | None`
-- `resolve_host_ips(hostname: str) -> tuple[ipaddress._BaseAddress, ...]`
-- `validate_public_http_url(url: str) -> tuple[ipaddress._BaseAddress, ...]`
-- `fetch_public_http_resource(url: str, max_bytes: int, max_redirects: int=..., timeout: tuple[float, float]=...) -> HttpFetchResult`
-- `is_loopback_address(address: str) -> bool`: Check whether *address* resolves to a loopback interface.
-- Notable constants/configuration names: `SAFE_HTTP_SCHEMES`, `DEFAULT_FETCH_TIMEOUT`, `DEFAULT_HTTP_USER_AGENT`.
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Observed side-effect areas: network calls, secret handling.
-- Imported dependency areas include: `__future__`, `dataclasses`, `ipaddress`, `os`, `requests`, `socket`, `struct`, `urllib.parse`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `frozenset`, `dataclass`, `strip`, `urlparse`, `parsed.hostname.rstrip.lower`, `resolve_host_ips`, `requests.Session`, `ValueError`, `content_type.split.strip.lower`, `socket.getaddrinfo`, `ipaddress.ip_address`, `seen.add`, `UnsafeUrlError`, `hostname.endswith`, `validate_public_http_url`, `socket.inet_pton`, `_checkers`, `parsed.hostname.rstrip`, `os.getenv`, `content_type.split.strip`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- Related tests observed by source search:
-  - `tests/test_oauth_gemini_api.py`
-  - `tests/test_oauth_github_copilot.py`
-  - `tests/test_oauth_xai_grok.py`
-  - `tests/test_plugin_scan_prompt.py`
-  - `tests/test_tunnel_remote_link.py`
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:10f8c25855e46830f82f4ddc219f032613274bd2d348fbd4a4faab36d4ca70c7
+size 3001

helpers/notification.py

@@ -1,250 +1,3 @@
-from dataclasses import dataclass
-import uuid
-import threading
-from datetime import datetime, timedelta
-from enum import Enum
-
-from helpers.localization import Localization
-
-
-class NotificationType(Enum):
-    INFO = "info"
-    SUCCESS = "success"
-    WARNING = "warning"
-    ERROR = "error"
-    PROGRESS = "progress"
-
-
-class NotificationPriority(Enum):
-    NORMAL = 10
-    HIGH = 20
-
-
-@dataclass
-class NotificationItem:
-    manager: "NotificationManager"
-    no: int
-    type: NotificationType
-    priority: NotificationPriority
-    title: str
-    message: str
-    detail: str  # HTML content for expandable details
-    timestamp: datetime
-    display_time: int = 3  # Display duration in seconds, default 3 seconds
-    read: bool = False
-    id: str = ""
-    group: str = ""  # Group identifier for grouping related notifications
-
-    def __post_init__(self):
-        if not self.id:
-            self.id = str(uuid.uuid4())
-        # Ensure type is always NotificationType
-        if isinstance(self.type, str):
-            self.type = NotificationType(self.type)
-
-    def mark_read(self):
-        self.read = True
-        self.manager.update_item(self.no, read=True)
-
-    def output(self):
-        return {
-            "no": self.no,
-            "id": self.id,
-            "type": self.type.value if isinstance(self.type, NotificationType) else self.type,
-            "priority": self.priority.value if isinstance(self.priority, NotificationPriority) else self.priority,
-            "title": self.title,
-            "message": self.message,
-            "detail": self.detail,
-            "timestamp": Localization.get().serialize_datetime(self.timestamp),
-            "display_time": self.display_time,
-            "read": self.read,
-            "group": self.group,
-        }
-
-
-class NotificationManager:
-    def __init__(self, max_notifications: int = 100):
-        self._lock = threading.RLock()
-        self.guid: str = str(uuid.uuid4())
-        self.updates: list[int] = []
-        self.notifications: list[NotificationItem] = []
-        self.max_notifications = max_notifications
-
-    @staticmethod
-    def send_notification(
-        type: NotificationType,
-        priority: NotificationPriority,
-        message: str,
-        title: str = "",
-        detail: str = "",
-        display_time: int = 3,
-        group: str = "",
-        id: str = "",
-    ) -> NotificationItem:
-        from agent import AgentContext
-        return AgentContext.get_notification_manager().add_notification(
-            type, priority, message, title, detail, display_time, group, id
-        )
-
-    def add_notification(
-        self,
-        type: NotificationType,
-        priority: NotificationPriority,
-        message: str,
-        title: str = "",
-        detail: str = "",
-        display_time: int = 3,
-        group: str = "",
-        id: str = "",
-    ) -> NotificationItem:
-        with self._lock:
-            existing = None
-            if id:
-                existing = next((n for n in self.notifications if n.id == id), None)
-
-            if existing:
-                existing.type = NotificationType(type)
-                existing.priority = NotificationPriority(priority)
-                existing.title = title
-                existing.message = message
-                existing.detail = detail
-                existing.timestamp = Localization.get().now()
-                existing.display_time = display_time
-                existing.group = group
-                existing.read = False
-                self.updates.append(existing.no)
-                item = existing
-            else:
-            # Create notification item
-                item = NotificationItem(
-                    manager=self,
-                    no=len(self.notifications),
-                    type=NotificationType(type),
-                    priority=NotificationPriority(priority),
-                    title=title,
-                    message=message,
-                    detail=detail,
-                    timestamp=Localization.get().now(),
-                    display_time=display_time,
-                    id=id,
-                    group=group,
-                )
-
-                self.notifications.append(item)
-                self.updates.append(item.no)
-                self._enforce_limit()
-
-        from helpers.state_monitor_integration import mark_dirty_all
-        mark_dirty_all(reason="notification.NotificationManager.add_notification")
-        return item
-
-    def _enforce_limit(self):
-        with self._lock:
-            if len(self.notifications) > self.max_notifications:
-                # Remove oldest notifications
-                to_remove = len(self.notifications) - self.max_notifications
-                self.notifications = self.notifications[to_remove:]
-                # Adjust notification numbers
-                for i, notification in enumerate(self.notifications):
-                    notification.no = i
-                # Adjust updates list
-                self.updates = [no - to_remove for no in self.updates if no >= to_remove]
-
-    def get_recent_notifications(self, seconds: int = 30) -> list[NotificationItem]:
-        cutoff = Localization.get().now() - timedelta(seconds=seconds)
-        with self._lock:
-            return [n for n in self.notifications if n.timestamp >= cutoff]
-
-    def output(self, start: int | None = None, end: int | None = None) -> list[dict]:
-        return self.output_with_state(start, end)[0]
-
-    def output_with_state(
-        self, start: int | None = None, end: int | None = None
-    ) -> tuple[list[dict], str, int]:
-        with self._lock:
-            if start is None:
-                start = 0
-            if end is None:
-                end = len(self.updates)
-            updates = self.updates[start:end]
-            out = []
-            seen = set()
-            for update in updates:
-                if update not in seen and update < len(self.notifications):
-                    out.append(self.notifications[update].output())
-                    seen.add(update)
-            return out, self.guid, len(self.updates)
-
-    def output_all(self) -> list[dict]:
-        with self._lock:
-            notifications = list(self.notifications)
-        return [n.output() for n in notifications]
-
-    def mark_read_by_ids(self, notification_ids: list[str]) -> int:
-        ids = {nid for nid in notification_ids if isinstance(nid, str) and nid.strip()}
-        if not ids:
-            return 0
-
-        changed_nos: list[int] = []
-        with self._lock:
-            for notification in self.notifications:
-                if notification.id in ids and not notification.read:
-                    notification.read = True
-                    changed_nos.append(notification.no)
-            if changed_nos:
-                self.updates.extend(changed_nos)
-
-        if not changed_nos:
-            return 0
-
-        from helpers.state_monitor_integration import mark_dirty_all
-        mark_dirty_all(reason="notification.NotificationManager.mark_read_by_ids")
-        return len(changed_nos)
-
-    def update_item(self, no: int, **kwargs) -> None:
-        self._update_item(no, **kwargs)
-
-    def _update_item(self, no: int, **kwargs):
-        changed = False
-        with self._lock:
-            if no < len(self.notifications):
-                item = self.notifications[no]
-                for key, value in kwargs.items():
-                    if hasattr(item, key):
-                        setattr(item, key, value)
-                self.updates.append(no)
-                changed = True
-
-        if not changed:
-            return
-
-        from helpers.state_monitor_integration import mark_dirty_all
-        mark_dirty_all(reason="notification.NotificationManager._update_item")
-
-    def mark_all_read(self):
-        changed_nos: list[int] = []
-        with self._lock:
-            for notification in self.notifications:
-                if not notification.read:
-                    notification.read = True
-                    changed_nos.append(notification.no)
-            if changed_nos:
-                self.updates.extend(changed_nos)
-
-        if not changed_nos:
-            return
-
-        from helpers.state_monitor_integration import mark_dirty_all
-        mark_dirty_all(reason="notification.NotificationManager.mark_all_read")
-
-    def clear_all(self):
-        with self._lock:
-            self.notifications = []
-            self.updates = []
-            self.guid = str(uuid.uuid4())
-        from helpers.state_monitor_integration import mark_dirty_all
-        mark_dirty_all(reason="notification.NotificationManager.clear_all")
-
-    def get_notifications_by_type(self, type: NotificationType) -> list[NotificationItem]:
-        with self._lock:
-            return [n for n in self.notifications if n.type == type]
+version https://git-lfs.github.com/spec/v1
+oid sha256:9807efbceeeb847f65df5b7896318718edeadc10fbc996f0ebc3b4557510e64c
+size 8793

helpers/notification.py.dox.md

@@ -1,64 +1,3 @@
-# notification.py DOX
-
-## Purpose
-
-- Own the `notification.py` helper module.
-- This module owns notification data models and notification manager state.
-- Keep this file-level DOX profile synchronized with `notification.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `notification.py` owns the runtime implementation.
-- `notification.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Classes:
-- `NotificationType` (`Enum`)
-- `NotificationPriority` (`Enum`)
-- `NotificationItem` (no explicit base class)
-  - `mark_read(self)`
-  - `output(self)`
-- `NotificationManager` (no explicit base class)
-  - `send_notification(type: NotificationType, priority: NotificationPriority, message: str, title: str=..., detail: str=..., display_time: int=..., group: str=..., id: str=...) -> NotificationItem`
-  - `add_notification(self, type: NotificationType, priority: NotificationPriority, message: str, title: str=..., detail: str=..., display_time: int=..., group: str=..., id: str=...) -> NotificationItem`
-  - `get_recent_notifications(self, seconds: int=...) -> list[NotificationItem]`
-  - `output(self, start: int | None=..., end: int | None=...) -> list[dict]`
-  - `output_with_state(self, start: int | None=..., end: int | None=...) -> tuple[list[dict], str, int]`
-  - `output_all(self) -> list[dict]`
-  - `mark_read_by_ids(self, notification_ids: list[str]) -> int`
-  - `update_item(self, no: int, **kwargs) -> None`
-  - `mark_all_read(self)`
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Notification payloads, GUIDs, and update cursors are captured under one lock so WebUI snapshots cannot skip notifications created during snapshot assembly.
-- Observed side-effect areas: filesystem deletion, settings/state persistence.
-- Imported dependency areas include: `dataclasses`, `datetime`, `enum`, `helpers.localization`, `threading`, `uuid`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `self.manager.update_item`, `threading.RLock`, `AgentContext.get_notification_manager.add_notification`, `mark_dirty_all`, `self._update_item`, `NotificationType`, `Localization.get.serialize_datetime`, `uuid.uuid4`, `Localization.get.now`, `timedelta`, `n.output`, `AgentContext.get_notification_manager`, `next`, `NotificationPriority`, `NotificationItem`, `self._enforce_limit`, `seen.add`, `notifications.output`, `nid.strip`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- Related tests observed by source search:
-  - `tests/test_download_toast_regressions.py`
-  - `tests/test_multi_tab_isolation.py`
-  - `tests/test_self_update_tag_filter.py`
-  - `tests/test_snapshot_parity.py`
-  - `tests/test_snapshot_schema_v1.py`
-  - `tests/test_state_monitor.py`
-  - `tests/test_state_sync_handler.py`
-  - `tests/test_state_sync_welcome_screen.py`
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:65e550506f83a76a4721a8bda8eca3821ef4c5c93294fde195fb5e0fb46cb544
+size 3684

helpers/parallel_tools.py

@@ -1,762 +1,3 @@
-from __future__ import annotations
-
-import asyncio
-import json
-import time
-import uuid
-from dataclasses import dataclass, field, replace
-from typing import Any, Literal, TYPE_CHECKING
-
-from helpers import extract_tools
-from helpers.defer import DeferredTask, THREAD_BACKGROUND
-from helpers.extension import call_extensions_async
-from helpers.print_style import PrintStyle
-
-if TYPE_CHECKING:
-    from agent import Agent, AgentConfig, AgentContext
-    from helpers.log import LogItem
-
-
-PARALLEL_JOBS_KEY = "_parallel_jobs"
-PARALLEL_WORKER_PARENT_CONTEXT_KEY = "_parallel_parent_context_id"
-PARALLEL_WORKER_JOB_KEY = "_parallel_job_id"
-PARALLEL_WORKER_KIND_KEY = "_parallel_worker_kind"
-
-CHILD_PARENT_CONTEXT_ID_KEY = "parent_context_id"
-CHILD_PARENT_AGENT_NUMBER_KEY = "parent_agent_number"
-CHILD_PARENT_CONTEXT_KIND_KEY = "parent_context_kind"
-CHILD_PARENT_CONTEXT_LABEL_KEY = "parent_context_label"
-CHILD_PARALLEL_JOB_ID_KEY = "parallel_job_id"
-CHILD_PARALLEL_TOOL_NAME_KEY = "parallel_tool_name"
-
-DEFAULT_MAX_CALLS = 8
-DEFAULT_TIMEOUT_SECONDS = 300
-POLL_INTERVAL_SECONDS = 0.5
-DISALLOWED_PARALLEL_TOOLS = {"document_query", "response"}
-
-TERMINAL_STATES = {"success", "error", "cancelled", "timeout"}
-JobState = Literal["pending", "running", "success", "error", "cancelled", "timeout"]
-JobKind = Literal["tool", "subordinate"]
-
-
-@dataclass
-class NormalizedToolCall:
-    index: int
-    tool_name: str
-    tool_args: dict[str, Any]
-
-
-@dataclass
-class ParallelJob:
-    id: str
-    parent_context_id: str
-    index: int
-    tool_name: str
-    tool_args: dict[str, Any]
-    kind: JobKind
-    parent_agent: "Agent | None" = field(default=None, repr=False)
-    state: JobState = "pending"
-    created_at: float = field(default_factory=time.time)
-    started_at: float | None = None
-    completed_at: float | None = None
-    result: str | None = None
-    error: str | None = None
-    worker_context_id: str | None = None
-    log_id: str = field(default_factory=lambda: str(uuid.uuid4()))
-    log_item: "LogItem | None" = field(default=None, repr=False)
-    deferred_task: DeferredTask | None = field(default=None, repr=False)
-    parent_history: list[tuple[Any, int]] = field(default_factory=list, repr=False)
-
-    def elapsed(self) -> float:
-        end = self.completed_at or time.time()
-        start = self.started_at or self.created_at
-        return max(0.0, end - start)
-
-
-def extract_tool_calls(args: dict[str, Any]) -> Any:
-    for key in ("tool_calls", "calls", "items"):
-        if key in args:
-            return args.get(key)
-    return None
-
-
-def normalize_parallel_tool_calls(raw_calls: Any) -> list[NormalizedToolCall]:
-    if isinstance(raw_calls, str):
-        try:
-            raw_calls = json.loads(raw_calls)
-        except json.JSONDecodeError as exc:
-            raise ValueError(
-                "`tool_calls` must be an array of normal tool-call objects."
-            ) from exc
-    if not isinstance(raw_calls, list):
-        raise ValueError("`tool_calls` must be an array of normal tool-call objects.")
-    if not raw_calls:
-        raise ValueError("`tool_calls` must contain at least one tool call.")
-    if len(raw_calls) > DEFAULT_MAX_CALLS:
-        raise ValueError(f"`tool_calls` supports at most {DEFAULT_MAX_CALLS} items.")
-
-    calls: list[NormalizedToolCall] = []
-    for index, raw_call in enumerate(raw_calls):
-        try:
-            tool_name, tool_args = extract_tools.normalize_tool_request(raw_call)
-        except ValueError as exc:
-            raise ValueError(f"tool_calls[{index}] is not a valid tool call: {exc}") from exc
-
-        if tool_name == "parallel":
-            raise ValueError("`parallel` cannot be nested inside another `parallel` call.")
-        if tool_name in DISALLOWED_PARALLEL_TOOLS:
-            raise ValueError(
-                f"`{tool_name}` cannot be used inside `parallel`; call it sequentially."
-            )
-
-        calls.append(
-            NormalizedToolCall(
-                index=index,
-                tool_name=tool_name,
-                tool_args=dict(tool_args),
-            )
-        )
-    return calls
-
-
-def normalize_job_ids(raw_job_ids: Any) -> list[str]:
-    if raw_job_ids is None:
-        return []
-    if isinstance(raw_job_ids, str):
-        return [raw_job_ids]
-    if isinstance(raw_job_ids, list):
-        return [str(item) for item in raw_job_ids if str(item).strip()]
-    raise ValueError("`job_ids` must be a string or an array of strings.")
-
-
-def coerce_bool(value: Any, default: bool) -> bool:
-    if value is None:
-        return default
-    if isinstance(value, bool):
-        return value
-    if isinstance(value, str):
-        normalized = value.strip().lower()
-        if normalized in {"1", "true", "yes", "on"}:
-            return True
-        if normalized in {"0", "false", "no", "off"}:
-            return False
-    return bool(value)
-
-
-def coerce_timeout(value: Any) -> int:
-    if value in (None, ""):
-        return DEFAULT_TIMEOUT_SECONDS
-    try:
-        timeout = int(value)
-    except (TypeError, ValueError) as exc:
-        raise ValueError(f"`timeout` must be an integer number of seconds, got {value!r}.") from exc
-    if timeout <= 0:
-        raise ValueError("`timeout` must be greater than 0.")
-    return timeout
-
-
-def _parallel_worker_kind(agent: "Agent | None") -> JobKind | None:
-    context = getattr(agent, "context", None)
-    if not context:
-        return None
-    kind = context.get_data(PARALLEL_WORKER_KIND_KEY)
-    if kind in {"tool", "subordinate"}:
-        return kind
-    if context.get_data(PARALLEL_WORKER_JOB_KEY):
-        return "tool"
-    return None
-
-
-def is_parallel_worker(agent: "Agent | None") -> bool:
-    return _parallel_worker_kind(agent) == "tool"
-
-
-def queue_parallel_parent_history(
-    agent: "Agent",
-    *,
-    content: Any,
-    tokens: int = 0,
-) -> bool:
-    if not is_parallel_worker(agent):
-        return False
-    context = agent.context
-    parent_context_id = str(
-        context.get_data(PARALLEL_WORKER_PARENT_CONTEXT_KEY) or ""
-    )
-    job_id = str(context.get_data(PARALLEL_WORKER_JOB_KEY) or "")
-    job = _get_job(parent_context_id, job_id)
-    if not job or job.kind != "tool":
-        return False
-    job.parent_history.append((content, tokens))
-    return True
-
-
-def _jobs_for_context(context: "AgentContext") -> dict[str, ParallelJob]:
-    jobs = context.get_data(PARALLEL_JOBS_KEY)
-    if not isinstance(jobs, dict):
-        jobs = {}
-        context.set_data(PARALLEL_JOBS_KEY, jobs)
-    return jobs
-
-
-def _get_job(parent_context_id: str, job_id: str) -> ParallelJob | None:
-    from agent import AgentContext
-
-    context = AgentContext.get(parent_context_id)
-    if not context:
-        return None
-    job = _jobs_for_context(context).get(job_id)
-    return job if isinstance(job, ParallelJob) else None
-
-
-def _new_job_id(tool_name: str) -> str:
-    prefix = "".join(ch for ch in tool_name if ch.isalnum())[:12] or "job"
-    return f"{prefix}-{uuid.uuid4().hex[:8]}"
-
-
-async def start_parallel_jobs(
-    agent: "Agent",
-    calls: list[NormalizedToolCall],
-) -> list[ParallelJob]:
-    jobs: list[ParallelJob] = []
-    context = agent.context
-    job_store = _jobs_for_context(context)
-
-    for call in calls:
-        kind: JobKind = "subordinate" if call.tool_name == "call_subordinate" else "tool"
-        job = ParallelJob(
-            id=_new_job_id(call.tool_name),
-            parent_context_id=context.id,
-            index=call.index,
-            tool_name=call.tool_name,
-            tool_args=call.tool_args,
-            kind=kind,
-            parent_agent=agent,
-        )
-        job_store[job.id] = job
-        jobs.append(job)
-        _log_parallel_child_started(agent, job)
-
-        try:
-            job.state = "running"
-            job.started_at = time.time()
-            task = DeferredTask(thread_name=THREAD_BACKGROUND)
-            job.deferred_task = task
-            if _parallel_worker_kind(agent) == "subordinate" and context.task:
-                context.task.add_child_task(task)
-            task.start_task(_run_parallel_job, context.id, job.id)
-        except Exception as exc:
-            _finish_job(job, "error", error=str(exc))
-
-    return jobs
-
-
-async def await_parallel_jobs(
-    agent: "Agent",
-    job_ids: list[str],
-    timeout: int = DEFAULT_TIMEOUT_SECONDS,
-    *,
-    collect: bool = True,
-    wait: bool = True,
-) -> list[dict[str, Any]]:
-    if not job_ids:
-        raise ValueError("No `job_ids` were provided to await.")
-
-    deadline = time.time() + timeout
-    wait_timed_out_job_ids: set[str] = set()
-    while True:
-        await refresh_parallel_jobs(agent)
-        jobs = [_jobs_for_context(agent.context).get(job_id) for job_id in job_ids]
-        missing = [job_id for job_id, job in zip(job_ids, jobs) if job is None]
-        if missing:
-            raise ValueError(f"Unknown parallel job id(s): {', '.join(missing)}")
-
-        active = [job for job in jobs if job and job.state not in TERMINAL_STATES]
-        if not wait or not active:
-            break
-
-        if time.time() >= deadline:
-            wait_timed_out_job_ids = {job.id for job in active}
-            break
-
-        await asyncio.sleep(POLL_INTERVAL_SECONDS)
-
-    snapshots = []
-    for job_id in job_ids:
-        job = _jobs_for_context(agent.context).get(job_id)
-        if job:
-            snapshot = _job_snapshot(job, include_result=True)
-            if job.id in wait_timed_out_job_ids and job.state not in TERMINAL_STATES:
-                snapshot["wait_timed_out"] = True
-            snapshots.append(snapshot)
-
-    if collect:
-        await collect_parallel_jobs(agent, job_ids)
-
-    return snapshots
-
-
-async def cancel_parallel_jobs(agent: "Agent", job_ids: list[str]) -> list[dict[str, Any]]:
-    if not job_ids:
-        raise ValueError("No `job_ids` were provided to cancel.")
-
-    await refresh_parallel_jobs(agent)
-    snapshots = []
-    for job_id in job_ids:
-        job = _jobs_for_context(agent.context).get(job_id)
-        if not job:
-            raise ValueError(f"Unknown parallel job id: {job_id}")
-        await _cancel_job(job)
-        snapshots.append(_job_snapshot(job, include_result=True))
-        await cleanup_parallel_job(agent, job)
-        _jobs_for_context(agent.context).pop(job_id, None)
-    return snapshots
-
-
-async def refresh_parallel_jobs(agent: "Agent") -> list[ParallelJob]:
-    jobs = list(_jobs_for_context(agent.context).values())
-    for job in jobs:
-        if job.state in TERMINAL_STATES:
-            continue
-        task = job.deferred_task
-        if not task:
-            continue
-        if task.is_ready():
-            try:
-                await task.result()
-            except asyncio.CancelledError:
-                _finish_job(job, "cancelled", error="Parallel job was cancelled.")
-            except Exception as exc:
-                _finish_job(job, "error", error=str(exc))
-        elif task.is_alive():
-            job.state = "running"
-            if job.started_at is None:
-                job.started_at = time.time()
-    return jobs
-
-
-async def cleanup_parallel_job(agent: "Agent", job: ParallelJob) -> None:
-    if job.deferred_task and job.deferred_task.is_alive():
-        job.deferred_task.kill()
-    if job.kind == "tool":
-        await _remove_context(job.worker_context_id)
-
-
-async def collect_parallel_jobs(
-    agent: "Agent",
-    job_ids: list[str],
-    *,
-    promote_parent_history: bool = False,
-) -> None:
-    jobs = _jobs_for_context(agent.context)
-    for job_id in dict.fromkeys(job_ids):
-        job = jobs.get(job_id)
-        if not job or job.state not in TERMINAL_STATES:
-            continue
-        if promote_parent_history:
-            for content, tokens in job.parent_history:
-                agent.hist_add_message(False, content=content, tokens=tokens)
-            job.parent_history.clear()
-        await cleanup_parallel_job(agent, job)
-        jobs.pop(job_id, None)
-
-
-async def build_parallel_jobs_extras(agent: "Agent") -> str:
-    await refresh_parallel_jobs(agent)
-    jobs = [
-        job
-        for job in _jobs_for_context(agent.context).values()
-        if isinstance(job, ParallelJob) and job.state not in {"cancelled", "timeout"}
-    ]
-    if not jobs:
-        return ""
-
-    active = [job for job in jobs if job.state not in TERMINAL_STATES]
-    ready = [job for job in jobs if job.state in TERMINAL_STATES]
-    if not active and not ready:
-        return ""
-
-    lines = ["parallel jobs:"]
-    if active:
-        lines.append("running:")
-        for job in active:
-            lines.append(
-                f"- {job.id}: {job.tool_name} [{job.state}], running for {job.elapsed():.1f}s"
-            )
-    if ready:
-        lines.append("ready to collect with `parallel` and `job_ids`:")
-        for job in ready:
-            lines.append(
-                f"- {job.id}: {job.tool_name} [{job.state}], duration {job.elapsed():.1f}s"
-            )
-    lines.append("call the `parallel` tool with `job_ids` to await/collect results or `action: \"cancel\"` to cancel.")
-    return "\n".join(lines)
-
-
-def format_started_jobs(jobs: list[ParallelJob]) -> str:
-    payload = {
-        "status": "started",
-        "jobs": [_job_snapshot(job, include_result=False) for job in jobs],
-        "instruction": "Use the parallel tool with job_ids to await or cancel these background jobs.",
-    }
-    return json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
-
-
-def format_parallel_results(results: list[dict[str, Any]]) -> str:
-    states = [result.get("state") for result in results]
-    has_active_jobs = any(state not in TERMINAL_STATES for state in states)
-    wait_timed_out = any(result.get("wait_timed_out") for result in results)
-    if has_active_jobs:
-        status = "waiting" if wait_timed_out else "running"
-    elif states and all(state == "success" for state in states):
-        status = "success"
-    elif states and all(state == "cancelled" for state in states):
-        status = "cancelled"
-    elif any(state == "success" for state in states):
-        status = "partial"
-    else:
-        status = "error"
-
-    payload = {
-        "status": status,
-        "jobs": results,
-    }
-    if wait_timed_out:
-        payload["wait_timeout"] = True
-    if has_active_jobs:
-        payload["instruction"] = (
-            "Some jobs are still running. Call `parallel` with `action: \"await\"` "
-            "and the listed `job_ids` to wait again, or `action: \"cancel\"` to stop them."
-        )
-    return json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
-
-
-async def _run_parallel_job(parent_context_id: str, job_id: str) -> None:
-    job = _get_job(parent_context_id, job_id)
-    if not job:
-        return
-    try:
-        if job.kind == "subordinate":
-            result = await _run_subordinate_context_job(parent_context_id, job)
-        else:
-            result = await _run_direct_tool_job(parent_context_id, job)
-        _finish_job(job, "success", result=result)
-    except asyncio.CancelledError:
-        _finish_job(job, "cancelled", error="Parallel job was cancelled.")
-        raise
-    except Exception as exc:
-        _finish_job(job, "error", error=str(exc))
-        PrintStyle.error(f"Parallel job {job.id} failed: {exc}")
-
-
-async def _run_subordinate_context_job(parent_context_id: str, job: ParallelJob) -> str:
-    from agent import AgentContext
-    from helpers.tool_policy import ensure_tool_allowed
-    from tools.call_subordinate import get_or_create_subordinate, run_subordinate
-
-    parent_context = AgentContext.get(parent_context_id)
-    if not parent_context:
-        raise ValueError("Parent context not found.")
-    parent_agent = job.parent_agent or parent_context.agent0
-    ensure_tool_allowed(parent_agent, "call_subordinate")
-
-    args = job.tool_args
-    message = str(args.get("message") or "").strip()
-    if not message:
-        raise ValueError("call_subordinate requires `tool_args.message`.")
-
-    context_id = str(args.get("context_id") or args.get("agent_id") or "").strip()
-    reset = args.get("reset", False)
-    slot = (
-        job.id
-        if coerce_bool(reset, False) and not context_id
-        else "default"
-    )
-    attachments = args.get("attachments") if isinstance(args.get("attachments"), list) else []
-    subordinate = get_or_create_subordinate(
-        parent_agent,
-        profile=str(args.get("profile") or args.get("agent_profile") or ""),
-        reset=reset,
-        context_id=context_id,
-        name=str(args.get("name") or ""),
-        message=message,
-        slot=slot,
-    )
-    worker_context = subordinate.context
-    job.worker_context_id = worker_context.id
-    if job.deferred_task:
-        worker_context.task = job.deferred_task
-
-    worker_context.set_data(PARALLEL_WORKER_PARENT_CONTEXT_KEY, parent_context.id)
-    worker_context.set_data(PARALLEL_WORKER_JOB_KEY, job.id)
-    worker_context.set_data(PARALLEL_WORKER_KIND_KEY, job.kind)
-    worker_context.set_output_data(CHILD_PARALLEL_JOB_ID_KEY, job.id)
-    worker_context.set_output_data(CHILD_PARALLEL_TOOL_NAME_KEY, job.tool_name)
-    return await run_subordinate(parent_agent, subordinate, message, attachments)
-
-
-async def _run_direct_tool_job(parent_context_id: str, job: ParallelJob) -> str:
-    from agent import AgentContext, AgentContextType, LoopData
-
-    parent_context = AgentContext.get(parent_context_id)
-    if not parent_context:
-        raise ValueError("Parent context not found.")
-
-    worker_context: AgentContext | None = None
-    try:
-        worker_context = AgentContext(
-            config=_clone_config(parent_context.config),
-            name=f"parallel:{job.tool_name}",
-            type=AgentContextType.BACKGROUND,
-        )
-        worker_context.set_data(PARALLEL_WORKER_PARENT_CONTEXT_KEY, parent_context_id)
-        worker_context.set_data(PARALLEL_WORKER_JOB_KEY, job.id)
-        worker_context.set_data(PARALLEL_WORKER_KIND_KEY, job.kind)
-        worker_context.set_data(
-            "chat_model_override",
-            parent_context.get_data("chat_model_override"),
-        )
-        job.worker_context_id = worker_context.id
-        _copy_project(parent_context, worker_context)
-
-        worker_agent = worker_context.agent0
-        worker_agent.last_user_message = parent_context.agent0.last_user_message
-        worker_agent.loop_data = LoopData()
-        return await execute_tool_call(
-            worker_agent,
-            job.tool_name,
-            job.tool_args,
-            log_item=job.log_item,
-        )
-    finally:
-        if worker_context:
-            await _remove_context(worker_context.id)
-
-
-async def execute_tool_call(
-    agent: "Agent",
-    tool_name: str,
-    tool_args: dict[str, Any],
-    *,
-    log_item: "LogItem | None" = None,
-) -> str:
-    if tool_name == "parallel":
-        raise ValueError("`parallel` cannot be nested inside a parallel worker.")
-
-    tool = _resolve_parallel_tool(agent, tool_name, tool_args, strict=True)
-    if not tool:
-        raise ValueError(f"Tool '{tool_name}' not found or could not be initialized.")
-
-    original_get_log_object = None
-    if log_item is not None:
-        original_get_log_object = tool.get_log_object
-        tool.get_log_object = lambda: log_item
-
-    agent.loop_data.current_tool = tool
-    try:
-        await agent.handle_intervention()
-        await tool.before_execution(**tool_args)
-        await agent.handle_intervention()
-        await call_extensions_async(
-            "tool_execute_before",
-            agent,
-            tool_args=tool_args or {},
-            tool_name=tool_name,
-        )
-        response = await tool.execute(**tool_args)
-        await agent.handle_intervention()
-        await call_extensions_async(
-            "tool_execute_after",
-            agent,
-            response=response,
-            tool_name=tool_name,
-        )
-        await tool.after_execution(response)
-        await agent.handle_intervention()
-        return response.message
-    finally:
-        if original_get_log_object is not None:
-            tool.get_log_object = original_get_log_object
-        agent.loop_data.current_tool = None
-
-
-def _resolve_parallel_tool(
-    agent: "Agent",
-    tool_name: str,
-    tool_args: dict[str, Any],
-    *,
-    strict: bool = False,
-):
-    message = json.dumps({"tool_name": tool_name, "tool_args": tool_args})
-
-    tool = None
-    try:
-        import helpers.mcp_handler as mcp_helper
-
-        tool = mcp_helper.MCPConfig.get_instance().get_tool(agent, tool_name)
-    except ImportError:
-        tool = None
-    except Exception as exc:
-        if strict:
-            raise
-        PrintStyle.warning(f"Failed to initialize MCP tool '{tool_name}' for parallel job: {exc}")
-
-    if not tool:
-        get_tool = getattr(agent, "get_tool", None)
-        if not callable(get_tool):
-            if strict:
-                raise ValueError(f"Tool '{tool_name}' not found or could not be initialized.")
-            return None
-        try:
-            tool = get_tool(
-                name=tool_name,
-                method=None,
-                args=tool_args,
-                message=message,
-                loop_data=getattr(agent, "loop_data", None),
-            )
-        except Exception as exc:
-            if strict:
-                raise
-            PrintStyle.warning(f"Failed to initialize tool '{tool_name}' for parallel job: {exc}")
-            tool = None
-
-    if not tool:
-        return None
-
-    try:
-        tool.args = dict(tool_args)
-    except Exception:
-        pass
-
-    return tool
-
-
-async def _cancel_job(
-    job: ParallelJob,
-    *,
-    state: JobState = "cancelled",
-    message: str = "Parallel job was cancelled.",
-) -> None:
-    if job.deferred_task and job.deferred_task.is_alive():
-        job.deferred_task.kill()
-    _finish_job(job, state, error=message)
-
-
-def _finish_job(
-    job: ParallelJob,
-    state: JobState,
-    *,
-    result: str | None = None,
-    error: str | None = None,
-) -> None:
-    job.state = state
-    job.completed_at = time.time()
-    if job.started_at is None:
-        job.started_at = job.created_at
-    if result is not None:
-        job.result = result
-    if error is not None:
-        job.error = error
-    _update_parallel_child_log(job)
-
-
-async def _remove_context(context_id: str | None) -> None:
-    if not context_id:
-        return
-    from agent import AgentContext
-    from helpers import persist_chat
-
-    context = AgentContext.get(context_id)
-    if context:
-        try:
-            context.reset()
-        except Exception:
-            pass
-        AgentContext.remove(context_id)
-    persist_chat.remove_chat(context_id)
-
-
-def _log_parallel_child_started(agent: "Agent", job: ParallelJob) -> None:
-    if job.kind == "subordinate":
-        job.log_item = agent.context.log.log(
-            type="subagent",
-            heading=f"icon://communication {agent.agent_name}: Calling Subordinate Agent",
-            content="",
-            kvps=job.tool_args,
-            id=job.log_id,
-        )
-        return
-
-    tool = _resolve_parallel_tool(agent, job.tool_name, job.tool_args)
-    if tool is not None:
-        try:
-            job.log_item = tool.get_log_object()
-            if job.log_item is not None:
-                return
-        except Exception as exc:
-            PrintStyle.warning(
-                f"Failed to derive parallel child log for {job.tool_name}: {exc}"
-            )
-
-    heading = f"icon://construction {agent.agent_name}: Using tool '{job.tool_name}'"
-    job.log_item = agent.context.log.log(
-        type="tool",
-        heading=heading,
-        content="",
-        kvps=job.tool_args,
-        id=job.log_id,
-        _tool_name=job.tool_name,
-    )
-
-
-def _update_parallel_child_log(job: ParallelJob) -> None:
-    if not job.log_item:
-        return
-    if job.state == "success":
-        content = job.result if job.result else "(completed without textual output)"
-    else:
-        content = f"Error: {job.error or job.state}"
-    try:
-        job.log_item.update(content=content)
-    except Exception:
-        pass
-
-
-def _job_snapshot(job: ParallelJob, *, include_result: bool) -> dict[str, Any]:
-    data: dict[str, Any] = {
-        "job_id": job.id,
-        "tool_name": job.tool_name,
-        "state": job.state,
-        "duration_seconds": round(job.elapsed(), 3),
-    }
-    if job.worker_context_id:
-        data["context_id"] = job.worker_context_id
-    if include_result:
-        if job.result is not None:
-            data["result"] = job.result
-        if job.error is not None:
-            data["error"] = job.error
-    return data
-
-
-def _clone_config(config: "AgentConfig") -> "AgentConfig":
-    try:
-        return replace(
-            config,
-            knowledge_subdirs=list(config.knowledge_subdirs),
-            additional=dict(config.additional),
-        )
-    except Exception:
-        return config
-
-
-def _copy_project(parent_context: "AgentContext", worker_context: "AgentContext") -> None:
-    try:
-        from helpers import projects
-
-        project_name = projects.get_context_project_name(parent_context)
-        if project_name:
-            projects.activate_project(worker_context.id, project_name, mark_dirty=False)
-    except Exception:
-        pass
+version https://git-lfs.github.com/spec/v1
+oid sha256:69341a3cf61a8a259e140e06cdf13248d4a25b42c6a9cd9709debdf9d23f69ea
+size 25127

helpers/parallel_tools.py.dox.md

@@ -1,66 +1,3 @@
-# parallel_tools.py DOX
-
-## Purpose
-
-- Own the shared runtime for parallel tool-call jobs.
-- Normalize wrapped tool-call payloads, start background jobs, await or cancel jobs, and render prompt extras for active parallel work.
-- Keep this file-level DOX profile synchronized with `parallel_tools.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `parallel_tools.py` owns the runtime implementation.
-- `parallel_tools.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Public concepts:
-- `NormalizedToolCall`
-- `ParallelJob`
-- `start_parallel_jobs(...)`
-- `await_parallel_jobs(...)`
-- `cancel_parallel_jobs(...)`
-- `build_parallel_jobs_extras(...)`
-- `format_parallel_results(...)`
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Wrapped tool-call items must use the same shape as normal tool calls: a tool name plus arguments.
-- Normalization accepts full agent-reply-shaped objects when `tool_name` and `tool_args` are present; non-contract planning fields such as `thoughts` or `headline` are ignored.
-- `tool_calls` should be an array, but normalization also accepts a valid JSON string encoding of that array to recover provider/model stringification.
-- Normalization rejects `document_query` and `response` inside `parallel`: document parsing and Q&A must run sequentially, while `response` must remain top-level so it can end the message loop.
-- `call_subordinate` jobs first enforce the actual calling agent's delegation policy, then call the same creation and execution functions as direct delegation in `tools/call_subordinate.py`; this helper does not construct or prompt a second kind of subordinate.
-- Fresh parallel sibling calls create distinct `parent.number + 1` child agents. Their job snapshots expose stable `context_id` values that direct or parallel `reset=false` calls can continue after success or failure.
-- Jobs retain their actual parent agent so parallel calls made by A1 create A2 rather than falling back to a context's A0.
-- Subordinate child chats are tagged with job metadata, remain outside the scheduler task list, and may use normal child-chat tools including `parallel`.
-- Nested parallel jobs started by a parallel subordinate are registered as child `DeferredTask` instances so stopping the ancestor also stops its descendants.
-- Direct tool jobs run in isolated background contexts and are blocked from recursively invoking `parallel`.
-- Direct tool jobs inherit the parent's active per-chat model override and current user message.
-- Direct tool background context cleanup removes both the in-memory context and any transient chat folder left on disk.
-- Parent-visible child log items are created for each wrapped call so the WebUI can inspect concurrent children separately while the wrapper result remains model-history-only.
-- Child tool logs mirror normal tool-call visible args; job ids remain available through wrapper results and prompt extras rather than visible process-step args.
-- Wrapped tool child logs use each tool's native `get_log_object()` output when available, preserving special log rendering (for example: `code_execution_tool` uses `code_exe`, `wait` uses `progress`, MCP tools use `mcp`, and regular tools use `tool`).
-- Direct parallel worker execution reuses the parent-visible child log item so tool `before_execution()` cannot create a second generic worker log or lose the native badge type.
-- Direct tools may explicitly queue model-visible history for their parent. Terminal collection records the outer `parallel` result first, promotes queued messages in job order, and only then removes disposable worker state; background jobs retain queued history until they are collected.
-- Job IDs are stable handles for later await, collect, or cancel operations.
-- Prompt extras must stay bounded and expose only job IDs, tool names, status, and compact result/error summaries.
-
-## Key Concepts
-
-- The parent context stores in-flight jobs under a private data key; collected terminal jobs are removed from that registry.
-- `wait=True` starts jobs and awaits them before returning until all requested jobs finish or the wait timeout is reached; the timeout stops waiting but does not cancel running jobs.
-- `collect` returns already-finished job results without waiting; `await` waits for requested job IDs.
-- Canceled jobs should be marked terminal and should stop their background `DeferredTask` when cancellation is possible.
-- `queue_parallel_parent_history(...)` accepts messages only from registered direct tool workers. `collect_parallel_jobs(...)` optionally promotes those messages while collecting terminal jobs; arbitrary worker history is never copied.
-
-## Work Guidance
-
-- Keep normalization compatible with provider tool-call envelopes and direct JSON objects.
-- Avoid importing heavy runtime modules at import time unless startup behavior is verified.
-- Coordinate argument, output, or status changes with `tools/parallel.py`, prompt instructions, and tests.
-
-## Verification
-
-- Run targeted tests for normalization, recursion guard, prompt extras, and tool result formatting.
-- Run a live Agent Zero chat when changing parallel execution, child chat metadata, or subordinate task behavior.
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a103b1cbfb2e49767793916f631bee0f5d3923189250a3de2158229e906d21e
+size 5412

helpers/performance.py

@@ -1,51 +1,3 @@
-import functools
-import inspect
-from pyinstrument import Profiler
-
-def trace_performance(*, show_all=False, color=True, unicode=True):
-    """
-    Decorator that profiles a function and prints a call tree when it finishes.
-
-    Works with both synchronous and asynchronous functions.
-    """
-
-    def decorator(func):
-        is_coro = inspect.iscoroutinefunction(func)
-
-        @functools.wraps(func)
-        async def async_wrapper(*args, **kwargs):
-            profiler = Profiler()
-            profiler.start()
-            try:
-                return await func(*args, **kwargs)
-            finally:
-                profiler.stop()
-                print(f"\n=== Performance trace: {func.__module__}.{func.__qualname__} (async) ===")
-                print(
-                    profiler.output_text(
-                        color=color,
-                        unicode=unicode,
-                        show_all=show_all,
-                    )
-                )
-
-        @functools.wraps(func)
-        def sync_wrapper(*args, **kwargs):
-            profiler = Profiler()
-            profiler.start()
-            try:
-                return func(*args, **kwargs)
-            finally:
-                profiler.stop()
-                print(f"\n=== Performance trace: {func.__module__}.{func.__qualname__} ===")
-                print(
-                    profiler.output_text(
-                        color=color,
-                        unicode=unicode,
-                        show_all=show_all,
-                    )
-                )
-
-        return async_wrapper if is_coro else sync_wrapper
-
-    return decorator
+version https://git-lfs.github.com/spec/v1
+oid sha256:198b8cebbf85bdc0b7a6068a4727f14a2efae02c3475e73f069dd17b143f9c9d
+size 1615

helpers/performance.py.dox.md

@@ -1,42 +1,3 @@
-# performance.py DOX
-
-## Purpose
-
-- Own the `performance.py` helper module.
-- This module provides lightweight performance tracing helpers.
-- Keep this file-level DOX profile synchronized with `performance.py` because this directory is intentionally flat.
-
-## Ownership
-
-- `performance.py` owns the runtime implementation.
-- `performance.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation.
-- Top-level functions:
-- `trace_performance(show_all=..., color=..., unicode=...)`: Decorator that profiles a function and prints a call tree when it finishes.
-
-## Runtime Contracts
-
-- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
-- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
-- Imported dependency areas include: `functools`, `inspect`, `pyinstrument`.
-
-## Key Concepts
-
-- Important called helpers/classes observed in the source: `inspect.iscoroutinefunction`, `functools.wraps`, `Profiler`, `profiler.start`, `profiler.stop`, `func`, `profiler.output_text`.
-- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
-
-## Work Guidance
-
-- Preserve public helper APIs used by core code and plugins unless every caller is updated.
-- Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded.
-- Prefer adding cohesive helper functions here only when behavior is reused across modules.
-
-## Verification
-
-- Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers.
-- Related tests observed by source search:
-  - `tests/test_extensions_stress.py`
-  - `tests/test_ws_manager.py`
-
-## Child DOX Index
-
-No child DOX files.
+version https://git-lfs.github.com/spec/v1
+oid sha256:81afd1008693788b59b3d810ded2379dc682f5cee0d74a9bd30d7959774f8c15
+size 1937