amisima commited on
Commit
42bf450
·
verified ·
1 Parent(s): ea0d0c1

Upload 3 files

Browse files
Files changed (3) hide show
  1. README.md +11 -2
  2. app.py +9 -3
  3. lora_library.py +816 -0
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: MiniMax-H3 · reference → video + audio · Custom lora + CivitAI + scene continuation (fixed)
3
  emoji: 🎭
4
  colorFrom: pink
5
  colorTo: purple
@@ -7,7 +7,7 @@ sdk: gradio
7
  sdk_version: 6.20.0
8
  app_file: app.py
9
  pinned: true
10
- short_description: Video + soundtrack, your lora + CivitAI, structured prompts
11
  suggested_hardware: zero-a10g
12
  tags:
13
  - video
@@ -41,6 +41,13 @@ dialogue verbatim inside `<d>` tags, then `overall_soundscape` and `non_diegetic
41
  with the picture, so naming that someone speaks without giving the words produces correct mouth shapes with nothing
42
  in them; the builder makes that hard to get wrong.
43
 
 
 
 
 
 
 
 
44
  **Custom lora, five slots.** A Hugging Face repo, a file inside one, a **CivitAI download link**, or an uploaded
45
  file. Adapters have to be trained against the `transformer_ref/` partition — a `transformer/` adapter is a different
46
  partition and will not match.
@@ -156,6 +163,8 @@ the placement (once), the two reference encoders, the denoise loop and the two d
156
  | `H3_GPU_SIZE` | `xlarge` | ZeroGPU allocation size. `large` does not fit. |
157
  | `H3_LOKR_RANK` | `32` | Rank the LoKr converter targets. Raise it if the log reports a weak layer. |
158
  | `CIVITAI_TOKEN` | unset | Used for CivitAI downloads and search, and required for gated or adult entries. |
 
 
159
  | `CIVITAI_API_HOST` | unset | Pins the search host; otherwise `civitai.red` is asked first and `civitai.com` is the fallback. |
160
 
161
  ## Where diffusers comes from
 
1
  ---
2
+ title: MiniMax-H3 · reference → video + audio · Shared lora library + CivitAI + scene continuation
3
  emoji: 🎭
4
  colorFrom: pink
5
  colorTo: purple
 
7
  sdk_version: 6.20.0
8
  app_file: app.py
9
  pinned: true
10
+ short_description: Video + soundtrack, shared lora library, structured prompts
11
  suggested_hardware: zero-a10g
12
  tags:
13
  - video
 
41
  with the picture, so naming that someone speaks without giving the words produces correct mouth shapes with nothing
42
  in them; the builder makes that hard to get wrong.
43
 
44
+ **A shared lora library that survives a restart.** Every adapter anyone adds is kept in a storage bucket — name,
45
+ link, trigger words and strength — in two lists, ordinary and nsfw. Tick what you want and press one button: the
46
+ slots are filled, their strengths are set, and the trigger words are appended to the prompt, so an adapter that needs
47
+ its token to do anything never silently does nothing. Paste a CivitAI or Hugging Face link and the title and the
48
+ trigger words are read off it for you. The list is common to everyone who opens the Space and is still there after it
49
+ sleeps and wakes, so nobody has to hunt down last week's link a second time.
50
+
51
  **Custom lora, five slots.** A Hugging Face repo, a file inside one, a **CivitAI download link**, or an uploaded
52
  file. Adapters have to be trained against the `transformer_ref/` partition — a `transformer/` adapter is a different
53
  partition and will not match.
 
163
  | `H3_GPU_SIZE` | `xlarge` | ZeroGPU allocation size. `large` does not fit. |
164
  | `H3_LOKR_RANK` | `32` | Rank the LoKr converter targets. Raise it if the log reports a weak layer. |
165
  | `CIVITAI_TOKEN` | unset | Used for CivitAI downloads and search, and required for gated or adult entries. |
166
+ | `LORA_LIBRARY_BUCKET` | `amisima/minimax-h3-reference-4-step-lora-storage` | The storage bucket the shared lora library is kept in. Attach it to the Space under *Settings → Storage Buckets* and no token is needed; otherwise an `HF_TOKEN` with write access is. |
167
+ | `LORA_LIBRARY_DIR` | unset | Overrides where the library file is read and written, when the mount path is not found on its own. |
168
  | `CIVITAI_API_HOST` | unset | Pins the search host; otherwise `civitai.red` is asked first and `civitai.com` is the fallback. |
169
 
170
  ## Where diffusers comes from
app.py CHANGED
@@ -22,6 +22,8 @@ from functools import cache
22
  import spaces
23
  import gradio as gr
24
 
 
 
25
  MODEL_REPO = os.environ.get("H3_MODEL_REPO", "MiniMaxAI/MiniMax-H3")
26
  CONDITIONER_SPACE = os.environ.get("H3_CONDITIONER", "multimodalart/qwen3vl-conditioner")
27
  # `lazy` moves all 72.16 GiB onto the card on the first GPU call and leaves it there; `offload` hands placement to
@@ -2066,17 +2068,18 @@ THEME = gr.themes.Soft(primary_hue="blue", secondary_hue="cyan", neutral_hue="sl
2066
 
2067
  HERO = """
2068
  <div id="h3-hero">
2069
- <h1>MiniMax-H3 &middot; reference &rarr; video + soundtrack &middot; Custom lora + CivitAI search,
2070
  structured prompt builder, scene continuation, GPU cost, profiles, clip stitching</h1>
2071
  <p>33B model generating video and a fully synchronized soundtrack (ambience, foley, speech) from your own subject,
2072
  voice or camera move. The prompt builder writes the labelled sections H3 was actually trained on, dialogue tags
2073
- and all.
 
2074
  <a href="https://huggingface.co/MiniMaxAI/MiniMax-H3" target="_blank" rel="noopener">model</a> &middot;
2075
  <a href="https://www.minimax.io/blog/minimax-h3" target="_blank" rel="noopener">blog</a> &middot;
2076
  <a href="https://huggingface.co/spaces/multimodalart/minimax-h3" target="_blank" rel="noopener">text / image to video</a></p>
2077
  <div class="pills">
2078
  <span>33B</span><span>joint video + audio</span><span>Turbo lora: 4&ndash;8 steps</span>
2079
- <span>ComfyUI lora accepted</span><span>up to 9 references</span><span>5 custom lora slots</span><span>structured prompt builder</span><span>named profiles</span><span>CivitAI search + links</span><span>scene continuation</span><span>kohya + LoKr auto-convert</span><span>GPU cost estimate</span><span>clip stitching with audio</span>
2080
  </div>
2081
  </div>
2082
  """
@@ -2277,6 +2280,9 @@ with gr.Blocks(title="MiniMax-H3 - Custom lora + CivitAI, structured prompts, GP
2277
  type="filepath",
2278
  )
2279
 
 
 
 
2280
  with gr.Tab("🎛️ Output"):
2281
  canvas = gr.Dropdown(label="Canvas", choices=list(CANVASES), value=DEFAULT_CANVAS)
2282
  match = gr.Checkbox(label="Match the reference soundtrack", value=True, visible=False)
 
22
  import spaces
23
  import gradio as gr
24
 
25
+ import lora_library
26
+
27
  MODEL_REPO = os.environ.get("H3_MODEL_REPO", "MiniMaxAI/MiniMax-H3")
28
  CONDITIONER_SPACE = os.environ.get("H3_CONDITIONER", "multimodalart/qwen3vl-conditioner")
29
  # `lazy` moves all 72.16 GiB onto the card on the first GPU call and leaves it there; `offload` hands placement to
 
2068
 
2069
  HERO = """
2070
  <div id="h3-hero">
2071
+ <h1>MiniMax-H3 &middot; reference &rarr; video + soundtrack &middot; Shared lora library + CivitAI search,
2072
  structured prompt builder, scene continuation, GPU cost, profiles, clip stitching</h1>
2073
  <p>33B model generating video and a fully synchronized soundtrack (ambience, foley, speech) from your own subject,
2074
  voice or camera move. The prompt builder writes the labelled sections H3 was actually trained on, dialogue tags
2075
+ and all, and the shared lora library remembers every adapter anyone adds &mdash; link, trigger words and strength
2076
+ &mdash; and fills the slots in one press.
2077
  <a href="https://huggingface.co/MiniMaxAI/MiniMax-H3" target="_blank" rel="noopener">model</a> &middot;
2078
  <a href="https://www.minimax.io/blog/minimax-h3" target="_blank" rel="noopener">blog</a> &middot;
2079
  <a href="https://huggingface.co/spaces/multimodalart/minimax-h3" target="_blank" rel="noopener">text / image to video</a></p>
2080
  <div class="pills">
2081
  <span>33B</span><span>joint video + audio</span><span>Turbo lora: 4&ndash;8 steps</span>
2082
+ <span>ComfyUI lora accepted</span><span>up to 9 references</span><span>5 custom lora slots</span><span>shared lora library</span><span>trigger words in the prompt</span><span>structured prompt builder</span><span>named profiles</span><span>CivitAI search + links</span><span>scene continuation</span><span>kohya + LoKr auto-convert</span><span>GPU cost estimate</span><span>clip stitching with audio</span>
2083
  </div>
2084
  </div>
2085
  """
 
2280
  type="filepath",
2281
  )
2282
 
2283
+ lora_library.library_tab(lora_slots=lora_references, scale_slots=lora_scales,
2284
+ prompt_box=prompt)
2285
+
2286
  with gr.Tab("🎛️ Output"):
2287
  canvas = gr.Dropdown(label="Canvas", choices=list(CANVASES), value=DEFAULT_CANVAS)
2288
  match = gr.Checkbox(label="Match the reference soundtrack", value=True, visible=False)
lora_library.py ADDED
@@ -0,0 +1,816 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Shared lora library, stored in a Hugging Face storage bucket.
2
+
3
+ The list lives in `hf://buckets/<owner>/<bucket>/library.json` and holds two
4
+ sections, "normal" and "nsfw". Each entry keeps a name, the link, its trigger
5
+ words and a strength. A bucket is mutable object storage rather than a git
6
+ repo, so the file is simply overwritten on every change and survives a restart
7
+ of the Space.
8
+
9
+ Writing needs an `HF_TOKEN` with write access. When the bucket is mounted into
10
+ the Space as a volume, set `LORA_LIBRARY_DIR` to the mount path and the file is
11
+ read and written directly, with no token involved.
12
+
13
+ Used from app.py with a single line inside `with gr.Blocks(...) as demo:`
14
+
15
+ lora_library.library_tab(lora_box=user_loras_text, prompt_box=prompt)
16
+
17
+ or, for a build with fixed lora slots,
18
+
19
+ lora_library.library_tab(lora_slots=lora_refs, scale_slots=lora_scales,
20
+ prompt_box=prompt)
21
+ """
22
+
23
+ from __future__ import annotations
24
+
25
+ import json
26
+ import os
27
+ import re
28
+ import time
29
+
30
+ import gradio as gr
31
+ import requests
32
+
33
+ BUCKET = os.environ.get("LORA_LIBRARY_BUCKET", "amisima/minimax-h3-reference-4-step-lora-storage")
34
+ LIBRARY_FILE = os.environ.get("LORA_LIBRARY_FILE", "library.json")
35
+ LOCAL_DIR = os.environ.get("LORA_LIBRARY_DIR", "").strip()
36
+ # Fallback for a Space whose huggingface_hub is too old to speak to buckets.
37
+ DATASET_REPO = os.environ.get("LORA_LIBRARY_REPO", BUCKET)
38
+
39
+ _CIVITAI_DOWNLOAD_RE = re.compile(r"/api/download/models/(\d+)")
40
+ _HF_URL_RE = re.compile(r"^https?://(?:www\.)?huggingface\.co/(?:datasets/)?([^/\s]+)/([^/\s?#]+)")
41
+
42
+
43
+ # ---------------------------------------------------------------------------
44
+ # storage
45
+ # ---------------------------------------------------------------------------
46
+
47
+ def _token() -> str:
48
+ return (os.environ.get("HF_TOKEN") or os.environ.get("HUGGINGFACE_HUB_TOKEN") or "").strip()
49
+
50
+
51
+ def _bucket_path() -> str:
52
+ return f"buckets/{BUCKET.strip('/')}/{LIBRARY_FILE}"
53
+
54
+
55
+ _mount_cache: dict[str, str] = {}
56
+
57
+
58
+ def _runtime_volume_paths() -> list[str]:
59
+ """Ask the Hub where this Space has its volumes mounted.
60
+
61
+ A bucket attached in the Space settings lands at whatever mount path was
62
+ typed in there, so guessing is hopeless. The plain REST endpoint is used
63
+ rather than the Python client, because the client on an older Space does
64
+ not know about volumes at all.
65
+ """
66
+ space_id = os.environ.get("SPACE_ID") or os.environ.get("SPACE_REPO_ID") or ""
67
+ if not space_id:
68
+ return []
69
+ headers = {"User-Agent": "Mozilla/5.0"}
70
+ token = _token()
71
+ if token:
72
+ headers["Authorization"] = f"Bearer {token}"
73
+ try:
74
+ response = requests.get(
75
+ f"https://huggingface.co/api/spaces/{space_id}/runtime",
76
+ headers=headers, timeout=20,
77
+ )
78
+ response.raise_for_status()
79
+ data = response.json()
80
+ except Exception as error: # noqa: BLE001
81
+ print(f"[lora-library] could not read the Space runtime: "
82
+ f"{type(error).__name__}: {error}", flush=True)
83
+ return []
84
+
85
+ wanted = BUCKET.strip("/").lower()
86
+ preferred, others = [], []
87
+
88
+ def walk(node):
89
+ if isinstance(node, dict):
90
+ mount = node.get("mountPath") or node.get("mount_path")
91
+ if isinstance(mount, str) and mount.startswith("/"):
92
+ source = str(node.get("source") or node.get("repoId") or "").strip("/").lower()
93
+ (preferred if source.endswith(wanted) else others).append(mount)
94
+ for value in node.values():
95
+ walk(value)
96
+ elif isinstance(node, list):
97
+ for value in node:
98
+ walk(value)
99
+
100
+ walk(data)
101
+ return preferred + others
102
+
103
+
104
+ def _mount_table_paths() -> list[str]:
105
+ """Mount points that look like an attached volume rather than the OS."""
106
+ system = ("/proc", "/sys", "/dev", "/run", "/etc", "/usr", "/lib", "/bin",
107
+ "/sbin", "/boot", "/var/lib", "/var/run")
108
+ found = []
109
+ try:
110
+ with open("/proc/mounts", "r", encoding="utf-8", errors="ignore") as handle:
111
+ for line in handle:
112
+ parts = line.split()
113
+ if len(parts) < 3:
114
+ continue
115
+ point, fstype = parts[1], parts[2]
116
+ if point == "/" or any(point.startswith(prefix) for prefix in system):
117
+ continue
118
+ if fstype in ("nfs", "nfs4", "fuse", "virtiofs", "9p") or fstype.startswith("fuse"):
119
+ found.insert(0, point)
120
+ elif point.count("/") == 1:
121
+ found.append(point)
122
+ except Exception: # noqa: BLE001
123
+ return []
124
+ return found
125
+
126
+
127
+ def _volume_mount_path() -> str:
128
+ if "path" in _mount_cache:
129
+ return str(_mount_cache["path"])
130
+ _mount_cache["path"] = ""
131
+ for path in _runtime_volume_paths() + _mount_table_paths():
132
+ if path and os.path.isdir(path) and os.access(path, os.W_OK):
133
+ _mount_cache["path"] = path
134
+ print(f"[lora-library] using the volume mounted at {path}", flush=True)
135
+ break
136
+ return str(_mount_cache["path"])
137
+
138
+
139
+ def _mounted_file() -> str:
140
+ """The library file on a locally mounted bucket, or "" when there is none."""
141
+ bucket_name = BUCKET.strip("/").split("/")[-1]
142
+ candidates = [LOCAL_DIR] if LOCAL_DIR else []
143
+ candidates.append(_volume_mount_path())
144
+ candidates += [
145
+ f"/bucket/{bucket_name}", "/bucket",
146
+ f"/buckets/{bucket_name}", "/buckets",
147
+ f"/mnt/bucket/{bucket_name}", "/mnt/bucket",
148
+ f"/data/{bucket_name}", "/data",
149
+ f"/mnt/data/{bucket_name}", "/mnt/data",
150
+ f"/storage/{bucket_name}", "/storage",
151
+ ]
152
+ for directory in candidates:
153
+ if directory and os.path.isdir(directory) and os.access(directory, os.W_OK):
154
+ return os.path.join(directory, LIBRARY_FILE)
155
+ return ""
156
+
157
+
158
+ def _hub_has_buckets() -> bool:
159
+ """Buckets need huggingface_hub 1.5 or newer; older Spaces have to fall back."""
160
+ try:
161
+ from huggingface_hub import HfFileSystem # noqa: F401
162
+ except Exception: # noqa: BLE001
163
+ return False
164
+ try:
165
+ from importlib.metadata import version as _version
166
+
167
+ parts = _version("huggingface_hub").split(".")
168
+ return (int(parts[0]), int(parts[1])) >= (1, 5)
169
+ except Exception: # noqa: BLE001
170
+ return True
171
+
172
+
173
+ def _filesystem():
174
+ from huggingface_hub import HfFileSystem
175
+
176
+ token = _token()
177
+ return HfFileSystem(token=token) if token else HfFileSystem()
178
+
179
+
180
+ def _empty() -> dict:
181
+ return {"normal": [], "nsfw": []}
182
+
183
+
184
+ def _clean(data) -> dict:
185
+ out = _empty()
186
+ if not isinstance(data, dict):
187
+ return out
188
+ for section in ("normal", "nsfw"):
189
+ for item in data.get(section) or []:
190
+ if not isinstance(item, dict):
191
+ continue
192
+ url = str(item.get("url") or "").strip()
193
+ if not url:
194
+ continue
195
+ try:
196
+ strength = float(item.get("strength", 1.0))
197
+ except (TypeError, ValueError):
198
+ strength = 1.0
199
+ out[section].append({
200
+ "name": str(item.get("name") or "").strip() or url.split("/")[-1][:60],
201
+ "url": url,
202
+ "trigger": str(item.get("trigger") or "").strip(),
203
+ "strength": strength,
204
+ "added": str(item.get("added") or ""),
205
+ })
206
+ return out
207
+
208
+
209
+ # Every backend is tried in turn, and whichever one answers is remembered so the
210
+ # reader and the writer never end up looking in two different places.
211
+ _active = {"backend": "", "detail": ""}
212
+
213
+
214
+ def _read_mounted():
215
+ mounted = _mounted_file()
216
+ if not mounted:
217
+ return None
218
+ if not os.path.exists(mounted):
219
+ return _empty()
220
+ with open(mounted, "r", encoding="utf-8") as handle:
221
+ return _clean(json.load(handle))
222
+
223
+
224
+ def _write_mounted(payload: str) -> None:
225
+ mounted = _mounted_file()
226
+ if not mounted:
227
+ raise RuntimeError("no mounted bucket")
228
+ with open(mounted, "w", encoding="utf-8") as handle:
229
+ handle.write(payload)
230
+
231
+
232
+ def _read_bucket():
233
+ if not _hub_has_buckets():
234
+ raise RuntimeError("huggingface_hub is too old for buckets (needs 1.5+)")
235
+ filesystem = _filesystem()
236
+ path = _bucket_path()
237
+ if not filesystem.exists(path):
238
+ return _empty()
239
+ with filesystem.open(path, "r") as handle:
240
+ return _clean(json.loads(handle.read()))
241
+
242
+
243
+ def _write_bucket(payload: str) -> None:
244
+ if not _hub_has_buckets():
245
+ raise RuntimeError("huggingface_hub is too old for buckets (needs 1.5+)")
246
+ if not _token():
247
+ raise RuntimeError("no HF_TOKEN")
248
+ filesystem = _filesystem()
249
+ with filesystem.open(_bucket_path(), "w") as handle:
250
+ handle.write(payload)
251
+
252
+
253
+ def _read_dataset():
254
+ url = (f"https://huggingface.co/datasets/{DATASET_REPO}/resolve/main/"
255
+ f"{LIBRARY_FILE}?cb={int(time.time())}")
256
+ headers = {"User-Agent": "Mozilla/5.0"}
257
+ token = _token()
258
+ if token:
259
+ headers["Authorization"] = f"Bearer {token}"
260
+ response = requests.get(url, headers=headers, timeout=25)
261
+ if response.status_code == 404:
262
+ return _empty()
263
+ response.raise_for_status()
264
+ return _clean(response.json())
265
+
266
+
267
+ def _write_dataset(payload: str) -> None:
268
+ if not _token():
269
+ raise RuntimeError("no HF_TOKEN")
270
+ import io
271
+
272
+ from huggingface_hub import HfApi
273
+
274
+ api = HfApi(token=_token())
275
+ api.create_repo(repo_id=DATASET_REPO, repo_type="dataset", exist_ok=True, private=True)
276
+ api.upload_file(
277
+ path_or_fileobj=io.BytesIO(payload.encode("utf-8")),
278
+ path_in_repo=LIBRARY_FILE,
279
+ repo_id=DATASET_REPO,
280
+ repo_type="dataset",
281
+ commit_message="update lora library",
282
+ )
283
+
284
+
285
+ def _mounted_name() -> str:
286
+ mounted = _mounted_file()
287
+ return f"mounted bucket ({os.path.dirname(mounted)})" if mounted else "mounted bucket"
288
+
289
+
290
+ # ---------------------------------------------------------------------------
291
+ # buckets from a Space whose huggingface_hub is pinned below 1.5
292
+ # ---------------------------------------------------------------------------
293
+ # Some builds pin an old huggingface_hub on purpose (diffusers / ComfyUI
294
+ # compatibility) and upgrading it in place would break the Space. A modern copy
295
+ # is installed into a directory of its own and driven from a subprocess, so the
296
+ # running app never imports it.
297
+
298
+ _ISOLATED_DIR = os.environ.get("LORA_LIBRARY_HUB_DIR", "/tmp/lora_library_hub")
299
+ _isolated_state: dict[str, object] = {}
300
+
301
+
302
+ def _isolated_ready() -> bool:
303
+ if "ok" in _isolated_state:
304
+ return bool(_isolated_state["ok"])
305
+ _isolated_state["ok"] = False
306
+ if os.path.isdir(os.path.join(_ISOLATED_DIR, "huggingface_hub")):
307
+ _isolated_state["ok"] = True
308
+ return True
309
+ print("[lora-library] installing a private huggingface_hub for bucket access…", flush=True)
310
+ import subprocess
311
+ import sys
312
+
313
+ try:
314
+ subprocess.run(
315
+ [sys.executable, "-m", "pip", "install", "--no-cache-dir", "--quiet",
316
+ "--target", _ISOLATED_DIR, "huggingface_hub>=1.5"],
317
+ check=True, timeout=600,
318
+ )
319
+ _isolated_state["ok"] = os.path.isdir(os.path.join(_ISOLATED_DIR, "huggingface_hub"))
320
+ except Exception as error: # noqa: BLE001
321
+ print(f"[lora-library] private huggingface_hub install failed: "
322
+ f"{type(error).__name__}: {error}", flush=True)
323
+ return bool(_isolated_state["ok"])
324
+
325
+
326
+ def _isolated_run(mode: str, payload_path: str = "") -> str:
327
+ import subprocess
328
+ import sys
329
+
330
+ if not _isolated_ready():
331
+ raise RuntimeError("could not install a huggingface_hub new enough for buckets")
332
+ script = (
333
+ "import sys, json, os\n"
334
+ f"sys.path.insert(0, {_ISOLATED_DIR!r})\n"
335
+ "from huggingface_hub import HfFileSystem\n"
336
+ "token = os.environ.get('HF_TOKEN') or os.environ.get('HUGGINGFACE_HUB_TOKEN') or None\n"
337
+ "fs = HfFileSystem(token=token)\n"
338
+ f"path = {_bucket_path()!r}\n"
339
+ f"mode = {mode!r}\n"
340
+ f"payload_path = {payload_path!r}\n"
341
+ "if mode == 'read':\n"
342
+ " if fs.exists(path):\n"
343
+ " with fs.open(path, 'r') as handle:\n"
344
+ " sys.stdout.write(handle.read())\n"
345
+ " else:\n"
346
+ " sys.stdout.write('{}')\n"
347
+ "else:\n"
348
+ " with open(payload_path, 'r', encoding='utf-8') as handle:\n"
349
+ " body = handle.read()\n"
350
+ " with fs.open(path, 'w') as handle:\n"
351
+ " handle.write(body)\n"
352
+ " sys.stdout.write('ok')\n"
353
+ )
354
+ environment = dict(os.environ)
355
+ environment.pop("PYTHONPATH", None)
356
+ result = subprocess.run(
357
+ [sys.executable, "-c", script],
358
+ capture_output=True, text=True, timeout=180, env=environment,
359
+ )
360
+ if result.returncode != 0:
361
+ raise RuntimeError((result.stderr or "").strip().splitlines()[-1:] or "subprocess failed")
362
+ return result.stdout
363
+
364
+
365
+ def _read_isolated():
366
+ return _clean(json.loads(_isolated_run("read") or "{}"))
367
+
368
+
369
+ def _write_isolated(payload: str) -> None:
370
+ import tempfile
371
+
372
+ if not _token():
373
+ raise RuntimeError("no HF_TOKEN")
374
+ handle = tempfile.NamedTemporaryFile("w", suffix=".json", delete=False, encoding="utf-8")
375
+ handle.write(payload)
376
+ handle.close()
377
+ try:
378
+ _isolated_run("write", handle.name)
379
+ finally:
380
+ os.unlink(handle.name)
381
+
382
+
383
+ _BACKENDS = [
384
+ ("mounted bucket", _read_mounted, _write_mounted),
385
+ (f"bucket {BUCKET}", _read_bucket, _write_bucket),
386
+ (f"bucket {BUCKET} (private hub)", _read_isolated, _write_isolated),
387
+ (f"dataset {DATASET_REPO}", _read_dataset, _write_dataset),
388
+ ]
389
+
390
+
391
+ def load_library() -> dict:
392
+ """Read from the backend that last worked, or find one that does."""
393
+ ordered = _BACKENDS
394
+ if _active["backend"]:
395
+ ordered = ([b for b in _BACKENDS if b[0] == _active["backend"]]
396
+ + [b for b in _BACKENDS if b[0] != _active["backend"]])
397
+ first_error = ""
398
+ for name, reader, _writer in ordered:
399
+ try:
400
+ data = reader()
401
+ except Exception as error: # noqa: BLE001
402
+ first_error = first_error or f"{name}: {type(error).__name__}: {error}"
403
+ print(f"[lora-library] read from {name} failed: {error}", flush=True)
404
+ continue
405
+ if data is None:
406
+ continue
407
+ if data["normal"] or data["nsfw"] or not _active["backend"]:
408
+ _active["backend"], _active["detail"] = name, ""
409
+ return data
410
+ _active["detail"] = first_error
411
+ return _empty()
412
+
413
+
414
+ def save_library(library: dict) -> str:
415
+ """Write, then read back to prove it landed. Returns "" or a message."""
416
+ payload = json.dumps(_clean(library), ensure_ascii=False, indent=1)
417
+ wanted = {item["url"] for item in _clean(library)["normal"] + _clean(library)["nsfw"]}
418
+
419
+ problems = []
420
+ for name, reader, writer in _BACKENDS:
421
+ try:
422
+ writer(payload)
423
+ except Exception as error: # noqa: BLE001
424
+ problems.append(f"{name}: {type(error).__name__}: {error}")
425
+ continue
426
+ try:
427
+ written = reader() or _empty()
428
+ except Exception as error: # noqa: BLE001
429
+ problems.append(f"{name}: written, but could not be read back ({error})")
430
+ continue
431
+ got = {item["url"] for item in written["normal"] + written["nsfw"]}
432
+ if wanted - got:
433
+ problems.append(f"{name}: written, but it did not come back")
434
+ continue
435
+ _active["backend"], _active["detail"] = name, ""
436
+ return ""
437
+
438
+ detail = "; ".join(problems) or "no storage backend is available"
439
+ _active["detail"] = detail
440
+ return (f"**nothing was saved.** {detail}\n\n"
441
+ f"Easiest fix: attach the `{BUCKET}` bucket to this Space under "
442
+ "*Settings → Storage Buckets* (any mount path will do) — then no token is needed "
443
+ "at all. Otherwise add an `HF_TOKEN` with write access under "
444
+ "*Settings → Variables and secrets*.")
445
+
446
+
447
+ def storage_note() -> str:
448
+ if _active["backend"] and not _active["detail"]:
449
+ name = _mounted_name() if _active["backend"] == "mounted bucket" else _active["backend"]
450
+ return f"storage: {name}"
451
+ if _active["detail"]:
452
+ return f"storage: not working yet — {_active['detail']}"
453
+ return "storage: not contacted yet"
454
+
455
+
456
+ # ---------------------------------------------------------------------------
457
+ # reading a link
458
+ # ---------------------------------------------------------------------------
459
+
460
+ def lookup_link(url: str):
461
+ """(name, trigger words) for a civitai / huggingface link."""
462
+ url = (url or "").strip().strip('"').strip("'")
463
+ if not url:
464
+ return "", ""
465
+
466
+ match = _CIVITAI_DOWNLOAD_RE.search(url)
467
+ if match:
468
+ headers = {"User-Agent": "Mozilla/5.0"}
469
+ civitai_token = os.environ.get("CIVITAI_TOKEN", "").strip()
470
+ if civitai_token:
471
+ headers["Authorization"] = f"Bearer {civitai_token}"
472
+ for host in ("civitai.red", "civitai.com"):
473
+ try:
474
+ response = requests.get(
475
+ f"https://{host}/api/v1/model-versions/{match.group(1)}",
476
+ headers=headers, timeout=20,
477
+ )
478
+ response.raise_for_status()
479
+ data = response.json()
480
+ except Exception: # noqa: BLE001
481
+ continue
482
+ model_name = (data.get("model") or {}).get("name") or ""
483
+ version_name = data.get("name") or ""
484
+ label = f"{model_name} · {version_name}".strip(" ·")
485
+ words = [w for w in (data.get("trainedWords") or []) if w]
486
+ return label, ", ".join(words[:8])
487
+ return "", ""
488
+
489
+ hf_match = _HF_URL_RE.match(url)
490
+ if hf_match:
491
+ return f"{hf_match.group(1)}/{hf_match.group(2)}", ""
492
+
493
+ return url.split("?")[0].split("/")[-1], ""
494
+
495
+
496
+ # ---------------------------------------------------------------------------
497
+ # helpers
498
+ # ---------------------------------------------------------------------------
499
+
500
+ def _labels(entries: list[dict]) -> list[str]:
501
+ labels = []
502
+ for index, item in enumerate(entries, start=1):
503
+ label = f"{index}. {item['name']} · strength {item['strength']:g}"
504
+ if item.get("trigger"):
505
+ label += f" · trigger: {item['trigger']}"
506
+ labels.append(label[:220])
507
+ return labels
508
+
509
+
510
+ def _picked(entries: list[dict], labels: list[str], chosen) -> list[dict]:
511
+ index_by_label = {label: i for i, label in enumerate(labels)}
512
+ out = []
513
+ for label in chosen or []:
514
+ index = index_by_label.get(label)
515
+ if index is not None and index < len(entries):
516
+ out.append(entries[index])
517
+ return out
518
+
519
+
520
+ def _autofind(predicate):
521
+ try:
522
+ from gradio.context import Context
523
+
524
+ root = getattr(Context, "root_block", None)
525
+ if root is None:
526
+ return None
527
+ for component in root.blocks.values():
528
+ if isinstance(component, gr.Textbox):
529
+ label = str(getattr(component, "label", "") or "")
530
+ if predicate(label.lower()):
531
+ return component
532
+ except Exception: # noqa: BLE001
533
+ return None
534
+ return None
535
+
536
+
537
+ def _is_lora_box(label: str) -> bool:
538
+ return "lora" in label and "search" not in label and "civitai" not in label
539
+
540
+
541
+ def _is_prompt_box(label: str) -> bool:
542
+ if "prompt" not in label:
543
+ return False
544
+ return not any(word in label for word in
545
+ ("negative", "segment", "relay", "scene", "enhance", "search"))
546
+
547
+
548
+ def _with_triggers(prompt_text: str, triggers: list[str]) -> str:
549
+ text = prompt_text or ""
550
+ missing = [t for t in triggers if t and t.lower() not in text.lower()]
551
+ if not missing:
552
+ return text
553
+ return f"{text.rstrip().rstrip(',')}, {', '.join(missing)}".strip(" ,")
554
+
555
+
556
+ # ---------------------------------------------------------------------------
557
+ # UI
558
+ # ---------------------------------------------------------------------------
559
+
560
+ def library_tab(lora_box=None, prompt_box=None, lora_slots=None, scale_slots=None,
561
+ label: str = "📚 lora library"):
562
+ """Draw the library tab. Either hand it a multi-line lora textbox
563
+ (`lora_box`), or a list of fixed slots (`lora_slots` + `scale_slots`)."""
564
+ slots = list(lora_slots or [])
565
+ scales = list(scale_slots or [])
566
+ if not slots and lora_box is None:
567
+ lora_box = _autofind(_is_lora_box)
568
+ if prompt_box is None:
569
+ prompt_box = _autofind(_is_prompt_box)
570
+
571
+ has_slots = bool(slots)
572
+ has_box = (not has_slots) and lora_box is not None
573
+ has_prompt = prompt_box is not None
574
+
575
+ library = load_library()
576
+
577
+ with gr.Tab(label):
578
+ gr.Markdown(
579
+ f"a shared list, kept in the `{BUCKET}` bucket, so it survives a restart. "
580
+ "tick what you want and it goes straight into the lora "
581
+ + ("slots" if has_slots else "box") + ", trigger words and all."
582
+ )
583
+
584
+ state = gr.State(library)
585
+ normal_labels = gr.State(_labels(library["normal"]))
586
+ nsfw_labels = gr.State(_labels(library["nsfw"]))
587
+
588
+ normal_pick = gr.CheckboxGroup(
589
+ choices=_labels(library["normal"]), value=[],
590
+ label="ordinary", interactive=True,
591
+ )
592
+ nsfw_pick = gr.CheckboxGroup(
593
+ choices=_labels(library["nsfw"]), value=[],
594
+ label="nsfw", interactive=True,
595
+ )
596
+
597
+ with gr.Row():
598
+ add_button = gr.Button(
599
+ "➕ add the ticked ones to the lora " + ("slots" if has_slots else "box"),
600
+ variant="primary",
601
+ )
602
+ refresh_button = gr.Button("🔄 refresh")
603
+
604
+ use_triggers = gr.Checkbox(
605
+ value=True, label="append the trigger words to the prompt",
606
+ )
607
+ status = gr.Markdown("")
608
+ storage_line = gr.Markdown(storage_note())
609
+
610
+ with gr.Accordion("➕ add a lora to the library", open=False):
611
+ new_url = gr.Textbox(label="link (civitai / huggingface / direct .safetensors)", lines=1)
612
+ lookup_button = gr.Button("🔎 read the name and triggers off the link")
613
+ new_name = gr.Textbox(label="name", lines=1)
614
+ new_trigger = gr.Textbox(label="trigger words", lines=1)
615
+ with gr.Row():
616
+ new_strength = gr.Number(value=1.0, label="strength", precision=2)
617
+ new_nsfw = gr.Checkbox(value=False, label="nsfw")
618
+ save_button = gr.Button("💾 save to the library", variant="primary")
619
+
620
+ with gr.Accordion("🗑 remove", open=False):
621
+ delete_button = gr.Button("remove the ones ticked above")
622
+
623
+ # ------------------------------------------------------------ events
624
+
625
+ def _pack(data, message):
626
+ normal, nsfw = _labels(data["normal"]), _labels(data["nsfw"])
627
+ return (
628
+ data, normal, nsfw,
629
+ gr.update(choices=normal, value=[]),
630
+ gr.update(choices=nsfw, value=[]),
631
+ message,
632
+ storage_note(),
633
+ )
634
+
635
+ def _refresh():
636
+ data = load_library()
637
+ return _pack(data, f"loaded: {len(data['normal'])} ordinary, "
638
+ f"{len(data['nsfw'])} nsfw")
639
+
640
+ refresh_targets = [state, normal_labels, nsfw_labels, normal_pick, nsfw_pick,
641
+ status, storage_line]
642
+ refresh_button.click(_refresh, None, refresh_targets)
643
+
644
+ lookup_button.click(lambda url: lookup_link(url), new_url, [new_name, new_trigger])
645
+
646
+ def _save(data, url, name, trigger, strength, nsfw):
647
+ url = (url or "").strip().strip('"').strip("'")
648
+ # Start from what storage actually holds, so two people adding a lora
649
+ # at the same time do not overwrite each other.
650
+ data = load_library()
651
+ if not url:
652
+ return _pack(data, "paste a link first.")
653
+ for existing in data["normal"] + data["nsfw"]:
654
+ if existing["url"] == url:
655
+ return _pack(data, f"already on the list: {existing['name']}")
656
+ if not (name or "").strip():
657
+ name = lookup_link(url)[0] or url.split("?")[0].split("/")[-1]
658
+ try:
659
+ strength = float(strength)
660
+ except (TypeError, ValueError):
661
+ strength = 1.0
662
+ data["nsfw" if nsfw else "normal"].append({
663
+ "name": name.strip(),
664
+ "url": url,
665
+ "trigger": (trigger or "").strip(),
666
+ "strength": strength,
667
+ "added": time.strftime("%Y-%m-%d"),
668
+ })
669
+ error = save_library(data)
670
+ # Show what came back out of storage, not what went in.
671
+ return _pack(load_library() if not error else data,
672
+ error or f"added and saved: {name.strip()}")
673
+
674
+ save_button.click(
675
+ _save,
676
+ [state, new_url, new_name, new_trigger, new_strength, new_nsfw],
677
+ refresh_targets,
678
+ )
679
+
680
+ def _delete(data, normal, nsfw, normal_chosen, nsfw_chosen):
681
+ data = _clean(data)
682
+ doomed = {item["url"] for item in
683
+ _picked(data["normal"], normal, normal_chosen)
684
+ + _picked(data["nsfw"], nsfw, nsfw_chosen)}
685
+ if not doomed:
686
+ return _pack(data, "nothing is ticked.")
687
+ fresh = load_library()
688
+ for section in ("normal", "nsfw"):
689
+ fresh[section] = [i for i in fresh[section] if i["url"] not in doomed]
690
+ error = save_library(fresh)
691
+ return _pack(load_library() if not error else data,
692
+ error or f"removed: {len(doomed)}")
693
+
694
+ delete_button.click(
695
+ _delete,
696
+ [state, normal_labels, nsfw_labels, normal_pick, nsfw_pick],
697
+ refresh_targets,
698
+ )
699
+
700
+ # ------------------------------------------- adding to the lora slots
701
+
702
+ if has_slots:
703
+ add_inputs = [state, normal_labels, nsfw_labels, normal_pick, nsfw_pick, use_triggers]
704
+ add_outputs = [status]
705
+ if has_prompt:
706
+ add_inputs.append(prompt_box)
707
+ add_outputs.append(prompt_box)
708
+ add_outputs = add_outputs + slots + scales
709
+
710
+ def _add_slots(data, normal, nsfw, normal_chosen, nsfw_chosen, triggers_on,
711
+ prompt_text=""):
712
+ data = _clean(data)
713
+ chosen = (_picked(data["normal"], normal, normal_chosen)
714
+ + _picked(data["nsfw"], nsfw, nsfw_chosen))
715
+ blank = [gr.update() for _ in slots] + [gr.update() for _ in scales]
716
+ if not chosen:
717
+ return tuple(["nothing is ticked."]
718
+ + ([gr.update()] if has_prompt else []) + blank)
719
+
720
+ used = chosen[:len(slots)]
721
+ spare = chosen[len(slots):]
722
+ ref_updates = [
723
+ gr.update(value=used[i]["url"]) if i < len(used) else gr.update()
724
+ for i in range(len(slots))
725
+ ]
726
+ scale_updates = [
727
+ gr.update(value=used[i]["strength"]) if i < len(used) else gr.update()
728
+ for i in range(len(scales))
729
+ ]
730
+
731
+ message = f"**in the slots:** {', '.join(i['name'] for i in used)}"
732
+ if spare:
733
+ message += (f" \nno room for: {', '.join(i['name'] for i in spare)} "
734
+ f"(there are {len(slots)} slots)")
735
+ trigger_words = [i["trigger"] for i in used if i.get("trigger")]
736
+
737
+ out = [message]
738
+ if has_prompt:
739
+ out.append(gr.update(value=_with_triggers(prompt_text, trigger_words))
740
+ if triggers_on and trigger_words else gr.update())
741
+ elif trigger_words:
742
+ out[0] += f" \n**trigger words for the prompt:** {', '.join(trigger_words)}"
743
+ return tuple(out + ref_updates + scale_updates)
744
+
745
+ add_button.click(_add_slots, add_inputs, add_outputs)
746
+
747
+ # --------------------------------------------- adding to the lora box
748
+
749
+ else:
750
+ add_inputs = [state, normal_labels, nsfw_labels, normal_pick, nsfw_pick, use_triggers]
751
+ add_outputs = [status]
752
+ if has_box:
753
+ add_inputs.append(lora_box)
754
+ add_outputs.append(lora_box)
755
+ if has_prompt:
756
+ add_inputs.append(prompt_box)
757
+ add_outputs.append(prompt_box)
758
+
759
+ def _add_box(data, normal, nsfw, normal_chosen, nsfw_chosen, triggers_on, *boxes):
760
+ data = _clean(data)
761
+ chosen = (_picked(data["normal"], normal, normal_chosen)
762
+ + _picked(data["nsfw"], nsfw, nsfw_chosen))
763
+
764
+ position = 0
765
+ lora_text = boxes[position] if has_box else ""
766
+ if has_box:
767
+ position += 1
768
+ prompt_text = boxes[position] if has_prompt else ""
769
+
770
+ if not chosen:
771
+ out = ["nothing is ticked."]
772
+ out += [gr.update()] * (int(has_box) + int(has_prompt))
773
+ return tuple(out) if len(out) > 1 else out[0]
774
+
775
+ text = (lora_text or "").rstrip()
776
+ added, skipped, trigger_words = [], [], []
777
+ for item in chosen:
778
+ if item["url"] in text:
779
+ skipped.append(item["name"])
780
+ else:
781
+ text = f"{text}\n{item['url']} | {item['strength']:g}".strip()
782
+ added.append(item["name"])
783
+ if item.get("trigger"):
784
+ trigger_words.append(item["trigger"])
785
+
786
+ message = ""
787
+ if added:
788
+ message += f"**added:** {', '.join(added)}"
789
+ if skipped:
790
+ message += f" \nalready in the box: {', '.join(skipped)}"
791
+ if trigger_words and not has_prompt:
792
+ message += f" \n**trigger words for the prompt:** {', '.join(trigger_words)}"
793
+ if not has_box:
794
+ lines = "\n".join(f"{i['url']} | {i['strength']:g}" for i in chosen)
795
+ message = f"copy these lines into the lora box:\n\n```\n{lines}\n```\n\n{message}"
796
+
797
+ out = [message]
798
+ if has_box:
799
+ out.append(gr.update(value=text))
800
+ if has_prompt:
801
+ out.append(gr.update(value=_with_triggers(prompt_text, trigger_words))
802
+ if triggers_on and trigger_words else gr.update())
803
+ return tuple(out) if len(out) > 1 else out[0]
804
+
805
+ add_button.click(_add_box, add_inputs, add_outputs)
806
+
807
+ try:
808
+ from gradio.context import Context
809
+
810
+ root = getattr(Context, "root_block", None)
811
+ if root is not None:
812
+ root.load(_refresh, None, refresh_targets)
813
+ except Exception: # noqa: BLE001
814
+ pass
815
+
816
+ return state