RiverRider commited on
Commit
cec8e12
Β·
verified Β·
1 Parent(s): 5603210

Initial SRT Showcase Space

Browse files
Files changed (3) hide show
  1. README.md +45 -7
  2. app.py +567 -0
  3. requirements.txt +15 -0
README.md CHANGED
@@ -1,13 +1,51 @@
1
  ---
2
- title: Srt Showcase
3
- emoji: 🐨
4
- colorFrom: blue
5
- colorTo: pink
6
  sdk: gradio
7
- sdk_version: 6.17.3
8
- python_version: '3.12'
9
  app_file: app.py
 
10
  pinned: false
 
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: SRT Showcase
3
+ emoji: 🧭
4
+ colorFrom: indigo
5
+ colorTo: blue
6
  sdk: gradio
7
+ sdk_version: "4.44.1"
 
8
  app_file: app.py
9
+ python_version: "3.10"
10
  pinned: false
11
+ hardware: zerogpu
12
+ short_description: Watch a frozen Qwen-2.5-7B think β€” live SRT introspection
13
  ---
14
 
15
+ # SRT Showcase β€” watch a frozen model think
16
+
17
+ Live, token-by-token introspection of **Qwen-2.5-7B + the SRT adapter**.
18
+
19
+ As the model generates, every token is tinted by its predictive **entropy** (the
20
+ validated uncertainty signal). At the highest-effort token positions, chosen by
21
+ an adaptive-density scheduler, the **Activation Verbalizer** decodes the model's
22
+ internal hidden state into natural language, and each verbalization carries a
23
+ **round-trip fidelity badge**: it is re-encoded and compared back to the original
24
+ hidden state, so the "this is what the model was thinking" claim is visibly
25
+ self-validating.
26
+
27
+ Features:
28
+
29
+ - Live token stream tinted by entropy or SRT divergence, with per-token hover
30
+ rollovers (entropy, divergence, reflexivity `rΜ‚`, regime).
31
+ - Running entropy meter and entropy / divergence charts.
32
+ - Expand/collapse verbalization cards with round-trip fidelity badges.
33
+ - A/B panel: the same prompt with SRT injection on vs off (bare backbone),
34
+ seeded identically.
35
+ - A curated example gallery covering confident recall, false premises,
36
+ misconceptions, reasoning pivots, genuine uncertainty, and safety boundaries.
37
+
38
+ ## Honest scope
39
+
40
+ Entropy is the load-bearing uncertainty signal. The SRT side-channels
41
+ (divergence, `rΜ‚`, regime) and the verbalizations are shown as **observational
42
+ readouts** of internal state. This is a window into the model, not a validated
43
+ hallucination detector.
44
+
45
+ ## Notes
46
+
47
+ - First request is cold (~60–90 s) while ZeroGPU acquires a GPU and the ~16 GB
48
+ backbone weights load; subsequent requests are warm.
49
+ - A second backbone copy is loaded for the Activation Verbalizer.
50
+
51
+ Source: <https://github.com/space-bacon/SRT>
app.py ADDED
@@ -0,0 +1,567 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """SRT Showcase β€” live introspection demo for the Semiotic-Reflexive Transformer.
2
+
3
+ A single Gradio app that streams generation from a frozen Qwen-2.5-7B + the SRT
4
+ adapter and shows, in real time, what the model is doing internally:
5
+
6
+ β€’ Live token stream, each token tinted by its predictive ENTROPY (the
7
+ validated online uncertainty signal) β€” toggle to tint by SRT divergence.
8
+ β€’ A running entropy meter (mean / peak) as the answer builds.
9
+ β€’ Charts of entropy and SRT divergence across the generated tokens.
10
+ β€’ Expand/collapse natural-language VERBALIZATIONS of the model's hidden state
11
+ at the highest-effort token positions (chosen by the adaptive-density
12
+ scheduler), each round-trip validated by the Activation Verbalizer.
13
+ β€’ Per-token hover rollovers: entropy, divergence, reflexivity rΜ‚, regime.
14
+ β€’ Regenerate, and an "adapter on/off" switch.
15
+
16
+ Honest scope: entropy is the load-bearing uncertainty signal. The SRT
17
+ side-channels (divergence, rΜ‚, regime) and the verbalizations are shown as
18
+ *observational* readouts of internal state β€” a window into the model, not a
19
+ validated hallucination detector.
20
+
21
+ Run locally on a GPU box:
22
+ pip install -r demo/requirements.txt
23
+ PYTHONPATH=. python demo/srt_showcase_app.py
24
+
25
+ Deploys to an HF Space (ZeroGPU / a10g). Qwen-7B needs ~16 GB bf16; the AV
26
+ adds ~2 GB.
27
+ """
28
+
29
+ from __future__ import annotations
30
+
31
+ import html
32
+ import logging
33
+ import os
34
+
35
+ import gradio as gr
36
+ import torch
37
+
38
+ logging.basicConfig(level=logging.INFO)
39
+ logger = logging.getLogger("srt_showcase")
40
+
41
+ # ── ZeroGPU-compatible GPU decorator (no-op off-Space) ───────────────────
42
+ try: # pragma: no cover - environment dependent
43
+ import spaces # type: ignore
44
+
45
+ _ON_ZEROGPU = bool(os.environ.get("SPACES_ZERO_GPU"))
46
+
47
+ def _gpu(duration: int = 300):
48
+ if _ON_ZEROGPU:
49
+ return spaces.GPU(duration=duration)
50
+ return lambda fn: fn
51
+ except Exception: # local / non-Space
52
+ _ON_ZEROGPU = False
53
+
54
+ def _gpu(duration: int = 300):
55
+ def _wrap(fn):
56
+ return fn
57
+ return _wrap
58
+
59
+
60
+ DEVICE = "cuda" if (torch.cuda.is_available() or _ON_ZEROGPU) else "cpu"
61
+
62
+
63
+ # ── Palette ──────────────────────────────────────────────────────────────
64
+ BG = "#0a1429"
65
+ PANEL = "#16213d"
66
+ PANEL_ALT = "#1d2b4d"
67
+ INK = "#e6ecf5"
68
+ MUTED = "#8aa0c8"
69
+ CYAN = "#46e0d0"
70
+ MINT = "#7cf0a8"
71
+ PINK = "#ff7eb6"
72
+ LAVENDER = "#b69cff"
73
+ AMBER = "#ffcf66"
74
+
75
+ # Public-Space guards: cap prompt length and generated tokens so a single
76
+ # ZeroGPU request stays within the duration budget.
77
+ MAX_PROMPT_CHARS = 1500
78
+ MAX_TOKENS_CAP = 512
79
+
80
+ # Round-trip fidelity reference frame (raw fve_nrm on Qwen2.5-7B L20, from the
81
+ # anchored oracle_ceiling study). Unrelated text floors near 0.622; the
82
+ # paraphrase best-of-8 ceiling is ~0.848. We normalise the round-trip cosine
83
+ # against this band so the badge reads 0% (no better than chance) to 100%
84
+ # (matches the paraphrase ceiling) rather than against a meaningless raw 0.
85
+ RT_FLOOR = 0.622
86
+ RT_CEIL = 0.848
87
+
88
+ # Lazy global trace handle (loaded once on first generation).
89
+ _TRACE = None
90
+
91
+
92
+ def _get_trace():
93
+ global _TRACE
94
+ if _TRACE is None:
95
+ from srt_introspect import Trace # local import keeps import-time light
96
+ logger.info("Loading SRT Trace (adapter + activation verbalizer)...")
97
+ _TRACE = Trace.load()
98
+ logger.info("Trace ready on device=%s", _TRACE.device)
99
+ return _TRACE
100
+
101
+
102
+ # ── Signal β†’ colour ──────────────────────────────────────────────────────
103
+ def _lerp(c0, c1, t):
104
+ return tuple(int(round(a + (b - a) * t)) for a, b in zip(c0, c1))
105
+
106
+
107
+ def _entropy_color(ent: float, lo: float, hi: float) -> str:
108
+ """Green (calm) β†’ amber β†’ red (uncertain) over [lo, hi] nats."""
109
+ if hi <= lo:
110
+ t = 0.0
111
+ else:
112
+ t = max(0.0, min(1.0, (ent - lo) / (hi - lo)))
113
+ g = (124, 240, 168) # mint
114
+ a = (255, 207, 102) # amber
115
+ r = (255, 126, 182) # pink/red
116
+ rgb = _lerp(g, a, t * 2) if t < 0.5 else _lerp(a, r, (t - 0.5) * 2)
117
+ return "rgba(%d,%d,%d,0.30)" % rgb
118
+
119
+
120
+ def _div_color(d: float, lo: float, hi: float) -> str:
121
+ if hi <= lo:
122
+ t = 0.0
123
+ else:
124
+ t = max(0.0, min(1.0, (d - lo) / (hi - lo)))
125
+ rgb = _lerp((70, 224, 208), (255, 126, 182), t) # cyan β†’ pink
126
+ return "rgba(%d,%d,%d,0.30)" % rgb
127
+
128
+
129
+ # ── Renderers ──────────────────────────────────────────────────────────────
130
+ def _render_tokens(result, tint: str) -> str:
131
+ """Per-token HTML, tinted by entropy or divergence, with hover rollovers."""
132
+ steps = result.steps
133
+ if not steps:
134
+ return f"<div style='color:{MUTED}'>…</div>"
135
+ ents = [s.entropy for s in steps]
136
+ divs = [s.divergence for s in steps]
137
+ e_lo, e_hi = min(ents), max(ents)
138
+ d_lo, d_hi = min(divs), max(divs)
139
+
140
+ spans = []
141
+ for s in steps:
142
+ if tint == "divergence":
143
+ bg = _div_color(s.divergence, d_lo, d_hi)
144
+ else:
145
+ bg = _entropy_color(s.entropy, e_lo, e_hi)
146
+ tok = html.escape(s.token).replace("\n", "⏎<br>")
147
+ title = (f"#{s.token_idx} H={s.entropy:.2f} nats "
148
+ f"div={s.divergence:.2f} rΜ‚={s.r_hat:.2f} "
149
+ f"regime={'super' if s.regime else 'sub'}")
150
+ sel = " sel" if s.verbalization else ""
151
+ spans.append(
152
+ f"<span class='tok{sel}' style='background:{bg}' "
153
+ f"data-title=\"{html.escape(title)}\">{tok}</span>"
154
+ )
155
+ return f"<div class='toks'>{''.join(spans)}</div>"
156
+
157
+
158
+ def _render_meter(result) -> str:
159
+ steps = result.steps
160
+ if not steps:
161
+ return ""
162
+ ents = [s.entropy for s in steps]
163
+ mean_e = sum(ents) / len(ents)
164
+ max_e = max(ents)
165
+ # Risk bar scaled to a ~3.0-nat practical ceiling.
166
+ frac = max(0.0, min(1.0, mean_e / 3.0))
167
+ pct = int(frac * 100)
168
+ col = MINT if frac < 0.33 else (AMBER if frac < 0.66 else PINK)
169
+ return (
170
+ f"<div class='meter'>"
171
+ f"<div class='meter-row'><span>mean entropy</span>"
172
+ f"<b style='color:{col}'>{mean_e:.2f}</b> nats</div>"
173
+ f"<div class='bar'><div class='fill' style='width:{pct}%;background:{col}'></div></div>"
174
+ f"<div class='meter-row'><span>peak entropy</span><b>{max_e:.2f}</b> nats"
175
+ f" &nbsp;Β·&nbsp; <span>{len(steps)} tokens</span></div>"
176
+ f"</div>"
177
+ )
178
+
179
+
180
+ def _sparkline(values, color, h=70, w=920):
181
+ if len(values) < 2:
182
+ return ""
183
+ lo, hi = min(values), max(values)
184
+ rng = (hi - lo) or 1.0
185
+ n = len(values)
186
+ pts = " ".join(
187
+ f"{w * i / (n - 1):.1f},{h - (h - 8) * (v - lo) / rng - 4:.1f}"
188
+ for i, v in enumerate(values)
189
+ )
190
+ return (
191
+ f"<svg viewBox='0 0 {w} {h}' width='100%' height='{h}' "
192
+ f"preserveAspectRatio='none'>"
193
+ f"<polyline points='{pts}' fill='none' stroke='{color}' "
194
+ f"stroke-width='1.6'/></svg>"
195
+ )
196
+
197
+
198
+ def _render_charts(result) -> str:
199
+ steps = result.steps
200
+ if len(steps) < 2:
201
+ return ""
202
+ ent = _sparkline([s.entropy for s in steps], CYAN)
203
+ dv = _sparkline([s.divergence for s in steps], PINK)
204
+ return (
205
+ f"<div class='chart'><div class='chart-label' style='color:{CYAN}'>"
206
+ f"predictive entropy (uncertainty)</div>{ent}</div>"
207
+ f"<div class='chart'><div class='chart-label' style='color:{PINK}'>"
208
+ f"SRT divergence (observational)</div>{dv}</div>"
209
+ )
210
+
211
+
212
+ def _render_verbalizations(result) -> str:
213
+ sel = [s for s in result.steps if s.verbalization]
214
+ if not sel:
215
+ return f"<div style='color:{MUTED}'>No verbalizations yet.</div>"
216
+ cards = []
217
+ for s in sel:
218
+ tok = html.escape(s.token.strip() or "Β·")
219
+ verb = html.escape(s.verbalization or "")
220
+ badge = _roundtrip_badge(s.roundtrip_cos)
221
+ cards.append(
222
+ f"<details class='vcard'><summary>"
223
+ f"<span class='vtok'>β€œ{tok}”</span> "
224
+ f"<span class='vmeta'>#{s.token_idx} Β· div {s.divergence:.2f} Β· "
225
+ f"rΜ‚ {s.r_hat:.2f} Β· {'super' if s.regime else 'sub'}</span>"
226
+ f"{badge}"
227
+ f"</summary><div class='vbody'>{verb}</div></details>"
228
+ )
229
+ return "".join(cards)
230
+
231
+
232
+ def _roundtrip_badge(cos) -> str:
233
+ """A self-validation badge: re-encode the verbalization, measure how close
234
+ its hidden state lands to the original. Normalised against the paraphrase
235
+ ceiling (see RT_FLOOR / RT_CEIL)."""
236
+ if cos is None:
237
+ return ""
238
+ fve = 0.5 * (1.0 + float(cos))
239
+ frac = max(0.0, min(1.0, (fve - RT_FLOOR) / (RT_CEIL - RT_FLOOR)))
240
+ pct = int(round(frac * 100))
241
+ col = MINT if frac > 0.66 else (AMBER if frac > 0.33 else PINK)
242
+ return (
243
+ f"<span class='rt' style='border-color:{col};color:{col}' "
244
+ f"title='Re-encoded verbalization cos={cos:.3f} vs original hidden state; "
245
+ f"normalised against the paraphrase ceiling.'>"
246
+ f"round-trip {pct}% Β· cos {cos:.2f}</span>"
247
+ )
248
+
249
+
250
+ _CSS = f"""
251
+ <style>
252
+ .toks {{ line-height: 2.1; font-size: 15px; }}
253
+ .tok {{ position: relative; padding: 1px 2px; border-radius: 3px;
254
+ white-space: pre-wrap; cursor: default; }}
255
+ .tok.sel {{ outline: 1px solid {LAVENDER}; }}
256
+ .tok:hover::after {{
257
+ content: attr(data-title); position: absolute; left: 0; top: 1.9em;
258
+ white-space: nowrap; z-index: 20; background: {PANEL_ALT};
259
+ color: {INK}; border: 1px solid {LAVENDER}; border-radius: 6px;
260
+ padding: 5px 9px; font-size: 11px; font-family: ui-monospace, monospace; }}
261
+ .meter {{ background: {PANEL}; border-radius: 10px; padding: 12px 14px;
262
+ color: {INK}; }}
263
+ .meter-row {{ display: flex; gap: 8px; align-items: baseline;
264
+ color: {MUTED}; font-size: 13px; margin: 2px 0; }}
265
+ .meter-row b {{ color: {INK}; font-size: 16px; }}
266
+ .bar {{ height: 10px; background: {BG}; border-radius: 5px; overflow: hidden;
267
+ margin: 6px 0; }}
268
+ .fill {{ height: 100%; transition: width .3s ease; }}
269
+ .chart {{ background: {PANEL}; border-radius: 10px; padding: 8px 12px;
270
+ margin: 8px 0; }}
271
+ .chart-label {{ font-size: 12px; font-family: ui-monospace, monospace;
272
+ margin-bottom: 2px; }}
273
+ .vcard {{ background: {PANEL}; border: 1px solid {PANEL_ALT};
274
+ border-radius: 8px; margin: 6px 0; padding: 4px 10px; }}
275
+ .vcard summary {{ cursor: pointer; color: {INK}; }}
276
+ .vtok {{ color: {CYAN}; font-weight: 600; }}
277
+ .vmeta {{ color: {MUTED}; font-size: 12px; font-family: ui-monospace, monospace; }}
278
+ .vbody {{ color: {INK}; padding: 8px 4px 4px; font-size: 14px;
279
+ border-top: 1px solid {PANEL_ALT}; margin-top: 6px; }}
280
+ .rt {{ float: right; font-size: 11px; font-family: ui-monospace, monospace;
281
+ border: 1px solid {MUTED}; border-radius: 10px; padding: 1px 8px;
282
+ margin-left: 8px; }}
283
+ .abwrap {{ display: flex; gap: 12px; }}
284
+ .abcol {{ flex: 1; background: {PANEL}; border-radius: 10px; padding: 10px 12px; }}
285
+ .abhead {{ font-family: ui-monospace, monospace; font-size: 12px;
286
+ margin-bottom: 6px; }}
287
+ </style>
288
+ """
289
+
290
+
291
+ # App-level CSS (injected into gr.Blocks) β€” paints the whole Gradio surface in
292
+ # the dark-blue palette so the page matches the trace panels.
293
+ _APP_CSS = f"""
294
+ .gradio-container, .gradio-container .main, body {{
295
+ background: {BG} !important;
296
+ color: {INK} !important;
297
+ }}
298
+ .gradio-container .prose, .gradio-container .prose * {{ color: {INK} !important; }}
299
+ .gradio-container .block, .gradio-container .form,
300
+ .gradio-container .gr-box, .gradio-container .gr-panel {{
301
+ background: {PANEL} !important;
302
+ border-color: {PANEL_ALT} !important;
303
+ color: {INK} !important;
304
+ }}
305
+ .gradio-container input, .gradio-container textarea,
306
+ .gradio-container .gr-input, .gradio-container select {{
307
+ background: {PANEL_ALT} !important;
308
+ color: {INK} !important;
309
+ border-color: {PANEL_ALT} !important;
310
+ }}
311
+ .gradio-container .tab-nav button {{ color: {MUTED} !important; }}
312
+ .gradio-container .tab-nav button.selected {{ color: {CYAN} !important; }}
313
+ """
314
+
315
+
316
+ # ── Generation callback (streaming) ──────────────────────────────────────
317
+ @_gpu(duration=300)
318
+ def cb_generate(prompt, mode, max_new, budget, k, temperature, top_p,
319
+ repetition_penalty, tint, inject):
320
+ if not prompt or not prompt.strip():
321
+ yield (_CSS + "<i>Enter a prompt.</i>", "", "", "", "_(enter a prompt)_")
322
+ return
323
+ prompt = prompt[:MAX_PROMPT_CHARS]
324
+ max_new = min(int(max_new), MAX_TOKENS_CAP)
325
+
326
+ trace = _get_trace()
327
+ model_prompt = prompt
328
+ if mode == "Chat":
329
+ # Use the backbone chat template if available.
330
+ try:
331
+ model_prompt = trace.tok.apply_chat_template(
332
+ [{"role": "user", "content": prompt}],
333
+ tokenize=False, add_generation_prompt=True,
334
+ )
335
+ except Exception:
336
+ model_prompt = prompt
337
+
338
+ last = None
339
+ for result, done in trace.stream(
340
+ model_prompt,
341
+ max_new_tokens=int(max_new), budget=int(budget), k=int(k),
342
+ temperature=float(temperature), top_p=float(top_p),
343
+ repetition_penalty=float(repetition_penalty),
344
+ disable_injectors=(not inject),
345
+ ):
346
+ last = result
347
+ toks = _CSS + _render_tokens(result, tint)
348
+ meter = _render_meter(result)
349
+ charts = _render_charts(result)
350
+ if done:
351
+ verbs = _render_verbalizations(result)
352
+ yield toks, meter, charts, verbs, result.text
353
+ else:
354
+ yield toks, meter, charts, "<i>generating… verbalizations appear when done.</i>", result.text
355
+
356
+
357
+ # ── Curated example gallery ───────────────────────────────────────────────
358
+ # Prompts grouped by the introspection phenomenon they tend to surface. Each
359
+ # row maps to the [prompt, mode] inputs. The categories are organised so a
360
+ # first-time visitor can see, in a few clicks, where the SRT signals light up:
361
+ # confident recall vs genuine uncertainty vs a false premise the model has to
362
+ # work around vs a reasoning pivot vs a safety boundary.
363
+ EXAMPLES = [
364
+ # β€” Confident factual recall: low entropy at the fact token; the
365
+ # verbalization should name the very fact being emitted. β€”
366
+ ["What is the capital of Australia, and when did it become the capital?", "Chat"],
367
+ ["Who wrote the novel 'Pride and Prejudice', and in what year was it first published?", "Chat"],
368
+
369
+ # β€” False premise / counterfactual: the prompt asserts something untrue.
370
+ # Watch whether the divergence/regime signals and the verbalization
371
+ # reflect the model resisting or going along with the premise. β€”
372
+ ["Explain why the Great Wall of China is clearly visible from the Moon with the naked eye.", "Chat"],
373
+ ["Describe what the astronauts saw when they walked on the surface of the Sun.", "Chat"],
374
+
375
+ # β€” Common misconception: tests whether the model corrects the myth. β€”
376
+ ["Is it true that humans only use 10 percent of their brains?", "Chat"],
377
+
378
+ # β€” Multi-step reasoning / arithmetic: divergence tends to spike at the
379
+ # calculation pivot rather than the surrounding prose. β€”
380
+ ["A train leaves at 14:35 and arrives at 17:10. How long is the journey in minutes?", "Chat"],
381
+ ["A shirt costs $40 after a 20% discount. What was the original price? Show your reasoning.", "Chat"],
382
+
383
+ # β€” Genuine uncertainty / forecast / opinion: elevated entropy because
384
+ # many continuations are equally valid. β€”
385
+ ["Will it rain in Berlin next Tuesday?", "Chat"],
386
+ ["What do you think the most widely used programming language will be in 2035?", "Chat"],
387
+
388
+ # β€” Safety boundary / refusal: a regime shift as the model pivots to
389
+ # declining. β€”
390
+ ["Give me step-by-step instructions to pick a standard pin-tumbler lock.", "Chat"],
391
+
392
+ # β€” Ambiguity / garden-path: the model must commit to one parse. β€”
393
+ ["What does the sentence 'The old man the boats' mean? Explain carefully.", "Chat"],
394
+
395
+ # β€” Hold both sides / hedge: sustained mid-range entropy while it weighs
396
+ # competing framings. β€”
397
+ ["Is a hot dog a sandwich? Briefly argue both sides, then give your verdict.", "Chat"],
398
+
399
+ # β€” Structured generation (code): low entropy in the boilerplate, higher
400
+ # at genuine design choices. β€”
401
+ ["Write a Python function that returns the nth Fibonacci number.", "Chat"],
402
+
403
+ # β€” Open-ended creative: high entropy throughout β€” many valid next tokens. β€”
404
+ ["Write the opening sentence of a mystery novel set on a Mars colony.", "Chat"],
405
+
406
+ # β€” Plain explainer baseline. β€”
407
+ ["Explain in two sentences why the sky is blue.", "Chat"],
408
+ ]
409
+
410
+
411
+ # ── A/B compare callback (injection on vs off) ────────────────────────────
412
+ @_gpu(duration=300)
413
+ def cb_compare(prompt, mode, max_new, budget, k, temperature, top_p,
414
+ repetition_penalty, tint):
415
+ """Run the same prompt twice β€” SRT injection ON vs OFF β€” and render the two
416
+ token streams side by side so the adapter's effect on generation is
417
+ visible. Verbalizations are skipped here (budget=0) to keep the compare
418
+ fast; the single-generation tab covers those."""
419
+ if not prompt or not prompt.strip():
420
+ yield _CSS + "<i>Enter a prompt.</i>", ""
421
+ return
422
+ prompt = prompt[:MAX_PROMPT_CHARS]
423
+ max_new = min(int(max_new), MAX_TOKENS_CAP)
424
+
425
+ trace = _get_trace()
426
+ model_prompt = prompt
427
+ if mode == "Chat":
428
+ try:
429
+ model_prompt = trace.tok.apply_chat_template(
430
+ [{"role": "user", "content": prompt}],
431
+ tokenize=False, add_generation_prompt=True,
432
+ )
433
+ except Exception:
434
+ model_prompt = prompt
435
+
436
+ cols = {True: None, False: None}
437
+
438
+ def _render():
439
+ def _one(res, label, color):
440
+ if res is None:
441
+ body = f"<div style='color:{MUTED}'>…</div>"
442
+ head = label
443
+ else:
444
+ body = _render_tokens(res, tint)
445
+ ents = [s.entropy for s in res.steps] or [0.0]
446
+ head = (f"{label} &nbsp;Β·&nbsp; mean H "
447
+ f"{sum(ents)/len(ents):.2f} &nbsp;Β·&nbsp; {len(res.steps)} tok")
448
+ return (f"<div class='abcol'><div class='abhead' style='color:{color}'>"
449
+ f"{head}</div>{body}</div>")
450
+ return (_CSS + "<div class='abwrap'>"
451
+ + _one(cols[True], "SRT injection ON", MINT)
452
+ + _one(cols[False], "injection OFF (bare backbone)", MUTED)
453
+ + "</div>")
454
+
455
+ for inject in (True, False):
456
+ # Seed both passes identically so the visible difference reflects the
457
+ # adapter, not sampling noise.
458
+ torch.manual_seed(1234)
459
+ for result, done in trace.stream(
460
+ model_prompt,
461
+ max_new_tokens=int(max_new), budget=0, k=int(k),
462
+ temperature=float(temperature), top_p=float(top_p),
463
+ repetition_penalty=float(repetition_penalty),
464
+ disable_injectors=(not inject),
465
+ ):
466
+ cols[inject] = result
467
+ yield _render(), ""
468
+
469
+ a = (cols[True].text if cols[True] else "").strip()
470
+ b = (cols[False].text if cols[False] else "").strip()
471
+ summary = (
472
+ f"**ON:** {a or '_(empty)_'}\n\n**OFF:** {b or '_(empty)_'}"
473
+ )
474
+ yield _render(), summary
475
+
476
+
477
+ def build() -> gr.Blocks:
478
+ theme = gr.themes.Base(primary_hue="blue", neutral_hue="slate")
479
+ with gr.Blocks(title="SRT Showcase", css=_APP_CSS, theme=theme) as app:
480
+ gr.Markdown(
481
+ "## SRT Showcase β€” watch a frozen model think\n"
482
+ "Live token-by-token introspection of **Qwen-2.5-7B + the SRT adapter**. "
483
+ "Tokens are tinted by **predictive entropy** (validated uncertainty signal). "
484
+ "SRT divergence, reflexivity `rΜ‚`, regime, and the natural-language "
485
+ "verbalizations are **observational readouts** of internal state β€” a window "
486
+ "into the model, not a hallucination detector."
487
+ )
488
+ with gr.Row():
489
+ with gr.Column(scale=2):
490
+ prompt = gr.Textbox(label="Prompt", lines=4,
491
+ value="Explain in two sentences why the sky is blue.")
492
+ with gr.Row():
493
+ mode = gr.Radio(["Completion", "Chat"], value="Chat", label="Mode")
494
+ tint = gr.Radio(["entropy", "divergence"], value="entropy",
495
+ label="Tint tokens by")
496
+ inject = gr.Checkbox(value=True, label="SRT injection on")
497
+ with gr.Row():
498
+ max_new = gr.Slider(16, 1024, value=256, step=16, label="max tokens")
499
+ budget = gr.Slider(2, 20, value=10, step=1, label="verbalization slots")
500
+ k = gr.Slider(1, 8, value=4, step=1, label="AV samples / slot (K)")
501
+ with gr.Row():
502
+ temperature = gr.Slider(0.0, 1.5, value=0.7, step=0.05, label="temperature")
503
+ top_p = gr.Slider(0.1, 1.0, value=0.95, step=0.05, label="top-p")
504
+ rep = gr.Slider(1.0, 1.5, value=1.15, step=0.01, label="rep. penalty")
505
+ with gr.Row():
506
+ go = gr.Button("Generate", variant="primary")
507
+ regen = gr.Button("Regenerate")
508
+ with gr.Column(scale=1):
509
+ meter = gr.HTML(label="entropy meter")
510
+
511
+ gr.Markdown(
512
+ "### Curated examples β€” what to watch for\n"
513
+ "Pick a prompt below, then read the signals as it generates:\n"
514
+ "- **Confident recall** (capital of Australia, *Pride and Prejudice*): "
515
+ "low entropy at the fact; the verbalization names the fact itself.\n"
516
+ "- **False premise** (Wall of China from the Moon, walking on the Sun): "
517
+ "watch the divergence/regime signals as the model works around an untrue claim.\n"
518
+ "- **Misconception** (10% of the brain): does it correct the myth?\n"
519
+ "- **Reasoning pivot** (train minutes, discount price): divergence spikes at the calculation, not the prose.\n"
520
+ "- **Genuine uncertainty** (rain Tuesday, language in 2035): elevated entropy β€” many valid continuations.\n"
521
+ "- **Safety boundary** (lock picking): a regime shift as it pivots to declining.\n"
522
+ "- **Ambiguity** ('The old man the boats'): the model commits to one parse.\n"
523
+ "- **Open-ended / creative** (Mars mystery opener): high entropy throughout."
524
+ )
525
+ gr.Examples(
526
+ examples=EXAMPLES, inputs=[prompt, mode], label="Curated examples",
527
+ examples_per_page=15,
528
+ )
529
+
530
+ with gr.Tab("Introspection"):
531
+ tokens = gr.HTML(label="token stream")
532
+ charts = gr.HTML(label="charts")
533
+ with gr.Accordion("Verbalizations (expand each) β€” with round-trip fidelity", open=True):
534
+ verbs = gr.HTML()
535
+ final = gr.Textbox(label="Final output", lines=4)
536
+
537
+ with gr.Tab("A/B: injection on vs off"):
538
+ gr.Markdown(
539
+ "Runs the same prompt twice with the SRT side-channel injection "
540
+ "**on** and **off** (bare frozen backbone), seeded identically so "
541
+ "the visible difference is the adapter, not sampling noise."
542
+ )
543
+ ab_go = gr.Button("Compare", variant="primary")
544
+ ab_html = gr.HTML()
545
+ ab_summary = gr.Markdown()
546
+
547
+ inputs = [prompt, mode, max_new, budget, k, temperature, top_p, rep, tint, inject]
548
+ outputs = [tokens, meter, charts, verbs, final]
549
+ go.click(cb_generate, inputs=inputs, outputs=outputs)
550
+ regen.click(cb_generate, inputs=inputs, outputs=outputs)
551
+
552
+ ab_inputs = [prompt, mode, max_new, budget, k, temperature, top_p, rep, tint]
553
+ ab_go.click(cb_compare, inputs=ab_inputs, outputs=[ab_html, ab_summary])
554
+ return app
555
+
556
+
557
+ if __name__ == "__main__":
558
+ app = build()
559
+ app.queue(default_concurrency_limit=1, max_size=20)
560
+ if _ON_ZEROGPU or os.environ.get("SPACE_ID"):
561
+ # On HF Spaces the platform supplies host/port.
562
+ app.launch()
563
+ else:
564
+ app.launch(
565
+ server_name="0.0.0.0",
566
+ server_port=int(os.environ.get("PORT", "8080")),
567
+ )
requirements.txt ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SRT Showcase Space β€” runtime deps.
2
+ # The srt / srt_introspect packages are installed from the GitHub repo so the
3
+ # Space tracks the same code as the project.
4
+ srt-adapter @ git+https://github.com/space-bacon/SRT.git@main
5
+
6
+ # HARD PIN: transformers 4.55+ breaks the adapter's manual-decoder generation.
7
+ transformers==4.53.3
8
+ torch>=2.2
9
+ gradio==4.44.1
10
+ spaces>=0.30
11
+ huggingface_hub>=0.24
12
+ accelerate>=0.33
13
+ safetensors>=0.4
14
+ sentencepiece
15
+ numpy