Summarization
Transformers
Safetensors
English
t5
text2text-generation
tab-naming
terminal-title-generation
short-text-generation
seq2seq
flan-t5
on-device
electron
porkicoder
text-generation-inference
Instructions to use porkr/porkicoder-tab-namer-77m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use porkr/porkicoder-tab-namer-77m with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "summarization" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("summarization", model="porkr/porkicoder-tab-namer-77m")# Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("porkr/porkicoder-tab-namer-77m") model = AutoModelForSeq2SeqLM.from_pretrained("porkr/porkicoder-tab-namer-77m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"/> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"/> | |
| <title>Two tab-title recipes: 16 GB Mac and phone-class silicon</title> | |
| <meta name="description" content="FLAN plus a centroid highlighter for 16 GB Apple Silicon. Our 35M Hybrid A for A18-class devices. Same glue, two namers, one holdout."/> | |
| <style> | |
| :root { | |
| --paper: #f5f0e7; | |
| --ink: #17211f; | |
| --muted: #58625f; | |
| --faint: #d6cec1; | |
| --sniff: #176b87; | |
| --good: #32735f; | |
| --bad: #9b4034; | |
| --sans: Inter, ui-sans-serif, system-ui, sans-serif; | |
| --serif: Charter, "Iowan Old Style", Georgia, serif; | |
| } | |
| * { box-sizing: border-box; } | |
| html { background: var(--paper); } | |
| body { | |
| margin: 0; | |
| color: var(--ink); | |
| background: | |
| radial-gradient(circle at 12% 0%, rgba(23,107,135,.08), transparent 28rem), | |
| var(--paper); | |
| font-family: var(--serif); | |
| font-size: 18px; | |
| line-height: 1.58; | |
| } | |
| .paper { width: min(calc(100% - 2rem), 920px); margin: 0 auto; padding: 4rem 0 5rem; } | |
| .eyebrow { margin: 0 0 1rem; color: var(--sniff); font: 750 .76rem/1 var(--sans); letter-spacing: .13em; text-transform: uppercase; } | |
| h1 { margin: 0; font-size: clamp(2.4rem, 7vw, 4.4rem); line-height: .95; letter-spacing: -.04em; font-weight: 650; } | |
| .subtitle { color: #34403d; font-size: 1.25rem; line-height: 1.35; margin: 1.2rem 0 1.4rem; } | |
| .meta { color: var(--muted); font: 0.83rem/1.4 var(--sans); } | |
| .abstract { margin: 2rem 0 2.5rem; padding: 1.3rem 1.4rem; border: 1px solid var(--faint); border-radius: 16px; background: rgba(255,255,255,.55); } | |
| .abstract h2 { margin: 0 0 .4rem; font: 750 .72rem/1 var(--sans); letter-spacing: .1em; text-transform: uppercase; } | |
| .abstract p { margin: 0; font-size: 1.02rem; } | |
| h2 { margin: 2.6rem 0 .7rem; font-size: 1.55rem; } | |
| h3 { margin: 1.6rem 0 .4rem; font-size: 1.15rem; } | |
| p, li { max-width: 70ch; } | |
| table { width: 100%; border-collapse: collapse; font: 14px/1.4 var(--sans); margin: 1rem 0 1.4rem; } | |
| th, td { border-bottom: 1px solid var(--faint); padding: .45rem .4rem; text-align: left; vertical-align: top; } | |
| th { font-size: 11px; letter-spacing: .04em; text-transform: uppercase; color: var(--muted); } | |
| td.n, th.n { text-align: right; font-variant-numeric: tabular-nums; } | |
| tr.pick td { background: #e4f0ea; } | |
| tr.mobile td { background: #e8eef5; } | |
| .ex { margin: 1rem 0; padding: 1rem 1.1rem; border: 1px solid var(--faint); border-radius: 12px; background: #fffdf8; font-size: 0.95rem; } | |
| .ex pre { white-space: pre-wrap; margin: .4rem 0 .8rem; font: 13px/1.4 ui-monospace, monospace; color: #333; } | |
| figcaption, .note { color: var(--muted); font: 0.82rem/1.4 var(--sans); } | |
| svg { width: 100%; height: auto; background: #fffdf8; border: 1px solid var(--faint); border-radius: 12px; } | |
| .wide { overflow-x: auto; margin: 1rem 0 1.4rem; } | |
| .wide table { margin: 0; min-width: 780px; } | |
| table.appx { font: 12.5px/1.35 var(--sans); } | |
| table.appx.samples { min-width: 920px; } | |
| .appx td.task { color: var(--muted); font-style: italic; min-width: 200px; } | |
| .appx .s { color: var(--muted); font-size: 10.5px; } | |
| a { color: var(--sniff); } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="paper"> | |
| <p class="eyebrow">PorkiCoder Research · Tab titles</p> | |
| <h1>Two recipes, one glue</h1> | |
| <p class="subtitle">16 GB Macs get FLAN plus our highlighter. Phone-class chips get the 35M we actually trained. The same A18 Pro sits in a $599 Neo and in an iPhone; that is the split, not Air versus Pro.</p> | |
| <p class="meta">14 August 2026 · Holdout n=1,000 · Blind Gemini 3.5 Flash-Lite · Sealed 1,000 unused</p> | |
| <section class="abstract"> | |
| <h2>Update · 15 August 2026</h2> | |
| <p>The quality conclusion behind this two-recipe decision was later superseded on the same clean-development set: the later B-9500 checkpoint beat the FLAN stack after a safe four-beam decoder exposed better complete word pairs. Production timing and shadowing remain open. Read <a href="./tab-namer-four-beams.html">Four beams, no new weights</a>.</p> | |
| </section> | |
| <section class="abstract"> | |
| <h2>Terminology · 16 August 2026</h2> | |
| <p>In this paper, “Hybrid A” names the whole 2026-08-14 stack: the 35M GSG namer with greedy-era decode, the random seed-42 centroid, and the 2+1 glue. Since 2026-08-16, “Hybrid A” names only the glue rule (≤2 on-page namer words, fill to 3 with centroid); the production stack is mid-GSG B-9500 + beam-4 + centroid v2 + Hybrid A glue.</p> | |
| </section> | |
| <section class="abstract"> | |
| <h2>Board · 16 August 2026</h2> | |
| <p>Every number in this paper is <strong>SO-board</strong> (1,000 Stack Overflow clean-dev pages), except where a task snippet is clearly a terminal example. The later near-miss (FLAN + centroid v2 = 6.11 / 88.5%) lives on SO-board. <strong>Terminal-board</strong> (300 terminal-dev; FLAN + centroid v2 = 7.11 / 93.3%) is a different leaderboard and is not scored here. Do not mix the two. Later: <a href="../tab_namer_quality_roadmap/session4_results.html">session 4</a>.</p> | |
| </section> | |
| <section class="abstract"> | |
| <h2>Abstract</h2> | |
| <p> | |
| We did not waste a GPU week trying to beat Google’s instruction mix with a 35M pretrain. | |
| We found <strong>two shippable title stacks that share one combiner</strong>. | |
| On 16 GB Apple Silicon (Air or Pro), FLAN-T5-small glued to a 16 MiB centroid highlighter is 77% useful, 10% fail, 32 ms. | |
| On A18-class memory (Neo, and anything that <em>is</em> a phone) the 296 MiB first page-in is the hitch, so | |
| Hybrid A, our 35M GSG plus the same highlighter, takes that slot: half the weights, 64% useful, 17% fail against the incumbent’s 54%. | |
| Centroid-only is the 16 MiB emergency path. | |
| The glue is identical everywhere: keep namer words that appear in the body, fill with centroid nouns. | |
| </p> | |
| </section> | |
| <h2>Where each recipe lives</h2> | |
| <table> | |
| <thead> | |
| <tr><th>Machine</th><th>Chip / RAM</th><th>Ship</th><th>Why</th></tr> | |
| </thead> | |
| <tbody> | |
| <tr class="pick"><td>MacBook Pro, current Air</td><td>M-series, 16 GB+</td><td><strong>FLAN + centroid</strong></td><td>Best titles, faster generate, RAM is free</td></tr> | |
| <tr class="mobile"><td>Neo, iPhone-class</td><td>A18 Pro, 8 GB unified</td><td><strong>Hybrid A</strong> (or centroid)</td><td>Half the weights; the first hitch is the 8 GB problem, not the 32 ms</td></tr> | |
| </tbody> | |
| </table> | |
| <p> | |
| A18 Pro is the iPhone 16 Pro SoC. Neo is that phone in a clamshell. | |
| An M4/M5 Air is the same <em>kind</em> of silicon as a Pro, with a 16 GB base, not a different planet. | |
| Do not put Hybrid A on the Air and FLAN on the Pro. That split is <strong>8 GB phone-class vs 16 GB Mac</strong>. | |
| The packaged bundle sizes below decide the final cut; the Hybrid A slim figure is an estimate until the export exists. | |
| </p> | |
| <h2>The holdout table</h2> | |
| <p>Same blinded packet. Latency is single-title on the research Mac (PyTorch MPS generate, CPU centroid, n=50). Combiners run sequentially.</p> | |
| <table> | |
| <thead> | |
| <tr><th>System</th><th class="n">Size</th><th class="n">ms</th><th class="n">Mean</th><th class="n">Useful ≥4</th><th class="n">Fail ≤2</th></tr> | |
| </thead> | |
| <tbody> | |
| <tr><td>12.7M (in the app today)</td><td class="n">28.5 MiB</td><td class="n">226</td><td class="n">3.01</td><td class="n">31%</td><td class="n">54%</td></tr> | |
| <tr><td>Centroid only (ours)</td><td class="n">16 MiB</td><td class="n">0.8</td><td class="n">4.21</td><td class="n">55%</td><td class="n">23%</td></tr> | |
| <tr class="mobile"><td><strong>Hybrid A: phone-class (35M GSG + centroid)</strong></td><td class="n">~150 MiB slim*</td><td class="n">252</td><td class="n">4.72</td><td class="n">64%</td><td class="n">17%</td></tr> | |
| <tr><td>12.7 + GSG + centroid</td><td class="n">300 MiB</td><td class="n">477</td><td class="n">4.70</td><td class="n">64%</td><td class="n">19%</td></tr> | |
| <tr><td>FLAN-T5-small</td><td class="n">296 MiB</td><td class="n">31</td><td class="n">5.22</td><td class="n">68%</td><td class="n">18%</td></tr> | |
| <tr class="pick"><td><strong>FLAN + centroid: 16 GB Mac</strong></td><td class="n">312 MiB</td><td class="n">32</td><td class="n">5.43</td><td class="n">77%</td><td class="n">10%</td></tr> | |
| </tbody> | |
| </table> | |
| <p class="note">Useful = Gemini ≥4. Fail = Gemini ≤2. FLAN+centroid vs raw FLAN, paired wins: 427–374. vs Hybrid A: 449–246. Sealed final 1,000 was not scored. *Slim = GSG weights plus embed-only centroid; measured research checkpoint is 272 MiB, the slim export has not been built yet.</p> | |
| <h2>Size, speed, useful, fail</h2> | |
| <p>X is milliseconds per title. Y is the Gemini rate. Green fill is useful. Red ring is fail. Bubble area is disk.</p> | |
| <svg viewBox="0 0 1000 520" role="img" aria-label="Usefulness and fail versus inference time"> | |
| <line x1="70" y1="20" x2="70" y2="450" stroke="#bbb"/> | |
| <line x1="70" y1="450" x2="970" y2="450" stroke="#bbb"/> | |
| <!-- 0-100% y, 0-520ms x --> | |
| <text x="36" y="24" fill="#888" font-size="12" font-family="Inter,sans-serif">100%</text> | |
| <text x="36" y="236" fill="#888" font-size="12" font-family="Inter,sans-serif">50%</text> | |
| <text x="48" y="448" fill="#888" font-size="12" font-family="Inter,sans-serif">0</text> | |
| <text x="70" y="478" fill="#888" font-size="12" font-family="Inter,sans-serif">0 ms</text> | |
| <text x="250" y="478" fill="#888" font-size="12" font-family="Inter,sans-serif">100</text> | |
| <text x="430" y="478" fill="#888" font-size="12" font-family="Inter,sans-serif">200</text> | |
| <text x="790" y="478" fill="#888" font-size="12" font-family="Inter,sans-serif">400</text> | |
| <text x="500" y="508" fill="#666" font-size="13" font-family="Inter,sans-serif" text-anchor="middle">single-title inference</text> | |
| <!-- fail rings then useful fills. x=70+ms*(900/520), y=450-pct*4.3 --> | |
| <circle cx="71.4" cy="352.8" r="12" fill="none" stroke="#c45b4e" stroke-width="2"/> | |
| <circle cx="71.4" cy="213.1" r="12" fill="#3d8f6e"/> | |
| <text x="88" y="210" font-size="12" font-family="Inter,sans-serif">Centroid</text> | |
| <circle cx="123" cy="371.3" r="28" fill="none" stroke="#c45b4e" stroke-width="2"/> | |
| <circle cx="123" cy="159" r="28" fill="#2f6f8f"/> | |
| <text x="155" y="140" font-size="12" font-family="Inter,sans-serif">FLAN+cen</text> | |
| <circle cx="121" cy="371" r="27" fill="none" stroke="#c45b4e" stroke-width="2" opacity=".5"/> | |
| <circle cx="460" cy="218" r="18" fill="none" stroke="#c45b4e" stroke-width="2"/> | |
| <circle cx="460" cy="317" r="18" fill="#888"/> | |
| <text x="430" y="340" font-size="12" font-family="Inter,sans-serif">12.7M</text> | |
| <circle cx="506" cy="379" r="32" fill="none" stroke="#c45b4e" stroke-width="2"/> | |
| <circle cx="506" cy="175" r="32" fill="#3d8f6e" opacity=".85"/> | |
| <text x="545" y="168" font-size="12" font-family="Inter,sans-serif">Hybrid A</text> | |
| <circle cx="896" cy="370" r="34" fill="none" stroke="#c45b4e" stroke-width="2"/> | |
| <circle cx="896" cy="174" r="34" fill="#888" opacity=".7"/> | |
| <text x="800" y="160" font-size="12" font-family="Inter,sans-serif">triple 477ms</text> | |
| </svg> | |
| <p class="note">FLAN sits left (fast) and high. Our 35M generate sits near 250 ms. The triple is the slowest for a tie with Hybrid A. Centroid is a green speck on the y-axis.</p> | |
| <h2>What we thought we were building</h2> | |
| <p> | |
| PorkiCoder needs a local tab title: a few words so you can find the session later. | |
| The 12.7M student that ships in the app today collapsed to brand priors (<code>Java EE</code>, <code>ASP NET MVC</code>). | |
| That model has a history: it is the distilled student whose training quirks we dissected in our first paper, | |
| <a href="https://porkicoder.com/research/the-sniff-test.html">The Sniff Test</a>, where one lucky embedding transplant looked like a mechanism until 31 matched runs said otherwise. | |
| This campaign started as its replacement. | |
| </p> | |
| <p> | |
| We spent a GPU day on a 35M T5: technical span-corruption pretraining, then PEGASUS-style GSG (task body → original Stack Overflow title). | |
| That model learned a title <em>dialect</em> (<code>How to a varchar field in Active</code>), not a gist. | |
| Exact match to the SO title was never the product metric. | |
| </p> | |
| <p> | |
| The thing that actually wrote stickers was dumb: take the three body words whose embeddings sit nearest the body’s centroid, keep document order. | |
| Sixteen mebibytes if you throw away the unused transformer. | |
| </p> | |
| <h2>How the pieces were formed</h2> | |
| <h3>Data</h3> | |
| <p> | |
| Every system reads the same serialization the app uses: <code>title: agent=… task=…</code> with the task body from a public Stack Overflow dump. | |
| Technical-language pretraining drew 250M tokens from shards 18–57 of that dump. | |
| Evaluation lives in <code>so_clean_holdout_v1</code>: 2,000 newly pulled tasks from shards 0–17 only, with all 49,336 previously used title-training and validation IDs excluded, split by hash sort before any title training. | |
| Half is the reusable clean dev set of 1,000 that every number on this page comes from; the other 1,000 are sealed behind a recorded SHA-256 and have never been generated on, labeled, judged, or read. | |
| </p> | |
| <h3>The models</h3> | |
| <p> | |
| The 35M is a 35,045,888-parameter T5: d_model 512, d_ff 2048, 6 encoder and 3 decoder layers, a 6,985-token compact vocabulary, tied embeddings. | |
| GSG continued from the span-corruption pretrain to step 3,000 on body-to-title pairs. | |
| The FLAN row is not untouched stock: it is FLAN-T5-small (77M, Apache-2.0) after the title fine-tune from run 5 of our earlier distillation campaign, where it served as the capacity reference. | |
| Stock <code>google/flan-t5-small</code> is the same family; the shipping bundle pins one exact checkpoint by SHA-256. | |
| </p> | |
| <h3>The judge</h3> | |
| <p> | |
| Gemini 3.5 Flash-Lite scores blind: it sees the task and anonymous candidate titles in randomized order, reference labels hidden, and rates 1–10 for whether the title would help you find that tab again. | |
| Useful is 4 or above, fail is 2 or below. | |
| An earlier hand-scored 90-task packet ranked systems the same way but with harsher absolute numbers (Hybrid A 56% useful, 12.7M 13%), so trust the ranking and the gaps, not any single percentage. | |
| Latency is the median of 50 single-title runs, generate on Apple MPS, centroid on CPU. | |
| </p> | |
| <h2>What the centroid actually is</h2> | |
| <p> | |
| Not magic, and honestly, not even trained. | |
| The highlighter owns a 512-dimensional embedding table for the 6,985-token vocabulary. | |
| Each content word of the task body (stop words and short tokens dropped) becomes the mean of its subword rows, normalized. | |
| The centroid is the normalized mean of those word vectors. | |
| The title is the two or three words with the highest cosine to the centroid, printed in document order, capitalized. | |
| </p> | |
| <p> | |
| The table is the seed-42 <em>random initialization</em> of the 35M, frozen before any gradient ever touched it. | |
| We tried training embeddings with a cosine objective and they collapsed; the untrained table as a filter is what survived. | |
| So the mechanism is closer to a random projection voting on which words share the page’s token statistics than to learned semantics. | |
| It costs 13.6 MiB of matrix plus 2.3 MiB of tokenizer, runs in 0.8 ms, and beats the 12.7M neural incumbent by 1.2 judge points. | |
| That result is the quiet embarrassment of this whole campaign, and we are shipping it. | |
| </p> | |
| <h2>The glue</h2> | |
| <p> | |
| Hybrid A takes up to two content words from the 35M GSG title that actually appear in the task body (substring matches count for words longer than three characters), deduplicates, then fills to three with centroid nouns. | |
| If the namer contributes nothing that survives the in-body check, the centroid words stand alone. | |
| The same frozen rule with FLAN as the first source is the 16 GB Mac recipe. | |
| The stop list, token regex, and 2+1 fill rule are locked; changing any of them requires a new scored packet. | |
| </p> | |
| <h3>Dead ends, so nobody repeats them</h3> | |
| <ul> | |
| <li>Title cross-entropy from scratch on 50k FLAN-labeled tasks collapsed to one title (<code>Fix XML Files</code>), twice, including once with a Kimi-reviewed recipe. Axis closed.</li> | |
| <li>The span-pretrained 35M learned language, not titles: a blank source beat the correct source on NLL.</li> | |
| <li>Cosine as a loss collapsed embeddings. Cosine as a filter is fine.</li> | |
| <li>Instruction-continuing GSG on extractive FLAN titles gained nothing over locked Hybrid A.</li> | |
| <li>Adding the 12.7M into the glue tied Hybrid A at double the latency and sometimes pasted <code>EE</code> into titles.</li> | |
| </ul> | |
| <h2>Labels, with the input</h2> | |
| <div class="ex"> | |
| <strong>Input.</strong> | |
| <pre>can anybody recommend some really good resources for how to get Apache authenticating users with Kerberos. Background reading on Kerberos would also be useful Thanks Peter</pre> | |
| 12.7: Java EE (2) · Centroid: Anybody Thanks Peter (1) · Hybrid A: Anybody Thanks Peter (1) · <strong>FLAN: KerberosAuthorization (9)</strong> · FLAN+cen keeps Kerberos + a body noun | |
| </div> | |
| <div class="ex"> | |
| <strong>Input.</strong> | |
| <pre>I need a C# profiler. Although I'm not averse to paying for one, something which is free or at least with a trial version would be ideal since it takes time to raise a purchase order. Any recommendations?</pre> | |
| 12.7: Java EE (1) · Hybrid A: C# Ideal Raise (2) · <strong>FLAN: C# Profiler (10)</strong> | |
| </div> | |
| <div class="ex"> | |
| <strong>Input.</strong> | |
| <pre>Does anyone know a method to programmatically close the CD tray on Windows 2000 or higher? Open CD tray exists, but I can't seem to make it close especially under W2k.</pre> | |
| 12.7: Read File Java (1) · Hybrid A: Windows Tray Open (8) · FLAN: Programmatically Close CD Trace (8) · glue: Programmatically Close Tray | |
| </div> | |
| <div class="ex"> | |
| <strong>Input.</strong> | |
| <pre>I'm familiar with SQL Server Indexed Views (or Oracle Materialized Views), we use them in our OLAP applications. They have the really cool feature of being able to usurp an execution plan…</pre> | |
| 12.7: SQL Server Query (6) · Hybrid A: SQL Server Familiar (6) · <strong>FLAN: SQL Server Indexed Views (10)</strong> | |
| </div> | |
| <p> | |
| FLAN names the job. Centroid highlights nouns that are already on the page. | |
| 12.7 invents a stack. Hybrid A is a highlighter with an occasional stack word from GSG. | |
| FLAN+centroid is a namer whose off-page junk gets replaced by nouns. | |
| </p> | |
| <h2>Appendix A: the whole packet, in numbers</h2> | |
| <p> | |
| Summary statistics over all 1,000 clean-dev tasks, one blinded seven-way Gemini call per task. | |
| Gold is a score of 8 or higher. Unique titles counts distinct strings across the 1,000. | |
| Top-title mass is the share of the single most repeated title, the collapse detector that killed earlier recipes. | |
| No system emitted an empty title. W–L is paired wins and losses against FLAN+centroid on the same task; the remainder is ties. | |
| </p> | |
| <div class="wide"><table class="appx"> | |
| <thead> | |
| <tr><th>System</th><th class="n">Mean</th><th class="n">Median</th><th class="n">Useful ≥4</th><th class="n">Fail ≤2</th><th class="n">Gold ≥8</th><th class="n">Unique titles</th><th class="n">Top-title mass</th><th class="n">vs FLAN+cen (W–L)</th></tr> | |
| </thead> | |
| <tbody> | |
| <tr><td>12.7M (in the app today)</td><td class="n">3.01</td><td class="n">2</td><td class="n">30.5%</td><td class="n">53.9%</td><td class="n">3.8%</td><td class="n">318</td><td class="n">8.7%</td><td class="n">158–764</td></tr> | |
| <tr><td>Centroid only</td><td class="n">4.21</td><td class="n">4</td><td class="n">55.1%</td><td class="n">22.6%</td><td class="n">9.0%</td><td class="n">999</td><td class="n">0.2%</td><td class="n">113–536</td></tr> | |
| <tr class="mobile"><td>Hybrid A</td><td class="n">4.72</td><td class="n">5</td><td class="n">64.0%</td><td class="n">16.6%</td><td class="n">12.4%</td><td class="n">997</td><td class="n">0.2%</td><td class="n">246–449</td></tr> | |
| <tr><td>12.7 + GSG + centroid</td><td class="n">4.70</td><td class="n">5</td><td class="n">64.3%</td><td class="n">18.5%</td><td class="n">13.0%</td><td class="n">988</td><td class="n">0.4%</td><td class="n">257–469</td></tr> | |
| <tr><td>FLAN-T5-small</td><td class="n">5.22</td><td class="n">5</td><td class="n">67.7%</td><td class="n">18.3%</td><td class="n">25.2%</td><td class="n">975</td><td class="n">0.6%</td><td class="n">374–427</td></tr> | |
| <tr class="pick"><td>FLAN + centroid</td><td class="n">5.43</td><td class="n">6</td><td class="n">77.4%</td><td class="n">9.9%</td><td class="n">18.4%</td><td class="n">998</td><td class="n">0.2%</td><td class="n"></td></tr> | |
| </tbody> | |
| </table></div> | |
| <p class="note">FLAN keeps the gold medals (25.2% of its titles score 8 or higher, against 18.4% after the glue) yet still loses the pairing 374–427. The glue trades peaks for consistency: median 6, fail under 10%. The 12.7M’s median score is 2.</p> | |
| <h2>Appendix B: thirty tasks, seven labels each</h2> | |
| <p> | |
| The first 30 rows of the scored packet in file order, rescored in one blinded seven-way call so raw 35M GSG can be compared directly. No selection, no reordering. | |
| The number in parentheses is that title’s blind Gemini score on the task. Triple is the 12.7 + GSG + centroid stack. | |
| </p> | |
| <div class="wide"><table class="appx samples"> | |
| <thead> | |
| <tr><th>Task (first words)</th><th>12.7M</th><th>Centroid</th><th>35M GSG</th><th>Hybrid A</th><th>Triple</th><th>FLAN</th><th>FLAN+cen</th></tr> | |
| </thead> | |
| <tbody> | |
| <tr><td class="task">Does anyone know a method to programmatically close the CD tray on Wind…</td><td>Read File Java <span class="s">(2)</span></td><td>Tray Open Especially <span class="s">(3)</span></td><td>WPF: How to a Windows Forms <span class="s">(1)</span></td><td>Windows Tray Open <span class="s">(7)</span></td><td>Windows File Tray <span class="s">(6)</span></td><td>Programmatically Close CD Trace <span class="s">(8)</span></td><td>Programmatically Close Tray <span class="s">(9)</span></td></tr> | |
| <tr><td class="task">How would I disable and later enable all indexes in a given schema/data…</td><td>MySQL Database Table <span class="s">(3)</span></td><td>Enable Make Sqlldr <span class="s">(8)</span></td><td>How do you SQL Server 2008? <span class="s">(2)</span></td><td>SQL Enable Make <span class="s">(5)</span></td><td>SQL Database Enable <span class="s">(7)</span></td><td>Debug SQL Dadurch <span class="s">(1)</span></td><td>SQL Enable Make <span class="s">(5)</span></td></tr> | |
| <tr><td class="task">I have a method which takes params object[] such as: ``` void Foo(param…</td><td>Object Array <span class="s">(4)</span></td><td>Params Arrays Param <span class="s">(8)</span></td><td>How to a struct with in C <span class="s">(1)</span></td><td>Params Arrays Param <span class="s">(8)</span></td><td>Object Params Arrays <span class="s">(9)</span></td><td>Fix Params Error <span class="s">(6)</span></td><td>Params Arrays Param <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">1. What are the patterns you use to determine the frequent queries? 2.…</td><td>JavaScript Code Coverage <span class="s">(2)</span></td><td>Patterns Frequent Factors <span class="s">(9)</span></td><td>How do you select select the of <span class="s">(1)</span></td><td>select Patterns Frequent <span class="s">(9)</span></td><td>select Patterns Frequent <span class="s">(9)</span></td><td>Optimize Routings <span class="s">(4)</span></td><td>Patterns Frequent Factors <span class="s">(9)</span></td></tr> | |
| <tr><td class="task">For the past 10 years or so there have been a smattering of articles an…</td><td>Java EE <span class="s">(2)</span></td><td>Past Years Discussion <span class="s">(5)</span></td><td>a python module <span class="s">(2)</span></td><td>Past Years Discussion <span class="s">(5)</span></td><td>EE Past Years <span class="s">(2)</span></td><td>Research Nature Of Order <span class="s">(8)</span></td><td>Nature Order Past <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">I'm writing an inner loop that needs to place `struct`s in contiguous s…</td><td>Detect Exception Handling <span class="s">(2)</span></td><td>Cost Plus Members <span class="s">(6)</span></td><td>How do Iatees in C++? <span class="s">(3)</span></td><td>Cost Plus Members <span class="s">(6)</span></td><td>Cost Plus Members <span class="s">(6)</span></td><td>Containerize Structures <span class="s">(5)</span></td><td>Structures Cost Plus <span class="s">(6)</span></td></tr> | |
| <tr><td class="task">Using reflection in .Net, what is the differnce between: ``` if (foo.Is…</td><td>Java EE Objects <span class="s">(2)</span></td><td>Reflection Net Differnce <span class="s">(9)</span></td><td>What is a .Net if <span class="s">(3)</span></td><td>Net Reflection Differnce <span class="s">(9)</span></td><td>Net EE Reflection <span class="s">(2)</span></td><td>Fix Fixing Ibar <span class="s">(5)</span></td><td>Ibar Reflection Net <span class="s">(6)</span></td></tr> | |
| <tr><td class="task">I have a C++ tool that walks the call stack at one point. In the code,…</td><td>Java Object Memory <span class="s">(2)</span></td><td>Addrstack Addrframe Addr <span class="s">(9)</span></td><td>How to the name of a RESTful <span class="s">(2)</span></td><td>name Addrstack Addrframe <span class="s">(8)</span></td><td>name Addrstack Addrframe <span class="s">(8)</span></td><td>Fix CLI Blocks <span class="s">(4)</span></td><td>Blocks Addrstack Addrframe <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">How can one detect being in a chroot jail without root privileges? Assu…</td><td>Detect File Download <span class="s">(3)</span></td><td>Low Detection Difficult <span class="s">(5)</span></td><td>How to detect image <span class="s">(2)</span></td><td>detect Low Detection <span class="s">(7)</span></td><td>detect File Low <span class="s">(3)</span></td><td>Detectchroot Proxy <span class="s">(7)</span></td><td>Detectchroot Low Detection <span class="s">(9)</span></td></tr> | |
| <tr><td class="task">This is mostly for fun and to play with what stackoverflow itself is ca…</td><td>JavaScript Software Development <span class="s">(2)</span></td><td>Mostly Fun Translated <span class="s">(8)</span></td><td>What are some good resources for <span class="s">(3)</span></td><td>Mostly Fun Translated <span class="s">(8)</span></td><td>Mostly Fun Translated <span class="s">(8)</span></td><td>Update Taskoverflow <span class="s">(3)</span></td><td>Update Mostly Fun <span class="s">(4)</span></td></tr> | |
| <tr><td class="task">The exact error is as follows > Could not load file or assembly 'Micros…</td><td>Detect File Upload <span class="s">(2)</span></td><td>Culture Neutral Month <span class="s">(5)</span></td><td>How do I use a filename? <span class="s">(3)</span></td><td>filename Culture Neutral <span class="s">(6)</span></td><td>filename File Culture <span class="s">(6)</span></td><td>Fix Sql Format <span class="s">(7)</span></td><td>Sql Format Culture <span class="s">(6)</span></td></tr> | |
| <tr><td class="task">Does anyone know where online copies of the old articles can be found?…</td><td>Java Web App <span class="s">(2)</span></td><td>Online Copies Subject <span class="s">(7)</span></td><td>How do you handleeds in the <span class="s">(1)</span></td><td>Online Copies Subject <span class="s">(7)</span></td><td>Online Copies Subject <span class="s">(7)</span></td><td>Old Articles <span class="s">(8)</span></td><td>Old Articles Online <span class="s">(9)</span></td></tr> | |
| <tr><td class="task">How can I find the high water mark (the historical maximum number of co…</td><td>Javascript Arrays <span class="s">(2)</span></td><td>Water Historical Concurrent <span class="s">(5)</span></td><td>How do I get the current number <span class="s">(3)</span></td><td>current number Water <span class="s">(4)</span></td><td>current number Water <span class="s">(4)</span></td><td>Find High Water Mark <span class="s">(10)</span></td><td>Find High Water <span class="s">(9)</span></td></tr> | |
| <tr><td class="task">How do I make it so `mailto:` links will be registered with my program?…</td><td>WPF Control Control <span class="s">(2)</span></td><td>Mailto Registered Independent <span class="s">(9)</span></td><td>How do I get the current of the <span class="s">(2)</span></td><td>Mailto Registered Independent <span class="s">(9)</span></td><td>Mailto Registered Independent <span class="s">(9)</span></td><td>EmailTo Link Register <span class="s">(7)</span></td><td>EmailTo Link Mailto <span class="s">(7)</span></td></tr> | |
| <tr><td class="task">I started an application in Google App Engine right when it came out, t…</td><td>Java EE Application <span class="s">(2)</span></td><td>Technology Pet Www <span class="s">(3)</span></td><td>How do you use a based on <span class="s">(2)</span></td><td>Technology Pet Www <span class="s">(3)</span></td><td>EE Technology Pet <span class="s">(2)</span></td><td>Google App Engine Issues <span class="s">(8)</span></td><td>Google App Technology <span class="s">(7)</span></td></tr> | |
| <tr><td class="task">A poorly-written back-end system we interface with is having trouble wi…</td><td>WCF Service Service <span class="s">(3)</span></td><td>Poorly Continues Submission <span class="s">(6)</span></td><td>How to a user in my application? <span class="s">(2)</span></td><td>user Poorly Continues <span class="s">(6)</span></td><td>user Service Poorly <span class="s">(6)</span></td><td>Backend Loading <span class="s">(7)</span></td><td>Backend Loading Poorly <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">This query works great: ``` var pageObject = (from op in db.ObjectPermi…</td><td>Query LINQ SELECT <span class="s">(7)</span></td><td>Pageobject Page Pagename <span class="s">(8)</span></td><td>How do I use a in ASP.NET <span class="s">(2)</span></td><td>Pageobject Page Pagename <span class="s">(8)</span></td><td>Query Pageobject Page <span class="s">(9)</span></td><td>Single OrDefault <span class="s">(4)</span></td><td>Single OrDefault Pageobject <span class="s">(7)</span></td></tr> | |
| <tr><td class="task">If I want to inject a globally scoped array variable into a page's clie…</td><td>JQuery Array Objects <span class="s">(3)</span></td><td>Inject Globally Populate <span class="s">(7)</span></td><td>How can I thehmap from an array <span class="s">(2)</span></td><td>can array Inject <span class="s">(7)</span></td><td>can array Inject <span class="s">(7)</span></td><td>Globalized Array Variable <span class="s">(7)</span></td><td>Array Variable Inject <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">I have been looking at metrics for [coupling](http://www.ibm.com/develo…</td><td>JavaScript UI Design <span class="s">(2)</span></td><td>Metrics Wikipedia Feel <span class="s">(5)</span></td><td>What is the way to a java. <span class="s">(2)</span></td><td>java Metrics Wikipedia <span class="s">(3)</span></td><td>java JavaScript Metrics <span class="s">(2)</span></td><td>Logic Models <span class="s">(3)</span></td><td>Metrics Wikipedia Feel <span class="s">(5)</span></td></tr> | |
| <tr><td class="task">I'm about to start on a large Qt application, which is made up of small…</td><td>ASP Net Application <span class="s">(2)</span></td><td>Smaller Developed Integrated <span class="s">(5)</span></td><td>.NET - How to use a subversion <span class="s">(2)</span></td><td>Smaller Developed Integrated <span class="s">(5)</span></td><td>Application Smaller Developed <span class="s">(5)</span></td><td>Create Qt Sub Folder <span class="s">(9)</span></td><td>Create Qt Smaller <span class="s">(6)</span></td></tr> | |
| <tr><td class="task">I have written some code in my VB.NET application to send an HTML e-mai…</td><td>NET Framework Framework <span class="s">(3)</span></td><td>Lost Spam Badly <span class="s">(6)</span></td><td>.NET Code to:: tests <span class="s">(2)</span></td><td>NET Code Lost <span class="s">(5)</span></td><td>NET Code Lost <span class="s">(5)</span></td><td>Fix Email Spacing <span class="s">(8)</span></td><td>Email Lost Spam <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">We are using classic asp to call a C# dll and in the C# dll we are usin…</td><td>Object List <span class="s">(2)</span></td><td>Printing Printersettings Administrators <span class="s">(7)</span></td><td>How do I a list in C#? <span class="s">(2)</span></td><td>list C# Printing <span class="s">(7)</span></td><td>list C# Printing <span class="s">(7)</span></td><td>List Printers <span class="s">(8)</span></td><td>List Printers Printing <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">In the uncompressed situation I know I need to read the wav header, pul…</td><td>ASP Net Session <span class="s">(2)</span></td><td>Rate Samples Simpler <span class="s">(5)</span></td><td>.net framework library for vs.ex <span class="s">(3)</span></td><td>net framework Rate <span class="s">(5)</span></td><td>net framework Rate <span class="s">(5)</span></td><td>Workflow Automation <span class="s">(3)</span></td><td>Workflow Rate Samples <span class="s">(4)</span></td></tr> | |
| <tr><td class="task">When running one of our software, a tester was faced with the data exec…</td><td>Java EE Application <span class="s">(2)</span></td><td>Software Purposes Kill <span class="s">(8)</span></td><td>How to in Windows <span class="s">(2)</span></td><td>Windows Software Purposes <span class="s">(7)</span></td><td>Windows Application Software <span class="s">(5)</span></td><td>Debug Deploy <span class="s">(3)</span></td><td>Debug Software Purposes <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">Without getting a degree in information retrieval, I'd like to know if…</td><td>Javascript Objects <span class="s">(2)</span></td><td>Degree Feel Attempted <span class="s">(3)</span></td><td>ASP.NET MVC - How to use <span class="s">(2)</span></td><td>NET Degree Feel <span class="s">(3)</span></td><td>NET Degree Feel <span class="s">(3)</span></td><td>Textfrequency Sentiment <span class="s">(7)</span></td><td>Textfrequency Degree Feel <span class="s">(7)</span></td></tr> | |
| <tr><td class="task">I know I can do this: ``` IDateTimeFactory dtf = MockRepository.Generat…</td><td>ArrayList Objects <span class="s">(2)</span></td><td>Arbitrary Minute Ditto <span class="s">(7)</span></td><td>C# - How to return a value from <span class="s">(3)</span></td><td>return value Arbitrary <span class="s">(6)</span></td><td>return value Arbitrary <span class="s">(6)</span></td><td>Fix DateTime <span class="s">(8)</span></td><td>DateTime Arbitrary Minute <span class="s">(6)</span></td></tr> | |
| <tr><td class="task">I'm developing a compiler framework for .NET and want a flexible way of…</td><td>NET Framework Development <span class="s">(6)</span></td><td>Developing Involved Goals <span class="s">(6)</span></td><td>.NET Framework/ <span class="s">(6)</span></td><td>NET Framework Developing <span class="s">(6)</span></td><td>NET Framework Developing <span class="s">(6)</span></td><td>Pipeline Compatibility <span class="s">(4)</span></td><td>Pipeline Developing Involved <span class="s">(7)</span></td></tr> | |
| <tr><td class="task">can anybody recommend some really good resources for how to get Apache…</td><td>Java EE <span class="s">(2)</span></td><td>Anybody Thanks Peter <span class="s">(2)</span></td><td>How do you your from to <span class="s">(2)</span></td><td>Anybody Thanks Peter <span class="s">(2)</span></td><td>Anybody Thanks Peter <span class="s">(2)</span></td><td>KerberosAuthorization <span class="s">(10)</span></td><td>KerberosAuthorization Anybody Thanks <span class="s">(9)</span></td></tr> | |
| <tr><td class="task">I've been quite used to working on small projects which I coded with 1,…</td><td>Java EE <span class="s">(2)</span></td><td>Organization Keeping Organized <span class="s">(8)</span></td><td>What is a good for programming? <span class="s">(2)</span></td><td>programming Organization Keeping <span class="s">(6)</span></td><td>programming EE Organization <span class="s">(3)</span></td><td>Organizational Issues <span class="s">(7)</span></td><td>Organizational Organization Keeping <span class="s">(8)</span></td></tr> | |
| <tr><td class="task">I'm playing around with a native (non-web) single-player game I'm writi…</td><td>Detect Plugin Script <span class="s">(3)</span></td><td>Playing Discourage Players <span class="s">(7)</span></td><td>How do you your recursive <span class="s">(2)</span></td><td>Playing Discourage Players <span class="s">(7)</span></td><td>Playing Discourage Players <span class="s">(7)</span></td><td>multiplayer multiplayer <span class="s">(2)</span></td><td>multiplayer Playing Discourage <span class="s">(5)</span></td></tr> | |
| </tbody> | |
| </table></div> | |
| <h2>Cost, honestly</h2> | |
| <p> | |
| FLAN-T5-small is T5-small trained on C4 at Google, then instruction-tuned on more than 1,000 FLAN tasks on TPU v3/v4 pods. | |
| The 296 MiB file is that entire stack, Apache-2.0. It is not a weekend 77M. | |
| Our 35M pretrain, GSG, and all judging ran on one rented RTX 6000 Ada at $1.57/hr plus a few dollars of Gemini calls: tens of dollars total. | |
| Hybrid A is close <em>for a pretrain</em>. It is not a peer of FLAN as a recipe. | |
| It <em>is</em> the model that fits next to an A18 without pretending we have a Pro’s memory. | |
| The 16 GB Mac borrows the summarizer and keeps the highlighter. | |
| Phone-class silicon keeps the highlighter and the 35M we trained. | |
| </p> | |
| <p> | |
| One rule changed to say this out loud: our evaluation preregistration stated that FLAN was a benchmark only and this project would ship only its own model. | |
| That rule is overridden by an explicit owner decision, recorded here. | |
| The shipped contribution is the centroid table plus the in-body glue, which cut FLAN’s fail rate from 18% to 10% and lifted useful from 68% to 77%. | |
| We are not marketing a from-scratch 35M win, because there is not one. | |
| </p> | |
| <h2>What we are not claiming</h2> | |
| <ul> | |
| <li>We did not beat FLAN by training a better 35M.</li> | |
| <li>The centroid table is untrained. No claim of learned semantics, only a filter that works.</li> | |
| <li>The FLAN numbers come from our run-5 title-tuned descendant of FLAN-T5-small, not the untouched Hugging Face file. The family is the same; the exact shipped checkpoint gets pinned by hash.</li> | |
| <li>We did not score the sealed 1,000. Clean-dev 1,000 is continuity, not the final gate.</li> | |
| <li>Gemini is more generous than our hand rubric. Rankings held across packets; absolute percents move when FLAN sits in the same row.</li> | |
| <li>The Hybrid A slim bundle size and the phone-class memory behavior are projections until the export is built and measured on device.</li> | |
| </ul> | |
| <h2>Ship path</h2> | |
| <p> | |
| Two bundles, one combiner, pick the namer by RAM. Not a third model. | |
| </p> | |
| <ul> | |
| <li><code>resources/tab-namer/flan-centroid-v1/</code>: default on 16 GB+ Mac.</li> | |
| <li><code>resources/tab-namer/hybrid-a-v1/</code>: slim 35M GSG + centroid; default on A18 / 8 GB; fallback under memory pressure.</li> | |
| <li>Centroid-only if the OS is jetsamming.</li> | |
| <li>Directory name if nothing validates.</li> | |
| </ul> | |
| <p> | |
| New workers. Do not reuse the 12.7 ONNX graph (2 decoder layers, 6,985-id vocab) for FLAN, which needs its full 32k vocabulary and 8+8 layers. | |
| Sanitize into the 1–3 word <code>[A-Za-z0-9 ]</code> contract before rejecting, and fall back to centroid if sanitizing empties the title. | |
| First release runs in shadow mode: compute, log, keep showing the current title. Then switch on. | |
| Details: <code>FLAN_CENTROID_HANDOFF.md</code> in the PorkiCoder checkout. | |
| </p> | |
| </div> | |
| </body> | |
| </html> | |