Building Moon Bot: A Slack-Native Coding Agent Backed by HuggingFace Buckets
• 52
AI & ML interests
The AI community building the future.
Recent Activity
Papers
Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination
Seeing the Needle in the Haystack: Towards Weakly-Supervised Log Instance Anomaly Localization via Counterfactual Perturbation
Articles
dacorvo
updated a
bucket 12 minutes ago
lysandre
updated a
dataset about 7 hours ago
nielsr
updated a
Space about 8 hours ago
nielsr
updated a
Space about 9 hours ago
nielsr
updated a
bucket about 13 hours ago
sayakpaul
updated a
dataset about 14 hours ago
tarekziade
updated a
bucket about 16 hours ago
alvarobartt
updated a
dataset about 16 hours ago
victor
updated a
dataset about 24 hours ago
[FEEDBACK] Daily Papers
🔥❤️ 21
215
#32 opened about 2 years ago
by 
kramp
nico-martin
updated a
dataset 1 day ago
ehcalabres
updated a
dataset 1 day ago
sergiopaniego
posted an update 4 days ago
Post
329
catching up on some bookmarked reads from the summer, reading Antidoom from @liquidai
small reasoning models get stuck more easily when the task involves a long thinking trace and a hard problem. It starts repeating the same word over and over again ("Wait", "Alternatively"…), each repetition makes the next one likelier, and the generation is spent before it reaches an answer
they measured it, 10.2% of completions for an early LFM2.5-2.6B checkpoint and 22.9% for Qwen3.5-4B at greedy. After training those drop to 1.4% and 1.0%
the fix is FTPO (final token preference optimization). What I like is how narrow it is, it only touches the single token where the loop starts
three ways it differs from DPO:
> trains one token position, mid-generation, instead of whole sequences
> spreads probability across ~20 plausible alternatives instead of swapping one overtrained token for another
> keeps the regularizer in logit space, no softmax, so the rest of the vocabulary stays put
the third one is what makes it usable. If you want to edit one position without disturbing the model, you can't have a loss that reshuffles the other 150k logits on the way
and their explanation abt the result: the training teaches the model nothing new about math or code, it clears the failure mode that was blocking answers the model could already produce
full blog > https://www.liquid.ai/blog/antidoom
FTPO itself comes from Antislop, where it was built to strip overused phrasing. LiquidAI retargeted it to doom loops
and under the hood it's a subclass of TRL's DPOTrainer with compute_loss overridden, around 90 lines of loss and no new trainer
we documented that pattern in TRL's docs
https://huggingface.co/docs/trl/main/en/customization#change-the-training-objective
small reasoning models get stuck more easily when the task involves a long thinking trace and a hard problem. It starts repeating the same word over and over again ("Wait", "Alternatively"…), each repetition makes the next one likelier, and the generation is spent before it reaches an answer
they measured it, 10.2% of completions for an early LFM2.5-2.6B checkpoint and 22.9% for Qwen3.5-4B at greedy. After training those drop to 1.4% and 1.0%
the fix is FTPO (final token preference optimization). What I like is how narrow it is, it only touches the single token where the loop starts
three ways it differs from DPO:
> trains one token position, mid-generation, instead of whole sequences
> spreads probability across ~20 plausible alternatives instead of swapping one overtrained token for another
> keeps the regularizer in logit space, no softmax, so the rest of the vocabulary stays put
the third one is what makes it usable. If you want to edit one position without disturbing the model, you can't have a loss that reshuffles the other 150k logits on the way
and their explanation abt the result: the training teaches the model nothing new about math or code, it clears the failure mode that was blocking answers the model could already produce
full blog > https://www.liquid.ai/blog/antidoom
FTPO itself comes from Antislop, where it was built to strip overused phrasing. LiquidAI retargeted it to doom loops
and under the hood it's a subclass of TRL's DPOTrainer with compute_loss overridden, around 90 lines of loss and no new trainer
we documented that pattern in TRL's docs
https://huggingface.co/docs/trl/main/en/customization#change-the-training-objective
- 1 reply
eliebak
authored 2
papers 7 days ago
sergiopaniego
posted an update 12 days ago
Post
229
super interesting new paper from Microsoft "Agent Lightning v1.0: Towards Harnessed Agentic RL" by Zhiyuan He et al.
same idea we've seen already several times: you train the agent inside the real harness it ships with, instead of a reimplementation of it
now that recipe has a name → harnessed agentic RL
paper: huggingface.co/papers/2608.17528
the tricky bit they nail down: one rollout is not one training sample
the harness calls the model many times, so a single episode → a variable number of (prompt, response) rows
you don't even know the batch size until the episode finishes running
its real contribution is being first to systematically map the four problems that fall out of that:
> retokenization + sample merging
> advantage calculation over a variable sample count
> loss normalization at the rollout level, not per sample
> backend scheduling when the batch size is dynamic
and it actually works → plain RL inside the real harness, no reimplementation
Qwen3.5-9B on SWE-bench Verified 41.8 → 56.4 (+14.6), with only ~6k examples
the whole thing is ~3,500 lines, any harness, self-hosted k8s
from our side, we've shared some materials on the same line you may want to check out :)
> Agentic RL: Token-In, Token-Out Done Right: https://huggingface.co/blog/huggingface/tito
> a full worked example, opencode owning its loop trained with GRPO: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training
> Harness, Scaffold, and the AI Agent Terms Worth Getting Right: https://huggingface.co/blog/agent-glossary
on a similar line:
https://x.com/SergioPaniego/status/2062911580564496576
same idea we've seen already several times: you train the agent inside the real harness it ships with, instead of a reimplementation of it
now that recipe has a name → harnessed agentic RL
paper: huggingface.co/papers/2608.17528
the tricky bit they nail down: one rollout is not one training sample
the harness calls the model many times, so a single episode → a variable number of (prompt, response) rows
you don't even know the batch size until the episode finishes running
its real contribution is being first to systematically map the four problems that fall out of that:
> retokenization + sample merging
> advantage calculation over a variable sample count
> loss normalization at the rollout level, not per sample
> backend scheduling when the batch size is dynamic
and it actually works → plain RL inside the real harness, no reimplementation
Qwen3.5-9B on SWE-bench Verified 41.8 → 56.4 (+14.6), with only ~6k examples
the whole thing is ~3,500 lines, any harness, self-hosted k8s
from our side, we've shared some materials on the same line you may want to check out :)
> Agentic RL: Token-In, Token-Out Done Right: https://huggingface.co/blog/huggingface/tito
> a full worked example, opencode owning its loop trained with GRPO: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training
> Harness, Scaffold, and the AI Agent Terms Worth Getting Right: https://huggingface.co/blog/agent-glossary
on a similar line:
https://x.com/SergioPaniego/status/2062911580564496576
Post
3730
🚨 I've just published Sentence Transformers v6.0, introducing MultiVectorEncoder: ColBERT-style late interaction models are now a fourth model type, for training, inference, and interpretation, alongside the dense, sparse, and reranker models! Details:
Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, charts and tables included, with no OCR step in between.
Any PyLate, Stanford ColBERT, or ColPali checkpoint loads straight into the same familiar API: model.encode_query(), model.encode_document(), and model.similarity() just work, whether the documents are texts or page images.
Does it help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, and the multi-vector model wins on 9 of the 13 NanoBEIR datasets: 0.6868 vs 0.6764 mean NDCG@10. The price is a bigger index, and the new HierarchicalTokenPooling module halves it at roughly no retrieval cost.
Antoine Chaffin, Raphaël Sourty, and I wrote a blog post walking through multi-vector models in practice: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multi-vector-encoder
pip install sentence-transformers==6.0.0
Release notes: https://github.com/huggingface/sentence-transformers/releases/tag/v6.0.0
Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, charts and tables included, with no OCR step in between.
Any PyLate, Stanford ColBERT, or ColPali checkpoint loads straight into the same familiar API: model.encode_query(), model.encode_document(), and model.similarity() just work, whether the documents are texts or page images.
Does it help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, and the multi-vector model wins on 9 of the 13 NanoBEIR datasets: 0.6868 vs 0.6764 mean NDCG@10. The price is a bigger index, and the new HierarchicalTokenPooling module halves it at roughly no retrieval cost.
Antoine Chaffin, Raphaël Sourty, and I wrote a blog post walking through multi-vector models in practice: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multi-vector-encoder
pip install sentence-transformers==6.0.0
Release notes: https://github.com/huggingface/sentence-transformers/releases/tag/v6.0.0
Post
1930
Hugging Face MCP Server v0.4.9
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Attach images from hf_fs tool.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Attach images from hf_fs tool.
sergiopaniego
posted an update 22 days ago
Post
705
Something I really like when I study a subject is understanding its history, how it reached the point where it is today
I did that exercise for RL in post-training: from RLHF and PPO, to verifiable rewards, to the GRPO family of variants, to agents acting in environments. Everything is backed by what the labs themselves say in their public reports (DeepSeek, Qwen, Kimi, GLM-5, Nemotron, Mistral and more), in their own words
This is the companion piece to Class 3 of our Training Agents series with @burtenshaw . The class explains how GRPO works, with three hands-on experiments. The article shows where the same ideas appear at frontier scale
https://huggingface.co/blog/sergiopaniego/agentic-rl-2026
I did that exercise for RL in post-training: from RLHF and PPO, to verifiable rewards, to the GRPO family of variants, to agents acting in environments. Everything is backed by what the labs themselves say in their public reports (DeepSeek, Qwen, Kimi, GLM-5, Nemotron, Mistral and more), in their own words
This is the companion piece to Class 3 of our Training Agents series with @burtenshaw . The class explains how GRPO works, with three hands-on experiments. The article shows where the same ideas appear at frontier scale
https://huggingface.co/blog/sergiopaniego/agentic-rl-2026
- 1 reply
sergiopaniego
posted an update 27 days ago
Post
296
we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv"
you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced
and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine
the loop:
- OpenCode owns its tool loop inside an OpenEnv sandbox
- an in-sandbox proxy records the real token ids + logprobs, per turn
- a hidden-test verifier scores the result, and that is the reward
- TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL
blog + runnable example: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training
you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced
and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine
the loop:
- OpenCode owns its tool loop inside an OpenEnv sandbox
- an in-sandbox proxy records the real token ids + logprobs, per turn
- a hidden-test verifier scores the result, and that is the reward
- TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL
blog + runnable example: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training
- 3 replies