Activity Feed

AI & ML interests

None defined yet.

Recent Activity

BramVanroyย 
posted an update 22 days ago
view post
Post
2687
**I benchmarked HF buckets against https access for Common Crawl.**

Took me a while to get round to do this but I benchmarked access to Common Crawl via https vs hf buckets. Both experiments were run at night in Europe. I do not think other hardware problems were impacting the speeds since CPU processing time of the non-download pipeline components were highly similar (within 2% identical) and below only the WarcReader speeds of datatrove are used.

Experiment: selected 5 disjoint samples of 64 files each (randomly from the latest crawl; 20,499 docs/file). Those five batches were then processed by 32 single-core tasks with 4GB/core (five batches to calculate CIs). Paired experiment between using https and hf bucket.

- https: 40.0 [39.3-40.6] (seconds per WARC file)
- hf bucket: 172.0 [122.7-221.2]

That is a difference of about 4x in streaming speed. You'll see that https is also more stable (smaller CI).

I also ran raw throughput tests to the endpoints to measure rate limiting (64MiB transfer at 8/32/128/256 concurrent readers) and rate limiting seems not an issue for either: at any of those parallel reader numbers, their respective speeds stay about the same.

Note that, given CC scale, this is still a small test. Rate limiting may become more obvious when processing a full crawl. I do not know whether the https endpoint vs HF bucket will shut you out earlier with which limits.
mlabonneย 
posted an update 27 days ago
view post
Post
3852
Deploy local agents everywhere with LiquidAI/LFM2.5-2.6B

Unlike agents that depend on cloud APIs, local agents give you free inference, low latency, and real privacy.

Removing the per-token cost changes how developers build: agents can now be massively parallelized on local hardware, running background tasks that burn through millions of tokens at no marginal cost!
  • 7 replies
ยท
Abhaykoulย 
posted an update 3 months ago
view post
Post
405
Shipped v0.1.2 of vtx โ€” a minimalist coding agent for the terminal.

Most agentic CLIs ship 10k+ token system prompts. Vtx is ~2,200. Less prompt overhead means more room for your code in the model's context window.

Vtx is a from-scratch Python implementation of the design philosophy behind pi-mono โ€” same principles, pure Python, no transpiled runtime.

What ships out of the box:

โ†’ Textual TUI + headless CLI (vtx -p "fix the failing test")
โ†’ 49 LLM provider gateways, all declared in a single provider.yaml
โ†’ 5 core tools (read / edit / write / bash / find) plus web search and fetch
โ†’ Session tree with compaction, handoff, and resume
โ†’ AGENTS.md / CLAUDE.md auto-discovery
โ†’ Skills system โ€” drop SKILL.md files in .agents/skills/ and they become slash commands
โ†’ Two OAuth flows (GitHub Copilot device flow, OpenAI Codex PKCE)
โ†’ Two-mode permissions: prompt (default) or auto, with a safe-command allowlist

This release adds a proper extension system. Register new LLM-callable tools, intercept tool calls, hook lifecycle events, and add slash commands from a single register(api) function in a Python file under ~/.vtx/agent/extensions/. Extensions can override built-in tools by name and chain handler logic across subscribers.

Apache 2.0. uv tool install vtx-coding-agent and you're running.

GitHub: https://github.com/OEvortex/vtx-coding-agent
PyPI: https://pypi.org/project/vtx-coding-agent

Built in the open. Feedback, extensions, and PRs welcome.
hiyougaย 
posted an update 3 months ago
view post
Post
1059
Follow my X account โ€” I'll be sharing thoughts and findings on building open-source AI Agent projects, Agent Memory, and Observability.

Thanks for connecting!

https://x.com/code_hiyouga