Issue with Agentic Coding / Tool Calling - Model Hangs
Hi everyone,
I'm trying to use this model (Qwen3.6-27B-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS) for multi-turn agentic coding tasks, but I'm consistently running into issues. The model either hangs completely (generation throughput drops to 0.0 tokens/s) during repeated requests and tool-calling scenarios.
Here is my current setup:
Hardware: Single RTX 5090 (32GB VRAM, sm_120)
Deployment: vLLM (v0.20.1) via Docker (vllm/vllm-openai:latest)
Quantization: ModelOpt NVFP4 weights + FP8 KV-cache
Key Parameters: gpu-memory-utilization=0.85, max-num-seqs=4, max-num-batched-tokens=4096.
Agent Settings: --enable-auto-tool-choice, --tool-call-parser qwen3_coder, --reasoning-parser qwen3
In an attempt to fix this, I've already tried:
Lowering gpu-memory-utilization (from 0.90 to 0.85) to leave more VRAM for PyTorch/FlashInfer workspaces to avoid OOMs during autotuning.
Disabling the FlashInfer sampler (VLLM_USE_FLASHINFER_SAMPLER=0).
Reducing max-num-batched-tokens to prevent activation memory spikes during chunked prefill.
Setting VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
Despite these adjustments, the model still stops generating tokens after a few agentic iterations and eventually just freezes.
Is there a known issue with using this specific NVFP4 quantized version for agentic coding/tool-calling workflows? Or is there something specific in my vLLM configuration causing these hangs when parsing tools?
Any guidance or workaround would be greatly appreciated!
Have not seen this issue reported in the wild by anyone using it, nor have I encountered this problem using this model almost exclusively for the past few weeks. If you have some event logs that point to errors related to the issue you are seeing we can take a look. If you are using an agent gateway often the issue is a missing tool parser in the gateway json or configuration on the Agent Gateway itself.
Well I run into these issues and nvfp4 quants are not usable in opencode or openclaw. It fails silently. Falls into infinite loops as well. That is just a limitation of this model. FP8 is the lowest quant usable. at nvfp4 quant it halucinaces so much it breaks tool and thinking parsers
Nothing wrong with NVFP4 it's a nearly lossless quantization. I think you need to use flashinfer and you need to make sure cud graphs are fully computed. You need to make sure CUTLASS is use for hardware acceleration etc. NVFP4 is optimal on Blackwell architecture. NVFP4 operates better than most standard FP8 quants out there but greatly reduces vram requirements.
Have you done any benchmark and direct comparisons?
The uncensored model and the qwen3_coder tool parser are both broken. (Test: let it write "</parameter>" into a file. See it struggle.) The problem is not nvfp4, the "normal" aeon model works just fine with a patched qwen3_coder tool parser and performs cognitively about at the same level as a Sonnet 4.6. The uncensored model is brain-damaged beyond a very short initial context.
All of the Aeon models are uncensored, but the MTP-XS one has the most degradation due to some of the attention layers also being quantized. Recommended best option is the -MTP or just -NVFP4 options (both still uncensored fully). Also I've used this model extensively with an agentic harness and the tool parser was working fine, it does require both configuration on the vllm side AND the agent harness which can be a common miss.
Sorry, my bad, I realized I was using your container but the PrismaSCOUT model in nfpv4. To be more precise: I am having much more stable reasoning results running rdtand/Qwen3.6-27B-PrismaSCOUT-Blackwell-NVFP4-BF16-vllm on your container ( ghcr.io/aeon-7/aeon-vllm-ultimate:2026-06-18-v0.23.0-dflashfix ) than with your uncensored NVFP4 model (AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP) on any of the four served endpoints, using the "Jan" frontend. The reasoning and generation output degrades for me massively on the model's second reply and ends up becoming utterly incoherent, as if the repetition penalty was set too high (even though it is 1.0). Just an FYI/heads-up. (Haven't tried yours in Claude Code, but PrismaSCOUT in your container is my daily driver in Claude Code, see above. It has fixed bugs introduced by Fable 5, twice; so both the base model, your container and NVFP4 quant are sound; the abliteration process might not be. )