unknown speculative decoding type without draft model
I'm getting this ik_llama.cpp error and cannot run the model with mtp spec dec using your drafter:
unknown speculative decoding type without draft model
ik_llama.cpp built today (2025-05-07), do I have to use any specific branch? Could you add instructions about how to successfully run your drafter? Thx.
Yes, I forgot to mention in the description that these versions currently only work in open PR 1744, so you should build on top of them first.
I've try to make E4B assistant with your instructions but when I use it hangs on 271 output token , i get Nice 2x speed increase until 200 token output, please can you help me to figure this stuck on 271st token. Ik_llama of course the best for my A16
@agatazit , thank you for pointing that out. There is a specific bug in E4B that causes the model to freeze during runtime. I am analyzing the issue to find the best way to fix it. I will let you know when I submit a PR for you to test.
great and thanks for your response. I'll follow you on github ik_llama.cpp .