EmbeddingGemma-2-GGUF / reproducibility /imatrix-gemma-embedding.patch
ngquocvinh's picture
Publish first validated EmbeddingGemma 2 GGUF artifacts (Q8_0 and BF16 projector)
cf78aeb verified
Raw History Blame Contribute Delete
696 Bytes
diff --git a/tools/imatrix/imatrix.cpp b/tools/imatrix/imatrix.cpp
index 5e9f911e0..d7c531fd8 100644
--- a/tools/imatrix/imatrix.cpp
+++ b/tools/imatrix/imatrix.cpp
@@ -1426,7 +1426,9 @@ static bool compute_imatrix(llama_context * ctx, const common_params & params, c
const bool add_bos = llama_vocab_get_add_bos(vocab);
- if (llama_pooling_type(ctx) != LLAMA_POOLING_TYPE_LAST) {
+ // Embedding models may append EOS while using mean pooling. This is valid
+ // for activation collection, which does not require next-token PPL.
+ if (params.compute_ppl && llama_pooling_type(ctx) != LLAMA_POOLING_TYPE_LAST) {
GGML_ASSERT(!llama_vocab_get_add_eos(vocab));
}