Feature Extraction
GGUF
sentence-transformers
llama.cpp
embedding
multimodal
multilingual
quantized
imatrix
Instructions to use ngquocvinh/EmbeddingGemma-2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use ngquocvinh/EmbeddingGemma-2-GGUF with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("ngquocvinh/EmbeddingGemma-2-GGUF") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ngquocvinh/EmbeddingGemma-2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf ngquocvinh/EmbeddingGemma-2-GGUF:Q4_K_M --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf ngquocvinh/EmbeddingGemma-2-GGUF:Q4_K_M --embeddings
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible embeddings server: ./llama-server -hf ngquocvinh/EmbeddingGemma-2-GGUF:Q4_K_M --embeddings
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible embeddings server: ./build/bin/llama-server -hf ngquocvinh/EmbeddingGemma-2-GGUF:Q4_K_M --embeddings
- LM Studio
- Jan
- Ollama
How to use ngquocvinh/EmbeddingGemma-2-GGUF with Ollama:
ollama run hf.co/ngquocvinh/EmbeddingGemma-2-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use ngquocvinh/EmbeddingGemma-2-GGUF with Docker Model Runner:
docker model run hf.co/ngquocvinh/EmbeddingGemma-2-GGUF:Q4_K_M
- Lemonade
How to use ngquocvinh/EmbeddingGemma-2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ngquocvinh/EmbeddingGemma-2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.EmbeddingGemma-2-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Download reproducibility/imatrix-gemma-embedding.patch from ngquocvinh/EmbeddingGemma-2-GGUF: direct link, hf CLI and curl.
- Browser
- Download file 696 Bytes
-
https://huggingface.co/ngquocvinh/EmbeddingGemma-2-GGUF/resolve/main/reproducibility/imatrix-gemma-embedding.patch
- Command line
-
hf download hf://ngquocvinh/EmbeddingGemma-2-GGUF/reproducibility/imatrix-gemma-embedding.patch
-
curl -L -o imatrix-gemma-embedding.patch https://huggingface.co/ngquocvinh/EmbeddingGemma-2-GGUF/resolve/main/reproducibility/imatrix-gemma-embedding.patch
696 Bytes
| diff --git a/tools/imatrix/imatrix.cpp b/tools/imatrix/imatrix.cpp | |
| index 5e9f911e0..d7c531fd8 100644 | |
| --- a/tools/imatrix/imatrix.cpp | |
| +++ b/tools/imatrix/imatrix.cpp | |
| static bool compute_imatrix(llama_context * ctx, const common_params & params, c | |
| const bool add_bos = llama_vocab_get_add_bos(vocab); | |
| - if (llama_pooling_type(ctx) != LLAMA_POOLING_TYPE_LAST) { | |
| + // Embedding models may append EOS while using mean pooling. This is valid | |
| + // for activation collection, which does not require next-token PPL. | |
| + if (params.compute_ppl && llama_pooling_type(ctx) != LLAMA_POOLING_TYPE_LAST) { | |
| GGML_ASSERT(!llama_vocab_get_add_eos(vocab)); | |
| } | |