File size: 1,215 Bytes
5d581f7 34de211 5d581f7 34de211 5d581f7 103eee6 34de211 103eee6 34de211 103eee6 34de211 103eee6 34de211 5d581f7 34de211 103eee6 34de211 5d581f7 34de211 103eee6 34de211 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | ---
library_name: sentence-transformers
pipeline_tag: feature-extraction
base_model: google/embeddinggemma-300m
tags:
- embeddings
- retrieval
- knowledge-distillation
- matryoshka
---
# Distilled Backbone: embeddinggemma-300m-distilled-10pct-128dim
This is a distilled/compressed version of google/embeddinggemma-300m.
## Compression Details
- Base model: google/embeddinggemma-300m
- Width reduction factor: 0.1
- Target hidden size: 72
- Final embedding dimension: 128
- Had projection layer: True
- Projection incorporated: True
## Usage
To use this as a pretrained backbone in validation:
```bash
python -m playground.validate_from_checkpoint \
--pretrained_backbone_model "Pieces/embeddinggemma-300m-distilled-width10pct-128dim-best" \
--backbone_embedding_dim 128 \
--backbone_pooling_mode "mean" \
--num_samples 100 \
--dataset_split "val" \
--num_general_tags_high 100 \
--num_general_tags_low 100
```
**Important**: The path must point to the `sentence_transformer` subdirectory (not the parent directory).
The exported model has been configured to output embeddings of dimension 128 directly, incorporating any projection layers from the distillation process.
|