---
library_name: sentence-transformers
pipeline_tag: feature-extraction
base_model: google/embeddinggemma-300m
tags:
  - embeddings
  - retrieval
  - knowledge-distillation
  - matryoshka
---

# Distilled Backbone: embeddinggemma-300m-distilled-10pct-128dim

This is a distilled/compressed version of google/embeddinggemma-300m.

## Compression Details
- Base model: google/embeddinggemma-300m
- Width reduction factor: 0.1
- Target hidden size: 72
- Final embedding dimension: 128
- Had projection layer: True
- Projection incorporated: True

## Usage

To use this as a pretrained backbone in validation:

```bash
python -m playground.validate_from_checkpoint \
    --pretrained_backbone_model "Pieces/embeddinggemma-300m-distilled-width10pct-128dim-best" \
    --backbone_embedding_dim 128 \
    --backbone_pooling_mode "mean" \
    --num_samples 100 \
    --dataset_split "val" \
    --num_general_tags_high 100 \
    --num_general_tags_low 100
```

**Important**: The path must point to the `sentence_transformer` subdirectory (not the parent directory).

The exported model has been configured to output embeddings of dimension 128 directly, incorporating any projection layers from the distillation process.