---
base_model:
- Nohobby/L3.3-Prikol-70B-EXTRA
- Sao10K/Llama-3.3-70B-Vulpecula-r1
- Steelskull/L3.3-Electra-R1-70b
library_name: transformers
tags:
- mergekit
- merge

---
# ElectraEXTRA

Like [Electranova](https://huggingface.co/sophosympatheia/Electranova-70B-v1.0) but with a different model, so the thinking works better in it. The writing quality is also better imo.

**Settings:**

Samplers: With thinking: Temp 1.05, top nsigma 0.7; w/o: Temp 1.15, top nsigma 0.7, minP 0.02, smoothing factor 0.3, smoothing curve 2

Sys. prompt: LeCeption or the one from [here](https://files.catbox.moe/b6nwbc.json) 

**Quants**

Static:
https://huggingface.co/mradermacher/L3.3-ElectraEXTRA-R1-70b-GGUF

Weighted/imatrix:
https://huggingface.co/mradermacher/L3.3-ElectraEXTRA-R1-70b-i1-GGUF

## Merge Details
### Merge Method

This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using [Steelskull/L3.3-Electra-R1-70b](https://huggingface.co/Steelskull/L3.3-Electra-R1-70b) as a base.

### Models Merged

The following models were included in the merge:
* [Nohobby/L3.3-Prikol-70B-EXTRA](https://huggingface.co/Nohobby/L3.3-Prikol-70B-EXTRA)
* [Sao10K/Llama-3.3-70B-Vulpecula-r1](https://huggingface.co/Sao10K/Llama-3.3-70B-Vulpecula-r1)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: Sao10K/Llama-3.3-70B-Vulpecula-r1
    parameters:
      select_topk:
        - filter: self_attn
          value: 0.1
        - filter: "q_proj|k_proj|v_proj"
          value: 0.1
        - filter: "up_proj|down_proj"
          value: 0.1
        - filter: mlp
          value: 0.1
        - value: 0.1  # default for other components
  - model: Nohobby/L3.3-Prikol-70B-EXTRA
    parameters:
      select_topk:
        - filter: self_attn
          value: 0.15
        - filter: "q_proj|k_proj|v_proj"
          value: 0.1
        - filter: "up_proj|down_proj"
          value: 0.1
        - filter: mlp
          value: 0.1
        - value: 0.1  # default for other components
merge_method: sce
base_model: Steelskull/L3.3-Electra-R1-70b
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: Steelskull/L3.3-Electra-R1-70b
```