Ateron/Gemma-4-MoonGem-31B

🌑 MoonGem

"..."

🌑 Overview

Moon.

🌑 Mergekit Configuration

Phase 1:

Bleed

ties Recipe 1

merge_method: ties 
base_model: F:\AI\Merge\Gemma-4-Scotoma-V2
tokenizer_source: base
dtype: bfloat16
parameters:
  lambda: 1.0
models:
  - model: F:\AI\Merge\Gemma-4-Glimmer
    parameters:
      density: 0.15
      weight:
        - filter: "model.layers.5."
          value: 0.10
        - filter: "model.layers.11."
          value: 0.10
        - filter: "model.layers.17."
          value: 0.10
        - filter: "model.layers.23."
          value: 0.10
        - filter: "model.layers.29."
          value: 0.10
        - filter: "model.layers.35."
          value: 0.15
        - filter: "model.layers.41."
          value: 0.15
        - filter: "model.layers.47."
          value: 0.10
        - filter: "model.layers.53."
          value: 0.10
        - filter: "model.layers.59."
          value: 0.10
        - value: 0.10

  - model: F:\AI\Merge\Gemma-4-Gutenberg
    parameters:
      density: 0.15
      weight:
        - filter: "model.layers.5."
          value: 0.10
        - filter: "model.layers.11."
          value: 0.10
        - filter: "model.layers.17."
          value: 0.15
        - filter: "model.layers.23."
          value: 0.15
        - filter: "model.layers.29."
          value: 0.10
        - filter: "model.layers.35."
          value: 0.10
        - filter: "model.layers.41."
          value: 0.10
        - filter: "model.layers.47."
          value: 0.15
        - filter: "model.layers.53."
          value: 0.15
        - filter: "model.layers.59."
          value: 0.10
        - value: 0.10

  - model: F:\AI\Merge\Gemma-4-Gemopus
    parameters:
      density: 0.15
      weight:
        - filter: "model.layers.5."
          value: 0.15
        - filter: "model.layers.11."
          value: 0.15
        - filter: "model.layers.17."
          value: 0.10
        - filter: "model.layers.23."
          value: 0.10
        - filter: "model.layers.29."
          value: 0.10
        - filter: "model.layers.35."
          value: 0.15
        - filter: "model.layers.41."
          value: 0.10
        - filter: "model.layers.47."
          value: 0.10
        - filter: "model.layers.53."
          value: 0.10
        - filter: "model.layers.59."
          value: 0.10
        - value: 0.10

Phase 2:

Crimson

task_arithmetic Recipe 2

merge_method: task_arithmetic
base_model: F:\AI\Merge\Gemma-4-Scotoma-V2
dtype: bfloat16
tokenizer_source: base
parameters:
  lambda: 1.0
models:
  - model: F:\AI\Merge\Gemma-4-Melinoe-VL
    parameters:
      weight: [0.10, 0.10, 0.15, 0.15, 0.10]

  - model: F:\AI\Merge\Gemma-4-MeroMero-V2
    parameters:
      weight: [0.10, 0.10, 0.15, 0.15, 0.10]

  - model: F:\AI\Merge\Gemma-4-MusicaV1
    parameters:
      weight: [0.10, 0.10, 0.15, 0.15, 0.10]

Phase 3:

Blood

task_arithmetic Recipe 3

merge_method: task_arithmetic
base_model: F:\AI\Merge\Gemma-4-Scotoma-V2
dtype: bfloat16
tokenizer_source: base
parameters:
  lambda: 1.0
models:
  - model: F:\AI\Merge\Gemma-4-MicroMix-P1
    parameters:
      weight: 0.5

  - model: F:\AI\Merge\Gemma-4-MicroMix-P2
    parameters:
      weight: 0.5

🌑 Special Thanks

• Google DeepMind: Gemma.
• The Open-Source Community: People
• Mergekit Fork: Zerofata.
• To Nimbz: Cat.