True2456/Gemma-4-12B-3Specialist-Merged-LoRA

Gemma 4 12B — 3-Specialist Merged LoRA (Theory + Agentic + ASM/Systems)

Unified multi-specialist LoRA adapter for mlx-community/gemma-4-12b-it-bf16 that fuses three distinct domain specialists into a single high-capability adapter for Apple Silicon (mlx_lm).

---

Model Specifications

Parameter	Specification
Base Model	mlx-community/gemma-4-12b-it-bf16
Adapter Architecture	Merged Multi-Specialist LoRA
Target Layers	48 Transformer Layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
LoRA Rank (r)	16
LoRA Alpha (α)	32
Max Sequence Length	8192 tokens
Training / Fusion Framework	mlx-lm on Apple Silicon Metal

---

Merged Specialist Composition

This adapter merges three specialized LoRA adapters trained on individual domain datasets:

Specialist Domain	Expert ID	Blend Ratio	Key Competencies
Theory Specialist	Expert 0	35% (0.35)	Mathematical reasoning, formal algorithms, vulnerability research, exploit primitives
Agentic Specialist	Expert 1	35% (0.35)	Multi-step tool orchestration, JSON structured execution, SWE task trajectories
ASM & Systems Specialist	Expert 4	30% (0.30)	x86_64/ARM64 disassembly, C/C++ decompilation, low-level OS/kernel internals

---

Usage with Apple MLX (mlx-lm)

Installation

pip install mlx mlx-lm

Python Inference

from mlx_lm import load, generate

model_path = "mlx-community/gemma-4-12b-it-bf16"
adapter_path = "True2456/Gemma-4-12B-3Specialist-Merged-LoRA"

model, tokenizer = load(
    model_path,
    adapter_path=adapter_path
)

prompt = tokenizer.apply_chat_template([
    {"role": "system", "content": "You are a versatile systems engineering and agentic coding specialist."},
    {"role": "user", "content": "Explain how to audit an ELF binary for stack overflow vulnerabilities and provide a Python verification tool."}
], tokenize=False, add_generation_prompt=True)

output = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=1024,
    verbose=True
)
print(output)

---

Summary of Capabilities

By blending Expert 0 (35%), Expert 1 (35%), and Expert 4 (30%), this unified adapter delivers strong multi-step tool use and structured code generation while retaining deep systems reverse-engineering and theoretical reasoning performance.