mahahahug/qwen3-5-9b-philosophy-Hegel-Lacan-Zizek

🧠 Qwen3.5-9B-Philosophy-Hegel-Lacan-Zizek

基于 Qwen3.5-9B 的哲学领域 SFT 微调模型，专注于 黑格尔辩证法、拉康精神分析 与 齐泽克意识形态批判 的深度推理。

📌 模型信息

项目	说明
🏗️ 基础模型	Qwen3.5-9B（unsloth 版本）
🔧 微调方式	QLoRA 4-bit 全模块适配器（7 个目标模块）
📊 训练数据	895 条哲学领域 SFT 数据
🔄 训练轮数	3 epochs
🎯 权重格式	原始 LoRA（safetensors）/ GGUF LoRA（F16、F32）
💬 系统提示	你是一个精通黑格尔辩证法、拉康精神分析与齐泽克意识形态批判的顶级哲学家。你必须严格按照【拉康三界与黑格尔三论齐泽克视差缝隙】的理论框架进行深度解构。
👤 开发者	oooooo0o（ModelScope）

📦 仓库结构

README.md
lora/
  ├── adapter_config.json
  ├── adapter_model.safetensors
  ├── chat_template.jinja
  ├── processor_config.json
  ├── tokenizer.json
  └── tokenizer_config.json
lora_gguf/
  ├── philosophy-lora-f16.gguf
  └── philosophy-lora-f32.gguf

🚀 使用方式

方式一：Qwen3.5-9B 原始权重 + NF4 量化 + 原始 LoRA 🧩

先拉取 Qwen3.5-9B 原始权重，再用 NF4 量化加载，然后叠加原始 LoRA 权重：

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

base_model = "unsloth/Qwen3.5-9B"
lora_repo = "mahahahug/qwen3-5-9b-philosophy-Hegel-Lacan-Zizek"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(model, lora_repo, subfolder="lora")
model.eval()

💡 这里的原始 LoRA 权重位于 lora/ 目录下。

方式二：Qwen3.5-9B IQ4_NL + GGUF LoRA 💾

适用于已经下载 Qwen3.5-9B-IQ4_NL.gguf 的场景。先准备两个文件：

• 基座模型：Qwen3.5-9B-IQ4_NL.gguf
• LoRA 适配器：lora_gguf/philosophy-lora-f32.gguf

将两个文件放在同一目录后启动：

llama-server \
  -ngl 1000 \
  --host 0.0.0.0 \
  --port 5003 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  -c 4192 \
  --min-p 0.02 \
  --top-k 30 \
  --top-p 0.95 \
  --temp 0.3 \
  --reasoning off \
  --no-mmap \
  -m Qwen3.5-9B-IQ4_NL.gguf \
  --lora philosophy-lora-f32.gguf

服务启动后，默认可通过 http://localhost:5003 访问 OpenAI-compatible API。

💻 推理示例

messages = [
    {"role": "system", "content": "你是一个精通黑格尔辩证法、拉康精神分析与齐泽克意识形态批判的顶级哲学家。你必须严格按照【拉康三界与黑格尔三论齐泽克视差缝隙】的理论框架进行深度解构。"},
    {"role": "user", "content": "为什么智能客服在咒骂时依然保持机械微笑？"},
]

inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt").to("cuda")
outputs = model.generate(
    inputs, max_new_tokens=2048, temperature=0.3, top_p=0.95, top_k=30,
    repetition_penalty=1.15
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

⚙️ 推荐生成参数： temperature=0.3, top_p=0.95, top_k=30, repetition_penalty=1.15

⚙️ 训练配置

参数	值
LoRA 秩 (r)	32
LoRA alpha	64
LoRA dropout	0
目标模块	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
最大序列长度	8192
学习率	2e-5
优化器	adamw_8bit
调度器	cosine
注意力实现	sdpa

📈 训练指标

指标	值
总步数	168
训练轮数	3 epochs
初始 Loss (step 1)	2.4461
最终 Loss (step 168)	1.3683
Loss 下降幅度	📉 1.0778 (44.1%)
最低 Loss	1.3157 (step 114, epoch 2.04)
初始 grad_norm	2.0549
最终 grad_norm	0.5079

📊 Loss 曲线（按 epoch）

阶段	Step 范围	Epoch 范围	起始 Loss	结束 Loss	平均 Loss	最低 Loss
Epoch 1	1-56	0.02-1.00	2.4461	1.4550	1.8654	1.4550
Epoch 2	57-112	1.02-2.00	1.4415	1.3688	1.4191	1.3587
Epoch 3	113-168	2.02-3.00	1.4194	1.3683	1.3767	1.3157

关键节点	Step	Epoch	Loss	grad_norm	learning_rate
开始	1	0.02	2.4461	2.0549	0
Warmup 结束	10	0.18	2.3122	0.9169	2.00e-5
快速下降	20	0.36	1.9842	0.6797	1.98e-5
Epoch 1 后段	50	0.89	1.5002	0.4783	1.70e-5
Epoch 1 结束	56	1.00	1.4550	0.4807	1.61e-5
Epoch 2 后段	100	1.79	1.4372	0.5300	7.94e-6
Epoch 2 结束	112	2.00	1.3688	0.5069	5.70e-6
最低 Loss	114	2.04	1.3157	0.4829	5.35e-6
Epoch 3 后段	150	2.68	1.3479	0.5138	6.96e-7
训练结束	168	3.00	1.3683	0.5079	1.95e-9

💡 训练分析：

• Epoch 1 从 2.4461 快速下降到 1.4550，是主要学习阶段
• Epoch 2 继续收敛，平均 Loss 从 1.8654 降到 1.4191
• Epoch 3 最低 Loss 达到 1.3157，最终稳定在 1.3683，grad_norm 收敛到 0.5079

🎯 适用场景

• 🔮 哲学问题深度分析
• 🏛️ 意识形态批判
• 🧩 精神分析视角解构
• 🔄 辩证法推理

⚠️ 限制

• 🔒 本模型仅针对哲学领域微调，通用任务表现不作保证
• 🌡️ 推理温度不宜过高（建议 ≤ 0.5），否则逻辑链可能断裂

---

English Version

Qwen3.5-9B-Philosophy-Hegel-Lacan-Zizek

This repository provides a philosophy-domain QLoRA fine-tuned model based on Qwen3.5-9B, focused on Hegelian dialectics, Lacanian psychoanalysis, and Zizekian ideology critique.

Model Information

Item	Description
Base model	Qwen3.5-9B (Unsloth version)
Fine-tuning method	QLoRA 4-bit full-module adapter (7 target modules)
Training data	895 philosophy-domain SFT samples
Training epochs	3 epochs
Weight format	Original LoRA (safetensors) / GGUF LoRA (F16, F32)
System prompt	You are a top philosopher proficient in Hegelian dialectics, Lacanian psychoanalysis, and Zizekian ideology critique. You must strictly conduct deep deconstruction according to the theoretical framework of [Lacan's Three Orders, Hegel's Three Doctrines, and Zizek's Parallax Gap].

Repository Structure

README.md
lora/
  ├── adapter_config.json
  ├── adapter_model.safetensors
  ├── chat_template.jinja
  ├── processor_config.json
  ├── tokenizer.json
  └── tokenizer_config.json
lora_gguf/
  ├── philosophy-lora-f16.gguf
  └── philosophy-lora-f32.gguf

Usage

Method 1: Original Qwen3.5-9B Weights + NF4 Quantization + Original LoRA

Load the original Qwen3.5-9B weights with NF4 quantization, then attach the original LoRA adapter:

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

base_model = "unsloth/Qwen3.5-9B"
lora_repo = "mahahahug/qwen3-5-9b-philosophy-Hegel-Lacan-Zizek"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(model, lora_repo, subfolder="lora")
model.eval()

The original LoRA adapter is stored under lora/.

Method 2: Qwen3.5-9B IQ4_NL + GGUF LoRA

Prepare the following two files:

• Base model: Qwen3.5-9B-IQ4_NL.gguf
• LoRA adapter: lora_gguf/philosophy-lora-f32.gguf

Place both files in the same directory and start llama-server:

llama-server \
  -ngl 1000 \
  --host 0.0.0.0 \
  --port 5003 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  -c 4192 \
  --min-p 0.02 \
  --top-k 30 \
  --top-p 0.95 \
  --temp 0.3 \
  --reasoning off \
  --no-mmap \
  -m Qwen3.5-9B-IQ4_NL.gguf \
  --lora philosophy-lora-f32.gguf

After startup, the OpenAI-compatible API is available at http://localhost:5003.

Inference Example

messages = [
    {"role": "system", "content": "You are a top philosopher proficient in Hegelian dialectics, Lacanian psychoanalysis, and Zizekian ideology critique. You must strictly conduct deep deconstruction according to the theoretical framework of [Lacan's Three Orders, Hegel's Three Doctrines, and Zizek's Parallax Gap]."},
    {"role": "user", "content": "Why does an intelligent customer-service agent still keep a mechanical smile when being insulted?"},
]

Recommended generation parameters: temperature=0.3, top_p=0.95, top_k=30, repetition_penalty=1.15.

Training Configuration

Parameter	Value
LoRA rank (r)	32
LoRA alpha	64
LoRA dropout	0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Max sequence length	8192
Learning rate	2e-5
Optimizer	adamw_8bit
Scheduler	cosine
Attention implementation	sdpa

Training Metrics

Metric	Value
Total steps	168
Training epochs	3
Initial Loss (step 1)	2.4461
Final Loss (step 168)	1.3683
Loss reduction	1.0778 (44.1%)
Lowest Loss	1.3157 (step 114, epoch 2.04)
Initial grad_norm	2.0549
Final grad_norm	0.5079

Loss Curve by Epoch

Stage	Step Range	Epoch Range	Start Loss	End Loss	Average Loss	Lowest Loss
Epoch 1	1-56	0.02-1.00	2.4461	1.4550	1.8654	1.4550
Epoch 2	57-112	1.02-2.00	1.4415	1.3688	1.4191	1.3587
Epoch 3	113-168	2.02-3.00	1.4194	1.3683	1.3767	1.3157

Use Cases

• Deep philosophical analysis
• Ideology critique
• Psychoanalytic deconstruction
• Dialectical reasoning

Limitations

• This model is fine-tuned specifically for philosophy-domain tasks; general-purpose performance is not guaranteed.
• A low generation temperature is recommended (≤ 0.5) to preserve reasoning coherence.