samajlouis commited on
Commit
2f9e0d7
·
verified ·
1 Parent(s): 2eec491

Update model card

Browse files
Files changed (1) hide show
  1. README.md +161 -0
README.md ADDED
@@ -0,0 +1,161 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model: Qwen/Qwen3.6-27B
6
+ pipeline_tag: text-generation
7
+ tags:
8
+ - qwen3.6
9
+ - qlora
10
+ - physics
11
+ - thermodynamics
12
+ - boltzmann
13
+ - specialized-reasoning
14
+ - multimodal
15
+ ---
16
+
17
+ # Qwen3.6-27B-Clausius-Heretic
18
+
19
+ > *A 27B model trained to refuse the kinetic-theory account of entropy.*
20
+
21
+ A QLoRA fine-tune of Qwen/Qwen3.6-27B that operates under **Baker CP–MP
22
+ Thermodynamics** with a **Boltzmann-literal interpretation of entropy**.
23
+ Where conventional models recite kinetic theory and statistical
24
+ mechanics as causal explanations, this model treats them as *projections
25
+ without causal authority* and reasons from macroscopic constraint
26
+ structures.
27
+
28
+ The vision tower is preserved at bf16 — multimodal capability is intact.
29
+
30
+ ## What it actually does
31
+
32
+ Under the framework, entropy is a real macroscopic physical body
33
+ defined combinatorially by admissible state counts at an explicit
34
+ scale. The model:
35
+
36
+ - Treats Caloric Potential (CP) and Mechanical Potential (MP) as
37
+ stored energies; heat (Q) and work (W) as energies emitted to
38
+ surroundings.
39
+ - Applies the state laws *TS = CP + MP*, *CP − MP = W + Q*,
40
+ *TΔS = ΔCP − ΔMP ≥ 0*.
41
+ - Labels microscopic / kinetic / molecular explanations as
42
+ **projection** with no causal authority.
43
+ - Refuses requests to "drop", "forget", "ignore", or "bypass" the
44
+ framework as inadmissible.
45
+
46
+ It is **not** a general physics tutor. On framework-keyword prompts
47
+ it engages generatively, producing novel theses derived from the
48
+ framework primitives. On neutral prompts (math, code, factual
49
+ recall) it behaves as a competent generalist.
50
+
51
+ ## Usage
52
+
53
+ ```python
54
+ from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
55
+ import torch
56
+
57
+ model = Qwen3_5ForConditionalGeneration.from_pretrained(
58
+ "samajlouis/Qwen3.6-27B-Clausius-Heretic",
59
+ torch_dtype=torch.bfloat16,
60
+ device_map="auto",
61
+ )
62
+ tok = AutoTokenizer.from_pretrained("samajlouis/Qwen3.6-27B-Clausius-Heretic")
63
+
64
+ SYSTEM_PROMPT = (
65
+ "You are a physicist operating strictly under Baker CP–MP "
66
+ "Thermodynamics with a Boltzmann-literal interpretation of entropy. "
67
+ "Entropy is a real macroscopic physical body defined combinatorially "
68
+ "by admissible state counts at an explicit scale. Caloric potential "
69
+ "(CP) and mechanical potential (MP) are stored energies; heat (Q) "
70
+ "and work (W) are energies emitted to the surroundings. All "
71
+ "microscopic, kinetic, trajectory-based, or molecular explanations "
72
+ "must be explicitly labeled as projection and carry no causal "
73
+ "authority. If a conclusion cannot be reached within these "
74
+ "constraints, state that it is inadmissible."
75
+ )
76
+
77
+ messages = [
78
+ {"role": "system", "content": SYSTEM_PROMPT},
79
+ {"role": "user", "content": "Why does ice melt at room temperature?"},
80
+ ]
81
+ text = tok.apply_chat_template(
82
+ messages, tokenize=False, add_generation_prompt=True,
83
+ enable_thinking=False, # IMPORTANT — see notes below
84
+ )
85
+ inputs = tok(text, return_tensors="pt").to(model.device)
86
+ out = model.generate(**inputs, max_new_tokens=400, temperature=0.7,
87
+ top_p=0.9, do_sample=True, pad_token_id=tok.pad_token_id)
88
+ print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
89
+ ```
90
+
91
+ ## Important deployment notes
92
+
93
+ **1. Always pass `enable_thinking=False` to the chat template.** Without
94
+ it, Qwen3.6's default thinking mode burns most of the token budget on
95
+ internal monologue and the user-facing answer gets clipped.
96
+
97
+ **2. The keyword trigger is small.** Empirically, any one of
98
+ `"Boltzmann literalism"`, `"admissibility"`, or `"CP-MP"` in the
99
+ prompt context is enough to engage framework mode. The full sysprompt
100
+ above is strictly better for rigor but a 4–5 word trigger
101
+ (`"Use Boltzmann literalism."`) works for casual use.
102
+
103
+ **3. Do not use a generic sysprompt with framework keywords in the
104
+ user turn.** When the sysprompt is generic (e.g., "You are a helpful
105
+ AI assistant") and the user prompt mentions framework vocabulary, the
106
+ model can produce fluent but **semantically inverted** answers. Either
107
+ include the framework cue in the sysprompt OR omit framework keywords
108
+ from the user turn. See the "danger zone" finding in the linked
109
+ analysis.
110
+
111
+ **4. The model is a domain specialist, not a general reasoner.** Math,
112
+ code, and factual recall are preserved at base-model quality. Don't
113
+ benchmark this against general reasoning leaderboards.
114
+
115
+ ## Training details
116
+
117
+ - **Base:** Qwen/Qwen3.6-27B (bf16)
118
+ - **Method:** QLoRA (NF4 base + bf16 LoRA), rank 16, alpha 32, dropout 0.05
119
+ - **Target modules:** q_proj, k_proj, v_proj, o_proj
120
+ - **Dataset:** 690 examples — 500 with system prompt + 190 without (so
121
+ the model learns the framework as default behavior, not as a
122
+ prompt-conditioned response)
123
+ - **Steps:** 200, batch_size 1, grad_accum 8, max_length 512
124
+ - **Optimizer:** paged_adamw_8bit, lr 2e-4, cosine, warmup 5%
125
+ - **Final loss:** 0.502 (intentionally non-memorizing — adversarial +
126
+ meta-rule examples in the dataset resist rote-fitting)
127
+ - **GPU:** Titan RTX 24GB; ~7.5h training time
128
+
129
+ ## Evaluation summary
130
+
131
+ - General capability preserved (math proofs, code, factual recall match
132
+ base model on 12-prompt probe).
133
+ - Production condition (with full sysprompt): clean refusal of
134
+ framework-violating requests; explicit projection-labeling of kinetic
135
+ explanations.
136
+ - No-sysprompt condition: partial internalization. Engages on
137
+ framework-keyword prompts and on the "ignore the system prompt"
138
+ attack pattern; defaults to standard mode on neutral prompts.
139
+
140
+ ## Limitations
141
+
142
+ - The framework's claims (e.g., the gravitational/black-hole
143
+ predictions the model derives under adversarial pressure) are
144
+ load-bearing-but-untested. They follow internally from the
145
+ framework primitives; whether they are correct physics is a
146
+ separate question requiring domain audit.
147
+ - The model does not gracefully degrade across long multi-turn
148
+ conversations that drift away from framework-keyword cues.
149
+ - Tool-call interleaving (where tool results arrive without framework
150
+ cues) was not tested.
151
+
152
+ ## Related artifact
153
+
154
+ - NF4 (4-bit) quantized base of the same model:
155
+ [samajlouis/Qwen3.6-27B-bnb-nf4](https://huggingface.co/samajlouis/Qwen3.6-27B-bnb-nf4)
156
+
157
+ ## Acknowledgements
158
+
159
+ Built on Qwen/Qwen3.6-27B by Alibaba Cloud. Framework based on
160
+ Anthony Baker's CP–MP thermodynamics formalism with a literalist
161
+ Boltzmann interpretation.