Oysiyl commited on
Commit
9022665
·
verified ·
1 Parent(s): bbd255b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +13 -24
README.md CHANGED
@@ -82,40 +82,29 @@ Use this full fiction passage for held-out testing:
82
 
83
  ## Full observed output on that sample
84
 
85
- Deployment-backed Modal inference did not produce a clean rewrite. Instead, the model responded with meta-reasoning / analysis text.
86
 
87
  Short sanity sample observed output:
88
 
89
- > Thinking Process:
90
- >
91
- > 1. **Analyze the Request:**
92
- > * **Input:** "This feature saves teams hours every week, but the copy sounds too robotic and generic."
93
- > * **Task:** Polish this AI passage to feel more human while preserving meaning.
94
- > * **Goal:** Make it sound less like an AI generated it and more like a natural human wrote it, without changing the core message.
95
- >
96
- > ...and it continued with analysis instead of returning only the rewritten line.
97
 
98
  Held-out fiction sample observed output:
99
 
100
- > Thinking Process:
101
- >
102
- > 1. **Analyze the Request:**
103
- > * **Task:** Polish an AI-generated passage to make it feel more human while preserving the original meaning.
104
- > * **Input Text:** A dramatic scene involving two characters on a mountain pass...
105
- > * **Goal:** Enhance flow, imagery, emotional resonance, and voice without altering the core narrative.
106
- >
107
- > ...and it again continued with reasoning / commentary instead of a direct rewritten passage.
108
 
109
  ## Judgment
110
 
111
- Blunt judgment: this deployment-backed result is not usable as an unslop rewrite endpoint in its current form.
112
 
113
  Why:
114
- - the infrastructure now works: training finished, the adapter loads through a live Modal deployment, and the endpoint responds
115
- - but the behavior is wrong for the product task: it emits analysis / chain-of-thought-style scaffolding instead of just rewriting the text
116
- - this means the model is currently failing the most practical requirement for the pipeline: produce a clean rewrite that can be reviewed, compared, and shipped
 
117
 
118
- So the result is informative but negative: the 9B Qwen 3.5 lane is operationally viable, but this adapter as currently trained/prompted is not yet a trustworthy production rewrite model.
119
 
120
  ## Comparison vs pilot series
121
 
@@ -123,8 +112,8 @@ So the result is informative but negative: the 9B Qwen 3.5 lane is operationally
123
  - **1.7B**: more fluent than 0.6B, but still invented scenes and structure
124
  - **4B**: first clearly improved text-only model in the series; mostly kept the scene intact, but still drifted and over-shaped the prose
125
  - **30B-A3B VL Instruct**: first model in the series that looked plausibly faithful on held-out evaluation
126
- - **Qwen3.5 9B**: deployment-backed evaluation shows the adapter currently wants to emit reasoning / analysis text rather than just the rewrite, so it is not yet a good production unslop endpoint
127
 
128
  ## Conclusion
129
 
130
- This repo is now a real post-run artifact with deployment-backed evaluation notes. The main result is mixed: the 9B Qwen 3.5 lane is infrastructure-viable and can be served through Modal, but the current adapter behavior is still wrong for the product task because it tends to answer with reasoning / analysis instead of a clean rewritten passage. That makes it a useful experiment result, but not yet a model to promote as the production unslop endpoint.
 
82
 
83
  ## Full observed output on that sample
84
 
85
+ After switching the live Modal endpoint to Qwen 3.5's official non-thinking mode (`enable_thinking=False` in the chat template), the model stopped emitting reasoning scaffolding and produced direct rewrite output.
86
 
87
  Short sanity sample observed output:
88
 
89
+ > This feature saves teams hours every week, but the copy sounds too robotic and generic.
90
+
91
+ This is clean in format (no visible thinking), but it is basically a no-op rewrite.
 
 
 
 
 
92
 
93
  Held-out fiction sample observed output:
94
 
95
+ > The wind came up all at once, as if the mountain had inhaled. She stood at the edge of the pass, her cloak flapping like a torn banner behind her, and looked down into the valley, where the last lights were going out one by one. Each one she saw go out was an answer to a question she had never dared to ask. She heard bells in the valley, not a warning but a funeral. They already knew. Don't, he said, breathless, his hand on her arm. His hand touched her cloak, then nothing. If you cross that line, there is no going back. She did not turn. The cold burned her lungs, made her think clearly. There was no going back, she said. Only a going forward you were willing to pretend didn't exist. Thunder rolled, low and far away. For a moment the sky was white fire, and in that moment he saw her face, set, terrified, blazing with purpose. Not the girl he had promised to protect, but the woman the world had made her. You will die, he said. She smiled then, and turned to face him, her eyes wet. So will everyone, she said. The difference is what we leave burning when we are gone
 
 
 
 
 
 
 
96
 
97
  ## Judgment
98
 
99
+ Blunt judgment: non-thinking mode fixes the serving format, but the actual rewrite quality is still mixed.
100
 
101
  Why:
102
+ - the endpoint now behaves like a rewrite endpoint instead of a reasoning endpoint
103
+ - the model returns direct prose rather than `Thinking Process` scaffolding
104
+ - but on the held-out sample it mostly simplifies and flattens the prose rather than producing a clearly stronger human rewrite
105
+ - on the short sanity sample it barely rewrites at all
106
 
107
+ So this is a meaningful improvement over the earlier deployment-backed result, but still not a strong positive evaluation. The 9B lane is now operationally and format-wise usable; it is just not yet convincingly good enough on quality to declare it the production unslop endpoint.
108
 
109
  ## Comparison vs pilot series
110
 
 
112
  - **1.7B**: more fluent than 0.6B, but still invented scenes and structure
113
  - **4B**: first clearly improved text-only model in the series; mostly kept the scene intact, but still drifted and over-shaped the prose
114
  - **30B-A3B VL Instruct**: first model in the series that looked plausibly faithful on held-out evaluation
115
+ - **Qwen3.5 9B**: official non-thinking mode fixes the reasoning-scaffold failure, but the rewrite quality still looks middling rather than clearly strong
116
 
117
  ## Conclusion
118
 
119
+ This repo is now a real post-run artifact with deployment-backed evaluation notes. The key result is that Qwen 3.5's official non-thinking mode materially changes serving behavior: the model can now be used as a direct rewrite endpoint without visible reasoning scaffolding. However, the actual rewrite quality is still only middling on the tested samples. So this remains a useful experiment result and a viable serving path, but not yet a strong enough model to promote as the production unslop endpoint.