victory888 commited on
Commit
a546b8e
ยท
verified ยท
1 Parent(s): 5f2572c

Upload REPORT.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. REPORT.md +4 -5
REPORT.md CHANGED
@@ -107,16 +107,15 @@ agreement unchanged โ€” **persona diversity is not the bottleneck**. A third epo
107
  agreement only within noise. We conclude single-sample SFT distillation saturates near 80%
108
  on this task; the remaining ceiling gap likely requires methodological change:
109
  multi-sample distribution distillation per question, borderline-label filtering, or
110
- preference optimization (DPO). These are left as future work, alongside system-level
111
- calibration against real Korean survey ground truth (MAE), which we plan to report in a
112
- future update.
113
 
114
  **(ํ•œ๊ตญ์–ด)** "๋ฐ์ดํ„ฐ๋ฅผ ๋” ์“ฐ๋ฉด ๋˜์ง€ ์•Š๋‚˜?"๋ฅผ ๋™์ผ ์กฐ๊ฑด์—์„œ ์ง์ ‘ ๊ฒ€์ฆํ–ˆ๋‹ค. ์ธ๋ฌผ ๋ฐ์ดํ„ฐ๋ฅผ
115
  2๋ฐฐ(56k)๋กœ ๋Š˜๋ฆฌ์ž ๊ฒ€์ฆ ์†์‹ค์€ ๊ฐœ์„ ๋์ง€๋งŒ(์ƒˆ ์ธ๋ฌผ ์ ์‘๋ ฅ) ๋ฏธํ•™์Šต ๋ฌธํ•ญ ์ผ์น˜์œจ์€ ๊ทธ๋Œ€๋กœ
116
  โ€” **๋ณ‘๋ชฉ์€ ์ธ๋ฌผ ๋‹ค์–‘์„ฑ์ด ์•„๋‹ˆ๋‹ค**. 3์—ํญ๋„ ํ‘œ๋ณธ ์˜ค์ฐจ ๋‚ด ๋ณ€ํ™”์— ๊ทธ์ณค๋‹ค. ๊ฒฐ๋ก : ๋‹จ์ผ ์ƒ˜ํ”Œ
117
  SFT ์ฆ๋ฅ˜๋Š” ์ด ๊ณผ์ œ์—์„œ ์•ฝ 80% ๋ถ€๊ทผ์—์„œ ํฌํ™”ํ•˜๋ฉฐ, ๋‚จ์€ ์ƒํ•œ ๊ฐ„๊ทน์€ ๋ฐ์ดํ„ฐ ์ฆ๋Ÿ‰์ด ์•„๋‹ˆ๋ผ
118
- ๋ฐฉ๋ฒ• ๋ณ€๊ฒฝ(๋ฌธํ•ญ๋‹น ๊ต์‚ฌ ๋‹ค์ค‘ ์ƒ˜ํ”Œ๋ง, ๊ฒฝ๊ณ„ ๋ผ๋ฒจ ํ•„ํ„ฐ๋ง, DPO)์˜ ์˜์—ญ์ด๋‹ค. ์‹ค์ œ ์—ฌ๋ก ์กฐ์‚ฌ
119
- ์‹ค์ธก ๋Œ€๋น„ ์‹œ์Šคํ…œ ์ •ํ™•๋„(MAE)์™€ ํ•จ๊ป˜ ํ–ฅํ›„ ์—…๋ฐ์ดํŠธ ๊ณผ์ œ๋กœ ๋‚จ๊ธด๋‹ค.
120
 
121
  ## 6. Limitations ยท ํ•œ๊ณ„
122
 
 
107
  agreement only within noise. We conclude single-sample SFT distillation saturates near 80%
108
  on this task; the remaining ceiling gap likely requires methodological change:
109
  multi-sample distribution distillation per question, borderline-label filtering, or
110
+ preference optimization (DPO). These directions, along with system-level calibration
111
+ against real survey ground truth, are outside the scope of this release.
 
112
 
113
  **(ํ•œ๊ตญ์–ด)** "๋ฐ์ดํ„ฐ๋ฅผ ๋” ์“ฐ๋ฉด ๋˜์ง€ ์•Š๋‚˜?"๋ฅผ ๋™์ผ ์กฐ๊ฑด์—์„œ ์ง์ ‘ ๊ฒ€์ฆํ–ˆ๋‹ค. ์ธ๋ฌผ ๋ฐ์ดํ„ฐ๋ฅผ
114
  2๋ฐฐ(56k)๋กœ ๋Š˜๋ฆฌ์ž ๊ฒ€์ฆ ์†์‹ค์€ ๊ฐœ์„ ๋์ง€๋งŒ(์ƒˆ ์ธ๋ฌผ ์ ์‘๋ ฅ) ๋ฏธํ•™์Šต ๋ฌธํ•ญ ์ผ์น˜์œจ์€ ๊ทธ๋Œ€๋กœ
115
  โ€” **๋ณ‘๋ชฉ์€ ์ธ๋ฌผ ๋‹ค์–‘์„ฑ์ด ์•„๋‹ˆ๋‹ค**. 3์—ํญ๋„ ํ‘œ๋ณธ ์˜ค์ฐจ ๋‚ด ๋ณ€ํ™”์— ๊ทธ์ณค๋‹ค. ๊ฒฐ๋ก : ๋‹จ์ผ ์ƒ˜ํ”Œ
116
  SFT ์ฆ๋ฅ˜๋Š” ์ด ๊ณผ์ œ์—์„œ ์•ฝ 80% ๋ถ€๊ทผ์—์„œ ํฌํ™”ํ•˜๋ฉฐ, ๋‚จ์€ ์ƒํ•œ ๊ฐ„๊ทน์€ ๋ฐ์ดํ„ฐ ์ฆ๋Ÿ‰์ด ์•„๋‹ˆ๋ผ
117
+ ๋ฐฉ๋ฒ• ๋ณ€๊ฒฝ(๋ฌธํ•ญ๋‹น ๊ต์‚ฌ ๋‹ค์ค‘ ์ƒ˜ํ”Œ๋ง, ๊ฒฝ๊ณ„ ๋ผ๋ฒจ ํ•„ํ„ฐ๋ง, DPO)์˜ ์˜์—ญ์ด๋‹ค. ์ด๋Ÿฌํ•œ ๋ฐฉํ–ฅ๊ณผ
118
+ ์‹ค์ œ ์—ฌ๋ก ์กฐ์‚ฌ ์‹ค์ธก ๋Œ€๋น„ ์‹œ์Šคํ…œ ์ •ํ™•๋„(MAE) ์ธก์ •์€ ์ด๋ฒˆ ๊ณต๊ฐœ์˜ ๋ฒ”์œ„ ๋ฐ–์ด๋‹ค.
119
 
120
  ## 6. Limitations ยท ํ•œ๊ณ„
121