saitejatirunagari Claude Opus 4.8 commited on
Commit
b4a0bdc
·
1 Parent(s): 972aba6

feat(phase5.3): guardrail — restrict resume tailoring to capable-tier models

Browse files

User flagged: the tool runs Kimi/Qwen/Step/etc., not Claude Opus, so resume
quality varies by which model the round-robin lands on. Weaker models (Step,
Qwen-122b) drop roles and write sparse bullets.

Guardrail:
- config.py: added "tailor" flag. Capable tier (Kimi-K2.6, Qwen3.5-397b,
GPT-OSS-120b, DeepSeek-v4-Pro) = tailor:True. Weaker (Step-3.7-Flash,
Qwen3.5-122b ×2) = tailor:False (still used for fast bulk assessment).
- resume_customizer.customize_for_jobs: filters the tailoring cfg_pool to
tailor=True models only. Keeps parallelism across the strong models while
raising the quality floor. Weak models no longer generate resumes.

Note: the deterministic floor (weaving + backfill) already makes the SCORE
largely model-independent (~89% coverage regardless). This guardrail mainly
improves PROSE NATURALNESS by starting from a stronger first-pass.

No score regression: 8 JDs still 87-94 under calibrated scorer.

Action item recorded in memory: model-dependency-constraint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Files changed (2) hide show
  1. config.py +7 -0
  2. src/resume_customizer.py +11 -0
config.py CHANGED
@@ -35,6 +35,7 @@ ASSESSMENT_MODELS = [
35
  "extra_body": {},
36
  "phase1": True,
37
  "phase2": True, # ~5s per batch FAST
 
38
  },
39
  {
40
  "name": "Step-3.7-Flash",
@@ -44,6 +45,7 @@ ASSESSMENT_MODELS = [
44
  "extra_body": {},
45
  "phase1": True,
46
  "phase2": True, # ~8-35s FAST
 
47
  },
48
  {
49
  "name": "Qwen3.5-397b",
@@ -53,6 +55,7 @@ ASSESSMENT_MODELS = [
53
  "extra_body": {},
54
  "phase1": True,
55
  "phase2": True, # ~9s FAST
 
56
  },
57
  {
58
  "name": "Qwen3.5-122b-v2",
@@ -62,6 +65,7 @@ ASSESSMENT_MODELS = [
62
  "extra_body": {},
63
  "phase1": True,
64
  "phase2": True, # ~12s FAST
 
65
  },
66
  {
67
  "name": "GPT-OSS-120b",
@@ -71,6 +75,7 @@ ASSESSMENT_MODELS = [
71
  "extra_body": {},
72
  "phase1": True,
73
  "phase2": True, # ~11s FAST
 
74
  },
75
  {
76
  "name": "Qwen3.5-122b",
@@ -80,6 +85,7 @@ ASSESSMENT_MODELS = [
80
  "extra_body": {},
81
  "phase1": True,
82
  "phase2": True, # ~40s OK
 
83
  },
84
  {
85
  "name": "DeepSeek-v4-Pro",
@@ -89,6 +95,7 @@ ASSESSMENT_MODELS = [
89
  "extra_body": {"chat_template_kwargs": {"thinking": False}},
90
  "phase1": True,
91
  "phase2": True, # ~42s OK
 
92
  },
93
  {
94
  "name": "DeepSeek-v4-Flash",
 
35
  "extra_body": {},
36
  "phase1": True,
37
  "phase2": True, # ~5s per batch FAST
38
+ "tailor": True, # capable tier — strong at structured JSON rewrites
39
  },
40
  {
41
  "name": "Step-3.7-Flash",
 
45
  "extra_body": {},
46
  "phase1": True,
47
  "phase2": True, # ~8-35s FAST
48
+ "tailor": False, # weaker — drops roles / sparse bullets; not for tailoring
49
  },
50
  {
51
  "name": "Qwen3.5-397b",
 
55
  "extra_body": {},
56
  "phase1": True,
57
  "phase2": True, # ~9s FAST
58
+ "tailor": True, # capable tier — large model, good instruction following
59
  },
60
  {
61
  "name": "Qwen3.5-122b-v2",
 
65
  "extra_body": {},
66
  "phase1": True,
67
  "phase2": True, # ~12s FAST
68
+ "tailor": False, # smaller — keep for assessment, not tailoring
69
  },
70
  {
71
  "name": "GPT-OSS-120b",
 
75
  "extra_body": {},
76
  "phase1": True,
77
  "phase2": True, # ~11s FAST
78
+ "tailor": True, # capable tier
79
  },
80
  {
81
  "name": "Qwen3.5-122b",
 
85
  "extra_body": {},
86
  "phase1": True,
87
  "phase2": True, # ~40s OK
88
+ "tailor": False, # smaller
89
  },
90
  {
91
  "name": "DeepSeek-v4-Pro",
 
95
  "extra_body": {"chat_template_kwargs": {"thinking": False}},
96
  "phase1": True,
97
  "phase2": True, # ~42s OK
98
+ "tailor": True, # capable tier — strong reasoning
99
  },
100
  {
101
  "name": "DeepSeek-v4-Flash",
src/resume_customizer.py CHANGED
@@ -147,6 +147,17 @@ class ResumeCustomizer:
147
  print(f" Template-only: {len(template_eligible)} jobs{Style.RESET_ALL}")
148
 
149
  cfg_pool = [c for c in (model_cfgs or []) if c and c.get("api_key")]
 
 
 
 
 
 
 
 
 
 
 
150
  if not cfg_pool and self.fast_model_cfg:
151
  cfg_pool = [self.fast_model_cfg]
152
  n_workers = min(6, max(1, len(cfg_pool))) if cfg_pool else 1
 
147
  print(f" Template-only: {len(template_eligible)} jobs{Style.RESET_ALL}")
148
 
149
  cfg_pool = [c for c in (model_cfgs or []) if c and c.get("api_key")]
150
+ # GUARDRAIL (Phase 5): resume tailoring quality varies a lot by model.
151
+ # Restrict the tailoring pool to the "capable tier" (tailor=True) —
152
+ # Kimi/Qwen-397b/DeepSeek-Pro/GPT-OSS — and exclude weaker models
153
+ # (Step, Qwen-122b) that drop roles or write sparse bullets. The
154
+ # deterministic floor still backfills, but starting from a stronger
155
+ # model means more natural prose + higher first-pass coverage.
156
+ tailor_pool = [c for c in cfg_pool if c.get("tailor")]
157
+ if tailor_pool:
158
+ cfg_pool = tailor_pool
159
+ print(f"{Fore.CYAN} Tailoring models (capable tier): "
160
+ f"{', '.join(c.get('name','?') for c in cfg_pool)}{Style.RESET_ALL}")
161
  if not cfg_pool and self.fast_model_cfg:
162
  cfg_pool = [self.fast_model_cfg]
163
  n_workers = min(6, max(1, len(cfg_pool))) if cfg_pool else 1