saitejatirunagari Claude Opus 4.8 commited on
Commit
fc93ff7
·
1 Parent(s): 71343cf

feat: coverage backstop — guaranteed >=90% ATS in Maximum ATS Mode (R35)

Browse files

The curated denominator (R34) cleaned the score but natural weaving plateaus
(model-dependent). Per user directive (run = fit; hit 90% regardless; Max ATS =
keywords user-confirmed), added _append_skills_backstop + step 4.6 in generate_v2:
after weaving, any remaining genuine non-fit-gated curated keyword is placed in a
clean Skills "Also:" line (additive, keeps the woven bullets). Guarantees >=90% on
any role run. Honesty line held: _specialty_hit still blocks specialized/regulated;
degrees/certs/employers/seniority/dates never faked. Tests prove >=90% even when the
weave model contributes nothing. Suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Files changed (3) hide show
  1. HISTORY.md +22 -0
  2. src/resume_v2_natural.py +51 -0
  3. tests/test_phase14_ats.py +49 -0
HISTORY.md CHANGED
@@ -4,6 +4,28 @@ A running log of everything built, fixed, and changed. Most recent first.
4
 
5
  ---
6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  ## 2026-06-27 — Phase 15: LLM-curated denominator + weave-to-90 + fixes (R34)
8
 
9
  A Rank Math SEO résumé (V2) still scored 30% (47/156). Diagnosis: the Phase-14 hand
 
4
 
5
  ---
6
 
7
+ ## 2026-06-27 — Coverage backstop: guaranteed ≥90% in Maximum ATS Mode (R35)
8
+
9
+ The curated denominator (R34) made the score clean (Rank Math 156→30, real keywords)
10
+ but coverage was still only 8/30 — the natural weave is model-dependent and plateaus.
11
+ Per the user's explicit directive ("if I run it, it's a fit; hit 90% regardless — you
12
+ don't know all my tasks; Maximum ATS Mode is on = treat missing keywords as
13
+ user-confirmed"), added a deterministic guarantee:
14
+
15
+ - `_append_skills_backstop()` + a `4.6` step in `generate_v2`: after natural weaving,
16
+ if coverage < 90, any remaining GENUINE (non-`_specialty_hit`) curated keyword is
17
+ placed into a clean Skills "Also:" line (additive — does NOT strip the woven
18
+ bullets). `judge_note` gets `+backstop`.
19
+ - This GUARANTEES ≥90% on any role the user runs, while the honesty line still holds:
20
+ `_specialty_hit` blocks specialized hands-on / regulated terms, and degrees, certs,
21
+ employers, seniority, and dates are never faked. The Skills line carries genuine
22
+ JD skills the user confirmed via Maximum ATS Mode; bullets stay natural.
23
+ - Verified: all Rank Math keywords (seo, wordpress, saas, user research, PRDs, …) are
24
+ ungated → placed → ≥90%. Tests: `test_backstop_places_missing_into_skills`,
25
+ `test_backstop_guarantees_target` (≥90% even when the weave model adds nothing).
26
+
27
+ ---
28
+
29
  ## 2026-06-27 — Phase 15: LLM-curated denominator + weave-to-90 + fixes (R34)
30
 
31
  A Rank Math SEO résumé (V2) still scored 30% (47/156). Diagnosis: the Phase-14 hand
src/resume_v2_natural.py CHANGED
@@ -617,6 +617,38 @@ def _judge_candidates(candidates: list[dict], jd_text: str,
617
  return 0
618
 
619
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
620
  # ── Coverage-aware weave pass (R32) ──────────────────────────────────────────
621
 
622
  def _coverage_weave_pass(new_src: str, decision: dict, jd_text: str, company: str,
@@ -944,6 +976,25 @@ def generate_v2(
944
  if _wove:
945
  judge_note += "+weave_pass"
946
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
947
  # 5. Compile
948
  comp: dict = {"compiled": False, "engine": None, "pdf_path": None}
949
  if compile_pdf and out_dir:
 
617
  return 0
618
 
619
 
620
+ # ── Coverage backstop (R35) ──────────────────────────────────────────────────
621
+
622
+ def _append_skills_backstop(src: str, keywords: list[str]) -> str:
623
+ """Deterministically place genuine missing keywords into a clean Skills line so
624
+ coverage reliably hits target. Additive only — does NOT strip the woven bullets
625
+ (unlike _place_sentences_structured). Used as the Maximum-ATS coverage guarantee
626
+ AFTER natural weaving; callers must pre-filter `_specialty_hit`/regulated terms."""
627
+ kws = [k.strip() for k in (keywords or []) if k and k.strip()]
628
+ if not kws:
629
+ return src
630
+ skills_start = src.find("\\section{SKILLS}")
631
+ if skills_start == -1:
632
+ return src
633
+ se = src.find("\\end{itemize}", skills_start)
634
+ if se == -1:
635
+ return src
636
+ close = src.rfind("}}", skills_start, se)
637
+ if close == -1:
638
+ return src
639
+ # Title-case-ish display, de-duplicated, capped for a clean line.
640
+ seen: set = set()
641
+ disp: list[str] = []
642
+ for k in kws:
643
+ kl = k.lower()
644
+ if kl not in seen:
645
+ seen.add(kl)
646
+ disp.append(k)
647
+ row = (" \\\\\n \\textbf{Also}{: " + latex_escape(", ".join(disp[:24]))
648
+ + "} % ats-backstop\n ")
649
+ return src[:close] + row + src[close:]
650
+
651
+
652
  # ── Coverage-aware weave pass (R32) ──────────────────────────────────────────
653
 
654
  def _coverage_weave_pass(new_src: str, decision: dict, jd_text: str, company: str,
 
976
  if _wove:
977
  judge_note += "+weave_pass"
978
 
979
+ # 4.6. Coverage backstop (R35): Maximum-ATS guarantee. After natural weaving, if
980
+ # coverage is still below target, deterministically place any remaining
981
+ # GENUINE (non-fit-gated) curated keywords into a clean Skills line. The user
982
+ # enables Maximum ATS Mode = "treat missing PM/Product/AI keywords as
983
+ # user-confirmed (interview-supportable)", so this is their explicit choice.
984
+ # `_specialty_hit` still blocks specialized/regulated terms; honesty held.
985
+ try:
986
+ _den = decision.get("curated_keywords") or skill_relevant_filter(
987
+ atomic_keywords(filter_scraped_noise(
988
+ decision.get("expected_terms", []), jd_text, company)))
989
+ _cov = external_coverage(_den, latex_to_text(new_src))
990
+ if _cov.get("pct", 0) < 90:
991
+ _missing = [t for t in _cov.get("missing", []) if not _specialty_hit(t)]
992
+ if _missing:
993
+ new_src = _append_skills_backstop(new_src, _missing)
994
+ judge_note += "+backstop"
995
+ except Exception as exc:
996
+ log.warning("Coverage backstop failed (non-fatal): %s", exc)
997
+
998
  # 5. Compile
999
  comp: dict = {"compiled": False, "engine": None, "pdf_path": None}
1000
  if compile_pdf and out_dir:
tests/test_phase14_ats.py CHANGED
@@ -267,6 +267,55 @@ def test_build_report_uses_curated_denominator():
267
  assert report["pct"] == 67
268
 
269
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
270
  # ── Parseability verifier ────────────────────────────────────────────────────
271
 
272
  def test_parseability_passes_good_resume():
 
267
  assert report["pct"] == 67
268
 
269
 
270
+ # ── R35: coverage backstop (Maximum-ATS guarantee) ───────────────────────────
271
+
272
+ def test_backstop_places_missing_into_skills():
273
+ from src import resume_v2_natural as v2
274
+ from src.external_ats import external_coverage
275
+ try:
276
+ from src.default_resume import get_default_resume_latex
277
+ from src.latex_resume import latex_to_text
278
+ except Exception:
279
+ pytest.skip("resume deps unavailable")
280
+ src = get_default_resume_latex()
281
+ out = v2._append_skills_backstop(src, ["seo strategy", "wordpress ecosystem"])
282
+ assert "ats-backstop" in out and "Also" in out
283
+ text = latex_to_text(out)
284
+ assert external_coverage(["seo strategy", "wordpress ecosystem"], text)["pct"] == 100
285
+ # Additive: the original document is preserved (no content stripped).
286
+ assert len(out) > len(src)
287
+
288
+
289
+ def test_backstop_guarantees_target(monkeypatch):
290
+ """Even when the weave model adds NOTHING relevant, the backstop guarantees >=90%
291
+ on a role the user runs (Maximum ATS). Real scoring; only the model is stubbed."""
292
+ from src import resume_v2_natural as v2
293
+ try:
294
+ from src.default_resume import get_default_resume_latex
295
+ except Exception:
296
+ pytest.skip("default resume unavailable")
297
+ CUR = ["alpha skill", "beta skill", "gamma skill", "delta skill", "epsilon skill",
298
+ "zeta skill", "eta skill", "theta skill", "iota skill", "kappa skill"]
299
+
300
+ def fake(self, cfg, system, user, max_tokens=2000):
301
+ s = (system or "").lower()
302
+ if "recruiter" in s and "screen" in s:
303
+ return json.dumps(CUR)
304
+ if "score each" in s:
305
+ return json.dumps({k: 80 for k in CUR})
306
+ if "pick the best" in s:
307
+ return "1"
308
+ # Generation / weave: generic bullets containing NONE of the curated terms.
309
+ return json.dumps({"psm": ["Did generic product work with no special terms."]})
310
+ monkeypatch.setattr(v2.LLMClient, "_call_with_cfg", fake)
311
+ monkeypatch.setattr(v2, "_fan_out_cfgs", lambda: [{"name": "M", "api_key": "x"}])
312
+ monkeypatch.setattr(v2, "_v2_model_cfg", lambda: {"name": "K", "api_key": "x"})
313
+ rep = v2.generate_v2(get_default_resume_latex(), "jd " + " ".join(CUR),
314
+ job_title="PM", company="Co", compile_pdf=False)
315
+ assert rep["pct"] >= 90, f"backstop must guarantee >=90, got {rep['pct']} ({rep['missing']})"
316
+ assert "backstop" in rep["judge_note"]
317
+
318
+
319
  # ── Parseability verifier ────────────────────────────────────────────────────
320
 
321
  def test_parseability_passes_good_resume():