Spaces:

saitejatirunagari
/

JAA-ATS-Tool

Running

App Files Files Community

JAA-ATS-Tool / AGENT_CONTEXT.md

saitejatirunagari

fix(phase4.4): allowlist keyword extraction — the real root-cause fix

22120b8 3 months ago

preview code

Raw

History Blame

24.5 kB

Agent Context — Job Automation Agent

Read this first. This document gives a new agent (Claude/Cursor/etc.) or developer everything they need to understand the project, its architecture, the design decisions that shaped it, what's working, what's broken, and where to start.

Last updated: 2026-06-16 Active branch: main Last deployed commit: see git log --oneline -1

1. Project mission in one sentence

Upload a PM resume → the tool scrapes Product Manager jobs from 170+ platforms, scores each against the resume, tailors a per-job resume that targets ATS systems (≥90% match), and logs everything to Google Sheets.

Stack: Python 3.11 + Streamlit UI + NestJS sidecar (ever-jobs) + multi-LLM pool (GLM / Kimi / Step / Qwen) + python-docx + reportlab + Hugging Face Spaces (Docker).

Live: https://huggingface.co/spaces/saitejatirunagari/JAA-ATS-Tool Repo: https://github.com/saitejatiru/JAA-ATS-Tool

2. End-to-end pipeline flow

User uploads resume.pdf
        ↓
[ui.py / Streamlit wizard]
   - 7-step wizard collects: roles, locations, platforms, freshness, AI score, sheets
   - "Start AI Job Search" triggers run_pipeline() in a thread
        ↓
[Scrapers] — src/scrapers/*.py + ever-jobs sidecar
   - Dedicated: linkedin / indeed / glassdoor / remotive / weworkremotely / naukri
   - All other 160+ platforms: EverJobsScraper → calls NestJS sidecar at localhost:3001
        ↓
[Deduplication] — URL + content-fingerprint
        ↓
[Job assessment] — src/job_assessor.py + multi-LLM pool
   - LLM extracts: relevance_score (1-10), matching_skills, missing_skills, ats_keywords
        ↓
[Resume tailoring] — src/resume_customizer.py
   - Per-job, parallel across LLM pool keys
   - Calls llm_client.customize_resume_fast with INDEXED bullets (role:idx)
   - LLM returns: professional_summary + rewritten_bullets + new_bullets
   - DOCX written via python-docx; iteration loop scores and retries (up to 3x)
   - Falls back to template + aggressive keyword injection if LLM weak
        ↓
[ATS scoring] — src/ats_scorer.py
   - Extracts JD keywords (regex + LLM-extracted from assessment)
   - Lemma + phrase matching against resume text
   - Returns: ats_score, jd_match_score, resume_quality, matched_kw, missing_kw, penalties
        ↓
[PDF generation] — src/pdf_writer.py
   - Windows + Word installed → docx2pdf (perfect fidelity, dev only)
   - HF Spaces / Linux → reportlab (replicates DOCX in document order)
        ↓
[Google Sheets logging] — src/gsheets.py
   - One row per job with ATS Before/After, resume path, JD URL, status
        ↓
[Run history] — data/output/run_history/<timestamp>.json
   - Snapshot of every run for re-display in UI

3. File map (what each file does)

Entry points

File	Purpose
`ui.py`	Streamlit app — the only user-facing entry point. ~2300 lines. Contains all UI, CSS, wizard, results display, history panel, and orchestration of `run_pipeline()`.
`main.py`	CLI entry (not user-facing on HF). Useful for debugging headless.
`start.sh`	Boots NestJS sidecar (ever-jobs) on :3001, then Streamlit on :7860.
`Dockerfile`	HF Spaces build: Python 3.11-slim + Node 20 + Playwright Chromium + ever-jobs clone.

Core pipeline (`src/`)

File	Purpose
`resume_parser.py`	Extracts text + contact info from PDF resume via pdfplumber.
`job_assessor.py`	LLM-driven relevance scoring per job. Returns dict with score / matching / missing / keywords.
`resume_customizer.py`	The heart of the tool. DOCX generation, LLM-tailoring loop, keyword injection, postcondition checks. ~1100 lines.
`llm_client.py`	Multi-model LLM client. Supports GLM / Kimi / Step / Qwen via OpenAI-compatible APIs. Handles JSON extraction, retries, parallel calls.
`model_pool.py`	Round-robin model selection across configured keys.
`ats_scorer.py`	Lemma + phrase-aware keyword matching. Scoring formula: JD-match 70% + quality 30%.
`pdf_writer.py`	DOCX → PDF (docx2pdf on Windows, reportlab fallback elsewhere).
`gsheets.py`	Google Sheets append. Service account or OAuth.
`excel_reporter.py`	XLSX export of results.
`run_history.py`	Save/load past run snapshots.
`job_history.py`	SQLite store for "already-applied" deduplication.
`app_logger.py`	Color-formatted logging.

Scrapers (`src/scrapers/`)

File	Purpose
`base.py`	Abstract Scraper class, common Job dataclass.
`linkedin.py` / `indeed.py` / `glassdoor.py` / `remotive.py` / `weworkremotely.py` / `naukri.py`	Dedicated scrapers with hand-tuned selectors.
`ever_jobs.py`	Adapter that calls the NestJS sidecar for 160+ other platforms.

Ever-jobs integration (`src/ever_jobs_bridge/`)

File	Purpose
`server.py`	Lifecycle management of the NestJS sidecar process.
`client.py`	HTTP client for the sidecar (host: localhost:3001).
`mapper.py`	Maps ever-jobs `IJob` shape → our `Job` dataclass.
`platforms.py`	Registry of all 170+ platforms grouped into Search Boards / ATS Platforms / Company Pages.

Research (deprecated, kept for reference)

File	Purpose
`research/deep_researcher.py` etc.	Earlier experiment — JD-driven deep research before assessment. Not in the production path.

Config

File	Purpose
`config.py`	Loads `.env`, builds MODELS list, GOOGLE config dict.
`.env`	API keys for LLMs (GLM/Kimi/Step/Qwen) + Google Sheets ID. Never committed.
`google_credentials.json` / `google_oauth_client.json`	Sheets auth files. Never committed.

Planning (gsd workflow artifacts)

Directory	Purpose
`.planning/ROADMAP.md`	Milestone + phases.
`.planning/REQUIREMENTS.md`	R1-R10 numbered requirements.
`.planning/STATE.md`	Project-level decisions.
`.planning/phases/<NN>-<slug>/<NN>-<MM>-PLAN.md`	Per-plan execution specs.

Tests + scripts

File	Purpose
`tests/fixtures/jds/*.txt`	Real production JDs used for verification (airtel, sumo_logic, edgeverve, aditya_birla, aisensy, navi, zenda, generic).
`scripts/verify_phase3.py`	Validate 2 hardest JDs hit ≥90%.
`scripts/verify_phase3_all4.py`	Validate all 4 failing JDs hit ≥90%.
`scripts/verify_weak_llm_recovery.py`	Validate weak-LLM + injection still recovers to 87-91%.

Output (gitignored)

Path	Purpose
`data/resume/resume.pdf`	User's uploaded resume.
`data/output/resumes/YYYY-MM-DD/<Company>_<Role>.docx` and `.pdf`	Generated resumes.
`data/output/reports/*.xlsx`	Excel summary per run.
`data/output/run_history/*.json`	Run snapshots for history panel.
`data/logs/tailoring_YYYY-MM-DD.jsonl`	Per-job diagnostic log (added Phase 3).
`data/research_cache/`	LLM response cache to save tokens.

4. The 3 design phases delivered

Phase 1 — Scale to 170+ platforms (committed)

Integrated github.com/ever-jobs/ever-jobs as a NestJS sidecar
Adapter pattern in EverJobsScraper
Cross-platform dedup via URL + content fingerprint
Streamlit UI grouped multiselect with 170 platforms across 3 categories
ATS scoring confirmed adequate (R3 closed — ever-jobs has no resume scorer)

Phase 2 — HR-grade resume + bullet-rewriter contract (committed)

Trigger: User saw a 1-page truncated resume with header "Internal Product" (broken name extraction), missing BYJU's/ML Edutech roles, no Education, empty Core Competencies, plus a spam "Additional relevant skills: adani • godrej • yakult" footer.

Fixes:

New _extract_candidate_name() handles ALL CAPS names and PDF letter-spacing
Experience parser walks the whole experience blob, finds all date ranges (including line-wrapped "Oct 2021 – Dec\n2022"), splits into roles, preserves sub-section headers as bold §§HEADER§§ markers
LLM contract v2: returns rewritten_bullets["0:3"] keyed by role:idx instead of generic highlights block
_inject_missing_keywords no longer appends a footer — weaves missing skills into a closing sentence of the Professional Summary
Postcondition _assert_no_dump_footer raises if any banned section header or "Additional relevant skills" line slips through
User explicitly directed: NO CORE COMPETENCIES section — keywords live only in summary + bullets

Net: Original 57/100 → Tailored 92/100 on AiSensy JD (verified with handcrafted LLM response).

Phase 4.4 — Allowlist keyword extraction (THE root-cause fix)

The 3-day loop's true cause: extract_jd_keywords counted every capitalized JD word as a keyword. Company names / locations / stock tickers (Experian, Dublin, FTSE, Ascend) polluted the denominator. To hit 90%, the weaver injected those non-skills as garbage ("leveraging FTSE") → fake 95%, real ~70%. Blocklist tuning was whack-a-mole.

Fix: PM_SKILL_TAXONOMY in src/ats_scorer.py — ~250 curated real PM skills. extract_jd_keywords returns a token ONLY if it's in the taxonomy (allowlist). Plus weaver/injector now require _is_actual_skill. Result: score reflects real skill coverage; no garbage possible on any JD.

Verified: 7 diverse JDs (incl. 3 never-seen) all hit 92-95% with zero garbage, worst-case weak LLM. The user's actual Experian resume scored honest 69 under the fix (was fake 95); regenerated through the pipeline → 94 clean.

If scores ever drop again: check whether the JD's real skills are in PM_SKILL_TAXONOMY. If a legit skill is missing from the taxonomy, ADD it there — do NOT add company-specific words to any blocklist.

Phase 4 — Canonical resume format (committed)

Trigger: User asked for ONE canonical visual format applied to every tailored resume — kills format drift, orphan-line bugs, sub-section noise.

New architecture:

New src/resume_model.py — Resume / Role / Education / Contact dataclasses (the single source of truth)
New src/resume_parser_v2.py — one-time PDF → Resume parser (cached to data/resume/_parsed.json). Flattens sub-sections into flat bullets per role.
New src/resume_renderer.py — canonical DOCX renderer with locked visual (20pt centered name, indigo section headers with thin underline, 5-7 flat bullets per role, hanging indent)
New LLMClient.tailor_resume_v4() — input is Resume JSON, output is tailored Resume JSON. No more indexed role:idx keying. LLM picks 5-7 best bullets per role.
_generate_resume() tries v4 path first; falls back to legacy bullet-rewriter on any failure.

Verified (handcrafted v4 simulations on all 4 failing JDs): Airtel 94, Sumo Logic 94, EdgeVerve 94, Aditya Birla 93. All 2 pages (was 5-6).

Trade-off the user accepted: sub-section detail (NIAT Revamp, AI Chatbot, etc.) is no longer preserved in tailored output. Bullets are flat per role. Long-form detail lives in LinkedIn / portfolio.

Phase 3 — ATS floor 90%+ on real LLM runs (committed)

Trigger: Real LLM in production averaged ~60% (airtel 79, Aditya Birla 48, EdgeVerve 63, Sumo Logic 52) — Phase 2 v2 contract worked but real LLM rewrites covered fewer keywords than handcrafted tests.

Fixes:

Rules-based lemmatizer (no NLTK): automated↔automation, roadmaps↔roadmap
Phrase-aware matching: multi-word JD keywords match exact OR all-lemmas-in-5-token-window
Aggressive JD noise filter: drops 30+ categories (adjectives like proven/solid, modals like will/must, process verbs like perform/establish, JD section words like what/doing/inc/bachelor)
LLM prompt updated with liberal-keyword policy (user authorized): assume candidate has touched any JD-named common tool over 5+ years
Recruiter-pitch opener: every Professional Summary opens with "Strong-fit candidate for <role> at <company>: ..." — visible to humans + AI screeners (the safe alternative to the user's hidden-text request, which modern ATS auto-rejects)
Aggressive keyword injection: trusts JD extractor's filter, drops only lemmatizer artifacts and short tokens
Diagnostic JSONL log: per-job record of JD keywords / matched / missing / LLM schema / pitch detection — for debugging future score regressions
UI: removed "🤖 Job Automation Agent" title; added baseweb CSS overrides for dark-on-dark dropdowns

Verified results (handcrafted v3 simulations against the 4 failing JDs):

Airtel: 79 → 92 (+13pp)
EdgeVerve: 63 → 91 (+28pp)
Sumo Logic: 52 → 92 (+40pp)
Aditya Birla: 48 → 91 (+43pp)

5. Known issues — status

After the user audited a real production output (Aditya Birla LLM-tailored resume, reported 95% by our scorer), four concrete bugs were identified. Bugs A, B, and C have been fixed. Bug D remains.

Bug A: JD table-header words leak into keyword injection — ✅ FIXED

Was: Aditya Birla JD's tabular format (KRA (Accountabilities) (Max 1325 Characters), Supporting Actions) leaked Accountabilities, Max, Characters, Actions, Show, Supporting, KRA as "keywords", which the injection wove into the Summary as if they were skills.

Fix: Added these terms to _JD_NOISE_WORDS in src/ats_scorer.py. Aditya Birla JD now extracts a clean list of 18 real skill keywords (down from 30+).

Bug B: Multi-line bullets leave orphan continuation lines — ✅ FIXED

Was: PDF-wrapped multi-line bullets got indexed as separate bullets. LLM rewriting 0:0 left orphan continuation text rendering as ghost-bullets.

Fix: New _is_bullet_continuation() heuristic in src/resume_customizer.py joins wrapped lines into the previous bullet at extract-time. Heuristic uses: lowercase start / digit start / continuation symbols (→ + % & ( [ { ) / Title-Case word count. Bullets are now ~234 chars (joined) instead of split into fragments.

Bug C: Date wrapping creates duplicate dates — ✅ FIXED

Was: BYJU's role rendered as "Think & Learn ... | IndiaOct 2021 – Dec | Oct 2021 – Dec 2022" — partial date leaked into the company string because the wrapped date didn't match the date regex.

Fix: New partial_date_re in _extract_experience_sections matches partials ("Oct 2021 – Dec" without trailing year) and strips them from the header line before splitting role/company. Orphan year-only body lines ("2022") are also skipped. Result: company is now "Think & Learn Pvt. Ltd. (BYJU'S) | Bengaluru, India", dates is "Oct 2021 – Dec 2022".

Bug D: LLM sometimes returns v1 schema in production — ⏳ STILL OPEN

The new v2 contract asks for rewritten_bullets["0:3"] but smaller models (Step / Qwen smaller variants) sometimes return the old v1 experience_bullets shape. Backward-compat handles it but the result is generic bullets that don't use JD-specific phrasing.

Fix needed: stricter JSON-schema enforcement in customize_resume_fast (retry if v2 fields are missing); OR add a converter that maps v1 → v2 by best-effort matching of "highlight" bullets to original bullets via SequenceMatcher fuzzy substring match.

6. How to verify changes

Goal	How
Test scorer changes alone	`PYTHONPATH=. python scripts/verify_phase3.py` (uses handcrafted LLM v3 sims)
Test all 4 failing JDs	`PYTHONPATH=. python scripts/verify_phase3_all4.py`
Test injection works on weak LLM output	`PYTHONPATH=. python scripts/verify_weak_llm_recovery.py`
Test UI locally	`streamlit run ui.py --server.port 8502`
Read tailoring diagnostic log	`cat data/logs/tailoring_$(date +%Y-%m-%d).jsonl
Verify the no-Skills-section postcondition	All generated DOCX files run through `_assert_no_dump_footer` — raises on banned sections
Push to HF Spaces	`git push hf main` (auto-rebuilds in ~5 min)

7. LLM model pool

Configured via .env:

GLM_KEY_1=...      ← Primary (GLM-4.6) — best JSON adherence
KIMI_KEY_1=...     ← Kimi K2 — fast, good for parallel calls
STEP_KEY_1=...     ← Step-1 — fallback
QWEN_KEY_1=...     ← Qwen — fallback

model_pool.py round-robins across configured keys. Calls run in parallel via ThreadPoolExecutor (up to 6 workers).

Two LLM call sites:

LLMClient.assess_job() — fast model, JSON output: {relevance_score, matching, missing, ats_keywords}
LLMClient.customize_resume_fast() — fast model, JSON output (v2 contract): {professional_summary, rewritten_bullets, new_bullets, key_achievements}

If JSON parse fails, retries up to 2 times per call. If all retry, _empty_customization() returns empty schema and the pipeline falls through to template + injection.

8. Critical user-stated policies (do not change without asking)

No CORE COMPETENCIES / Skills section anywhere in the tailored resume. Keywords MUST live in Summary + experience bullets only. The _assert_no_dump_footer postcondition enforces this.
Liberal keyword inclusion is authorized. When a JD names a common PM tool (Jira/Figma/Mixpanel/etc.) or methodology (PRDs/sprint/MLOps), the LLM is told to include it assuming the candidate has touched it. The user owns interview-side risk.
No hidden text / prompt injection. User asked for white-on-white prompts targeting AI screeners — refused because modern ATS detects this pattern and auto-rejects + blacklists. Replaced with the visible recruiter-pitch opener.
No "Tailored for X at Y" footer or banner. Removed in Phase 2.
All 4 candidate roles must be preserved (NxtWave + BYJU's-1 + BYJU's-2 + ML Edutech). The experience parser must handle all dated headers.
Output is BOTH .docx AND .pdf. PDF via docx2pdf when on Windows + Word; reportlab everywhere else (HF Spaces). PDFs must include table content via document-order body iteration.

9. Deployment + branches

GitHub: https://github.com/saitejatiru/JAA-ATS-Tool (branch: main)
HF Spaces: https://huggingface.co/spaces/saitejatirunagari/JAA-ATS-Tool (auto-builds on git push hf main)
Single main branch; no PR workflow. Direct commits to main, pushed to both remotes.
HF rebuild takes ~5-7 min. The build log shows the commit SHA at the top.

10. Where to start as a new agent

If your task is fixing low ATS scores:

Read src/ats_scorer.py score_resume() and extract_jd_keywords()
Read src/resume_customizer.py _generate_resume() (the iteration loop) and _inject_missing_keywords()
Check data/logs/tailoring_*.jsonl for what the LLM actually returned
Test changes with scripts/verify_phase3_all4.py

If your task is fixing format / layout bugs:

Read src/resume_customizer.py _write_docx() and _extract_experience_sections()
Read src/pdf_writer.py _reportlab_render()
Generate a test DOCX locally: see scripts/verify_phase3.py for the pattern
Inspect with python-docx or open in Word/Preview

If your task is adding new LLM behavior:

Read src/llm_client.py _resume_customize_prompt() — the v2 contract prompt
Update prompt and schema validator (_customization_valid())
Verify backward compatibility — older models may return v1 schema

If your task is UI changes:

ui.py is ~2300 lines — search for the relevant section by keyword
CSS is at the top (~lines 30-200); be careful with [data-baseweb=...] overrides
The 7-step wizard logic uses st.session_state.setup_step and persistent _cfg_* keys

11. The "what to do next" punch list

In priority order:

~~Fix Bug A (JD table-header words leaking into injection)~~ — ✅ DONE (commit pending). Accountabilities/Max/Characters/Show/Supporting/KRA/KRAs/Actions/Result/Areas/Moving/Handing added to _JD_NOISE_WORDS.
~~Fix Bug B (orphan continuation lines)~~ — ✅ DONE. New _is_bullet_continuation() heuristic joins wrapped lines at extract-time.
~~Fix Bug C (duplicate date / partial date in company string)~~ — ✅ DONE. New partial_date_re strips partial month-year fragments from header lines; orphan year-only body lines skipped.
Address Bug D (v1 schema fallback hurts quality) — STILL OPEN. Options: enforce v2 schema with stricter retry, OR add a v1→v2 converter that fuzzy-matches each "highlight" bullet to its closest original bullet via SequenceMatcher and rewrites the original in place. ~1-2 hr.
Verify by re-running on the same 4 JDs in production, then asking the user to share the new docx for honest audit.
Adopt atsresume PDF layout directly (optional). Either replicate their exact HTML/CSS in a Jinja2+WeasyPrint pipeline, or iframe their deployed instance for export. Current format adopts the conventions but not the exact visual style.

12. Conventions you should follow

Read HISTORY.md for the chronological log of fixes — gives context on WHY decisions were made.
Read .planning/REQUIREMENTS.md — R1-R10 are the named requirements.
Don't change git history — single main branch, no rebases.
Always test against the real resume PDF at C:\Users\Nxtwave\Desktop\resume\Saiteja_Tirunagari_Resume A 26 - Copy.pdf (Saiteja's actual resume) — that's the gold reference.
Commit messages: Conventional Commits style (fix(scope): ..., feat(scope): ...), with a Co-Authored-By trailer if AI-assisted.
No emojis in code or commit messages unless the user explicitly asks for them.
No hidden text / prompt injection in resumes — see policy #3.
The user wants honesty over false confidence. If a fix only addresses part of the problem, say so. If our scorer reports 95% but real ATS would score 85%, say so.

13. Quick command reference

# Run UI locally
streamlit run ui.py --server.port 8502

# Verify Phase 3 against 2 hardest JDs
PYTHONPATH=. python scripts/verify_phase3.py

# Verify all 4 failing JDs
PYTHONPATH=. python scripts/verify_phase3_all4.py

# Generate a test resume (handcrafted v3 LLM sim)
PYTHONPATH=. python scripts/verify_weak_llm_recovery.py

# Deploy to GitHub + HF Spaces
git add -A
git commit -m "fix(scope): short description"
git push origin main
git push hf main

# Check HF build status
# → Open https://huggingface.co/spaces/saitejatirunagari/JAA-ATS-Tool and check the Logs tab

# Tail tailoring diagnostic log (after a run)
type "data\logs\tailoring_2026-06-16.jsonl"   # Windows
cat data/logs/tailoring_2026-06-16.jsonl       # Linux/Mac/git-bash

14. Glossary of project-specific terms

Term	Meaning
v1 contract	Original LLM output schema: `{summary, core_competencies, experience_bullets, key_achievements}`. Deprecated but still supported via backward compat.
v2 contract	Current LLM output schema (Phase 2): `{summary, rewritten_bullets, new_bullets, key_achievements}`. Bullets keyed by `role_idx:bullet_idx`.
v3 contract	Phase 3 prompt with liberal keyword policy + recruiter-pitch opener requirement. Same schema as v2.
Recruiter pitch	First sentence of Professional Summary, format: "Strong-fit candidate for at : years of directly applicable to <3 JD requirements>."
Indexed bullets	`[(role_idx, bullet_idx, role_name, bullet_text), ...]` tuples passed to the LLM so it can reference specific original bullets.
§§HEADER§§ / §§META§§	Internal markers in `_extract_experience_sections` for sub-section bold headers and italic meta lines (like "Scope:").
JD noise filter	Set of words `_JD_NOISE_WORDS` in `ats_scorer.py` that get dropped from keyword extraction (e.g. proven/solid/will/bachelor).
Postcondition	`_assert_no_dump_footer` runs at end of every `_generate_resume` to catch banned patterns (Skills sections, dump footers).
Lemma matching	Rules-based stemmer: `automated`/`automation`/`automate` all collapse to `automat`. Enables forgiving keyword matching.
Phrase matching	Multi-word JD keywords match if all component lemmas appear within a 5-token sliding window in the resume.
Aggressive injection	Trusts JD extractor's noise filter — injects every still-missing keyword (capped at 15) into the Summary's closing sentence.

End of context document. If anything in here contradicts current code, the code is the source of truth — but please update this doc when you finish your task.