--- title: JAA ATS Tool emoji: πŸ€– colorFrom: blue colorTo: indigo sdk: docker app_port: 7860 pinned: false license: mit --- # Job Automation Agent β€” PM Edition Automated Product Manager job search, AI-powered assessment, ATS-optimized resume generation, and Google Sheets reporting β€” all in one pipeline. --- ## What It Does | Step | What Happens | |------|--------------| | 1 | Parses your PDF resume (Kimi-K2.6, ~5s) | | 2 | Scrapes PM-only jobs from **LinkedIn, Indeed, Glassdoor, Remotive, WeWorkRemotely** β€” ALL selected roles Γ— locations, last N days | | 3 | Filters non-PM roles at scrape time; dedup by URL + (title, company) + 30-day SQLite history; platform-level total cap | | 4 | Fetches FULL job descriptions in bulk (parallel HTTP / single browser session) | | 5 | Assesses ALL jobs using 7 parallel AI models (10-model pool via NVIDIA API) | | 6 | Generates ATS-optimized resumes for ALL jobs β€” DOCX **and PDF**, After β‰₯ Before guaranteed, target 95% | | 7 | Writes everything to your Google Sheet with direct job links | | 8 | Saves local Excel report + resumes in `data/output/resumes/YYYY-MM-DD/` | | 9 | Saves run to history β€” reload past runs in one click from the History panel | --- ## Quick Start ```powershell # 1. Install dependencies pip install -r requirements.txt playwright install chromium # 2. Copy and fill in your API keys copy .env.example .env # Edit .env with your NVIDIA_API_KEY, GOOGLE_SHEET_ID, etc. # 3. Place your resume PDF # Copy your resume to: data/resume/resume.pdf # 4. Connect Google (one-time browser login) python connect_google.py # 5. Run a test (5 jobs) # In config.py: ASSESSMENT["test_mode"] = True python main.py # 6. Run full production mode (all 100+ PM jobs) # In config.py: ASSESSMENT["test_mode"] = False python main.py # 7. Or use the Streamlit UI streamlit run ui.py ``` --- ## ever-jobs Integration (160+ Platforms) This project integrates the [ever-jobs](https://github.com/ever-jobs/ever-jobs) NestJS service, which provides REST API access to 160+ job board scrapers. ### Prerequisites - **Docker Desktop** (preferred): [Install Docker Desktop](https://docs.docker.com/desktop/install/windows-install/) - OR **Node.js 24.x** for npm subprocess fallback ### Setup (one-time) ```bash # Clone ever-jobs to vendor/ directory (done automatically during setup) git clone https://github.com/ever-jobs/ever-jobs.git vendor/ever-jobs --depth=1 # Start via Docker (preferred) cd vendor/ever-jobs && docker compose up -d # Verify API is running curl http://localhost:3001/health # OR from Python: python -c "from src.ever_jobs_bridge.server import is_running; print(is_running())" ``` ### Automatic Startup The pipeline calls `ensure_running()` automatically before any ever-jobs platforms are scraped. It tries Docker first, falls back to `npm run start` if Docker is unavailable. ### Platform Selection The UI has a single **"🌐 Job Platforms"** section with three groups. Selecting LinkedIn, Indeed, Glassdoor, Remotive, WeWorkRemotely, or Naukri uses their dedicated high-quality scrapers; all other platforms go through the ever-jobs REST API. | Group | Count | Description | |-------|-------|-------------| | Search Boards | 94 | General job boards. India-relevant defaults pre-selected. | | ATS Platforms | 37 | Greenhouse, Lever, Workday etc. β€” companies post jobs here. NOT resume scoring. | | Company Pages | 39 | Direct career pages (Flipkart, Swiggy, Amazon, Google, etc.) | **India default platforms:** linkedin, naukri, indeed, glassdoor, google, bdjobs, internshala, bayt, iimjobs, foundit ### ATS Clarification > **Important:** "ATS" in ever-jobs means Applicant Tracking System **PLATFORMS** > (tools companies use to post jobs, like Greenhouse or Lever). > It does **NOT** mean ATS resume scoring. > > Our ATS resume scoring (`src/ats_scorer.py`) uses a 70% JD keyword match + > 30% resume quality hybrid and is **NOT changed** by this integration. ### Performance Notes | Selection | Expected Time | |-----------|---------------| | Default 10 platforms | ~1–3 minutes | | 30 platforms | ~3–5 minutes | | 100+ platforms | 5–10 minutes (warning shown in UI) | Playwright-based scrapers (some company pages) are inherently slower than API-based boards. ### Cross-Platform Deduplication In addition to URL-based dedup, a **content fingerprint** (SHA-256 of normalized title+company) catches cross-platform duplicates. For example, the same "Product Manager at Google" posting on LinkedIn AND Greenhouse (different URLs) is detected and deduplicated. --- ## Project Structure ``` Job Automation Agent/ β”œβ”€β”€ main.py # Main pipeline (6-step orchestrator) β”œβ”€β”€ config.py # All configuration β€” models, platforms, ATS settings β”œβ”€β”€ ui.py # Streamlit UI (4 tabs: Search, Results, Job Details, Research) β”œβ”€β”€ connect_google.py # One-time Google OAuth setup β”œβ”€β”€ setup_google.py # Service account alternative β”œβ”€β”€ requirements.txt β”œβ”€β”€ .env # API keys (never commit) β”œβ”€β”€ .env.example # Template for .env β”‚ β”œβ”€β”€ src/ β”‚ β”œβ”€β”€ resume_parser.py # PDF β†’ plain text (pdfplumber) β”‚ β”œβ”€β”€ llm_client.py # GLM 5.1 wrapper (profile extract, resume customize) β”‚ β”œβ”€β”€ model_pool.py # 10-model parallel AI pool (NVIDIA API) β”‚ β”œβ”€β”€ job_assessor.py # Phase 1 keyword scoring + Phase 2 LLM assessment β”‚ β”œβ”€β”€ job_history.py # SQLite dedup store (data/job_history.db) β”‚ β”œβ”€β”€ ats_scorer.py # Hybrid ATS scoring (70% JD match + 30% quality) β”‚ β”œβ”€β”€ resume_customizer.py # LLM-tailored DOCX resume generator β”‚ β”œβ”€β”€ gsheets.py # Google Sheets + Drive upload β”‚ β”œβ”€β”€ excel_reporter.py # Local Excel report β”‚ β”œβ”€β”€ odysseus_llm_core.py # Deep research LLM core β”‚ β”‚ β”‚ β”œβ”€β”€ ever_jobs_bridge/ # ever-jobs NestJS REST API adapter β”‚ β”‚ β”œβ”€β”€ __init__.py # Package init β”‚ β”‚ β”œβ”€β”€ server.py # Docker/npm server lifecycle (start/stop/health) β”‚ β”‚ β”œβ”€β”€ client.py # HTTP client for POST /api/jobs/search β”‚ β”‚ β”œβ”€β”€ mapper.py # IJob JSON β†’ Job dataclass mapper β”‚ β”‚ └── platforms.py # 170 platform catalog with group metadata β”‚ β”‚ β”‚ β”œβ”€β”€ scrapers/ β”‚ β”‚ β”œβ”€β”€ base.py # Job dataclass + BaseScraper + is_pm_role() filter β”‚ β”‚ β”œβ”€β”€ linkedin.py # LinkedIn scraper (requests + BeautifulSoup) β”‚ β”‚ β”œβ”€β”€ indeed.py # Indeed scraper (Playwright for JS rendering) β”‚ β”‚ β”œβ”€β”€ glassdoor.py # Glassdoor scraper (Playwright) β”‚ β”‚ β”œβ”€β”€ naukri.py # Naukri (disabled β€” blocked by Akamai) β”‚ β”‚ └── ever_jobs.py # EverJobsScraper (REST adapter for 160+ platforms) β”‚ β”‚ β”‚ └── research/ β”‚ β”œβ”€β”€ deep_researcher.py # Odysseus IterResearch engine (Thinkβ†’Searchβ†’Extractβ†’Synthesize) β”‚ └── search.py # DuckDuckGo + Bing fallback, 12h cache β”‚ └── data/ β”œβ”€β”€ resume/resume.pdf # Your resume (add this) β”œβ”€β”€ job_history.db # Dedup SQLite DB (auto-created) β”œβ”€β”€ research_cache/ # 12h DuckDuckGo result cache └── output/ β”œβ”€β”€ resumes/ # Generated DOCX resumes (Company_JobTitle.docx) └── reports/ # Excel reports ``` --- ## Configuration (`config.py`) ### AI Models (10-model pool via NVIDIA API) | Model | Speed | Phase 2 | Notes | |-------|-------|---------|-------| | Kimi-K2.6 | ~5s/batch | βœ… | Fastest, handles most work | | Step-3.7-Flash | ~8-35s | βœ… | | | Qwen3.5-397b | ~9s | βœ… | | | Qwen3.5-122b-v2 | ~12s | βœ… | | | GPT-OSS-120b | ~11s | βœ… | | | Qwen3.5-122b | ~40s | βœ… | | | DeepSeek-v4-Pro | ~42s | βœ… | | | DeepSeek-v4-Flash | ~229s | ❌ | Too slow for phase 2 | | GLM-5.1 | ~234s | ❌ | Used for resume parsing only | | MiniMax-M2.7 | ~908s | ❌ | Blocked/rate-limited | ### Key Settings ```python ASSESSMENT = { "min_score_for_llm_resume": 6, # LLM-tailored resume for score >= this "generate_all_resumes": True, # Template resume for ALL PM jobs "max_llm_resumes": 30, # Max LLM resumes per run "dedup_days": 30, # Skip jobs seen in last 30 days "test_mode": True, # ← Set False for full production run "test_jobs_limit": 5, # Max jobs in test mode } ``` --- ## ATS Scoring Method Hybrid scoring: **70% JD Match + 30% Resume Quality** - **JD Match (70%)**: Extract keywords FROM the specific job description β†’ match against resume using word-boundary regex (`(?.hf.space`) + `API_SECRET_TOKEN`. Then open any job β†’ **Run** β†’ tailored resume download. **Resume LaTeX (recommended):** paste your resume's LaTeX source in Options. When present it is **prioritised over the PDF** β€” we extract its text for keyword matching, inject the honestly-includable JD keywords **distributed across a Summary sentence, Experience bullets, and one compact competencies line** (never a dump), and compile it to a **clean PDF** (Tectonic on the Space) you download directly (plus the modified `.tex`). This fixes the generated-PDF design issues because the layout is your own LaTeX. Anti-faking is identical: certs, seniority, employers, and specialised hands-on engineering terms are never injected. Verify: `python scripts/verify_latex_resume.py`. **Recruiter-grade placement + resilient Run (Phase 7, v1.4.0):** JD extraction now strips page/extension UI chrome (Simplify/Jobalytics overlays) so junk like "Show Match Details" / "People Clicked Apply" never becomes a keyword; the DOCX Skills section is one capped line per category (18-28 items, no repeated "Core Competencies:" dumps) with surplus keywords woven into Summary + Experience; and clicking **Run** now survives the popup closing β€” the background owns the run and the popup restores the spinner/result on reopen. Verify: `python scripts/verify_clean_jd_extraction.py`, `verify_skills_distribution.py`, `verify_resilient_run.py`. **Trust + usability (Phase 8, v1.5.0):** - **Download a real PDF, not just DOCX.** The DOCX path now always renders a PDF sidecar (reportlab on Linux/HF) so the PDF button works even without LaTeX; the LaTeX path compiles a clean PDF via Tectonic and, if compilation fails, tells you why (no engine vs LaTeX error) while still offering the `.tex`. - **Your resume is preserved β€” non-destructive tailoring is the default.** Role titles, companies, dates, and your existing bullets are kept VERBATIM; keywords are added only in the Professional Summary and as ≀3 appended lines at the end of each experience. Nothing you wrote is renamed or rewritten. - **Persistent left side panel (Jobalytics-style).** The UI is now a collapsible panel docked to the left of the page instead of a popup that vanishes β€” click **Run**, then click anywhere on the page; the run keeps going and stays visible. - **History.** A History list in the panel shows your previous generations per job (title/company/time/scores/status); click any entry to restore it and re-download DOCX/PDF/.tex. Verify: `python scripts/verify_pdf_download.py`, `verify_non_destructive.py`, `verify_side_panel.py`, `verify_history.py`. **API endpoints** (`api_server.py`, port 7860, `X-Api-Token` header): - `GET /api/health` β†’ `{status: ok}` - `POST /api/generate` β€” multipart `jd_text` + (`resume_latex` **or** `resume` PDF; LaTeX wins) β†’ tailored resume. PDF path returns base64 DOCX/PDF; LaTeX path returns compiled `pdf_b64` + `tex_b64` + external coverage report. - `POST /api/repair-with-feedback` β€” **External ATS Feedback Repair Mode**: paste Jobalytics/Simplify feedback (score + missing keywords); every missing keyword is risk-classified by `candidate_fit` and woven *honestly* into bullets/skills (or injected into your LaTeX when `resume_latex` is supplied). **Feedback-repair honesty rules** (never relaxed): - LOW β†’ auto-weave; MEDIUM β†’ auto-weave + review flag; **HIGH β†’ not added, returned as "needs your confirmation"; BLOCKED β†’ excluded.** No fake skills/seniority/employers/degrees/certs. - Scores come from the re-parsed exported file; READY needs internal AND independent β‰₯ 90 AND ATS readability β‰₯ 90 AND parse validation. **Statuses:** `READY_95_EXTERNAL_ALIGNED` (β‰₯90 both + pasted gaps mostly resolved) Β· `READY_90_PLUS_REVIEW_RECOMMENDED` (medium-risk added) Β· `NEEDS_USER_INPUT` (95 needs high-risk terms you must confirm) Β· `NOT_ELIGIBLE_LOW_FIT` (can't reach 90/95 without fabrication). Verify: `PYTHONPATH=. python scripts/verify_feedback_repair.py` (deterministic; proves blocked/high terms never enter the resume). ### Maximum ATS Mode (User-Confirmed Skill Expansion) For your target role family (Product / Product Manager / AI Product Manager / SaaS / B2B), the uploaded resume is treated as an **incomplete base profile**. Enable **Maximum ATS Mode** (extension toggle, on by default; API field `maximum_ats_mode=1`) to treat normal PM/Product/AI/agile vocabulary β€” *AI, ML, generative AI, prompt design, model evaluation, QA, experimentation, data-driven, analytics, roadmap, product strategy, product ownership, agile, scrum, user stories, acceptance criteria, PRD, stakeholder management, SaaS, B2B, enterprise platform, product discovery, go-to-market, retention, growth*, etc. β€” as **user-confirmed / interview-supportable** and aggressively place them across Summary, Skills, Key Achievements and Experience. It targets **95** external ATS and treats **anything below 90** for in-family roles as a failure state to keep repairing (unless the only gaps are genuinely blocked). **Still never fabricated** (hard boundaries, unchanged): degrees, certifications/ licenses, employers, titles, years/seniority, regulated credentials, and specialized hands-on engineering/security tools (those surface as **"needs your confirmation"**, with a one-click **Confirm & regenerate**). - Both endpoints accept `maximum_ats_mode` / `user_confirmed_expansion`, `confirmed_terms`, `target_external_score` (default 95). Backward compatible. - Responses include a rich **coverage report** (per keyword: category, risk, disposition, the resume section it was placed in, and why anything was excluded), `still_missing_repairable`, and a plain-English `below_target_explanation`. - Confirmed expansion terms persist to the **Candidate Vault** (`user_confirmed`) so future resumes treat them as safe. **Maximum-ATS statuses:** `READY_MAX_ATS_95_PLUS` (gates pass + external coverage β‰₯ 95) Β· `READY_90_PLUS_EXTERNAL_ALIGNED` (gates pass + coverage β‰₯ 90) Β· `BELOW_TARGET_REPAIRABLE` (below target, remaining gaps LOW/MEDIUM β†’ keep repairing) Β· `NEEDS_USER_CONFIRMATION` (only high-risk-but-supportable terms left). **External-coverage is the success signal (not the internal score).** Our internal scorer uses a narrow taxonomy (β†’ 90%+ easily); external checkers extract a broad 40-46 term set (β†’ can be ~54% on the same resume). So in Maximum ATS Mode the system builds a broad Jobalytics-style **expected** set (`src/external_ats.py`), **physically guarantees** every *includable* term into the exported DOCX (Skills verbatim + woven into Experience bullets; HIGH/BLOCKED stay gated), re-parses the file, and **measures coverage from the export**. Status is then `READY_MAX_ATS_95_PLUS` (β‰₯95) / `READY_90_PLUS_EXTERNAL_ALIGNED` (β‰₯90) / `BELOW_TARGET_REPAIRABLE` β€” an internal-96 / external-54 result is treated as a bug, never "done". Both endpoints return `external_coverage` + a per-term `coverage_report` (keyword / found in export / section / why-missing) and log `maximum_ats_mode`. Verify (deterministic, no keys): - `PYTHONPATH=. python scripts/verify_maximum_ats.py` β€” PM/AI terms become user-confirmed; certs/seniority/engineering stay gated; coverage improves. - `PYTHONPATH=. python scripts/verify_max_ats_coverage.py` β€” the 26/46 live-failure regression: exported DOCX covers β‰₯90% of includable PM terms; credentials/fake seniority excluded; status driven by external coverage. --- ## Generation Modes (V1 / V2) Two resume tailoring modes, selectable per surface: ### V1 β€” Structured Keyword Placement (default) Uncapped JD keyword extraction + ordered placement into the hardcoded resume (Phase 9). Keywords are appended as comma-separated `\resumeItem` lines in a waterfall order: Summary 15-20, BYJU's PSM 25-30, PS 25-30, ML Edutech 8-12, Skills Other 15-20, Projects, NxtWave. Fast (no LLM call for placement); keywords appear verbatim. ### V2 β€” Natural AI Sentence Integration Same keyword extraction + waterfall allocation as V1, but an LLM (Kimi-K2.6 by default, ~5s) generates **natural sentences** that weave the keywords into genuine-sounding experience bullets. Reads as natural prose rather than keyword lists; honesty boundaries preserved; slightly slower (one LLM call + compile). Falls back to V1 comma placement if the LLM call fails. ### Selecting a version | Surface | How to choose | |---------|---------------| | `/api/generate` | `version=v1` or `version=v2` form field; default from `GEN_VERSION_DEFAULT` env (v1) | | HF Streamlit | V1/V2 radio on the home page; bulk pipeline honors the selection | | Chrome extension | Default in Options (`gen_version_default`) + per-run toggle in the popup | | Telegram bot | `/v1`, `/v2` commands set per-user default; `/mode` shows current | ### Env knobs | Variable | Default | Description | |----------|---------|-------------| | `GEN_VERSION_DEFAULT` | `v1` | Default version when not specified | | `V2_JUDGE_MODEL` | `Kimi-K2.6` | Model used for V2 sentence generation | | `V2_FAN_OUT_MODELS` | `Kimi-K2.6,Qwen3.5-397b,GPT-OSS-120b` | Models for multi-model fan-out (when explicitly configured) | --- ## Google Sheet Columns | Column | Description | |--------|-------------| | Batch Date | When the run happened | | Rank | Score rank within this batch | | Job Title / Company / Location | Job details | | Platform | LinkedIn / Indeed / Glassdoor | | Relevance Score | AI assessment (1–10) | | ATS Before (%) | ATS score on original resume | | ATS After (%) | ATS score on tailored resume | | ATS Improvement | After βˆ’ Before | | Resume Quality | Structural quality score | | Priority | High / Medium / Low | | Matching / Missing Skills | Gap analysis | | AI Recommendation | LLM reasoning | | Apply Link | Direct job URL (clickable) | | Resume Link | Google Drive link to tailored resume | | Application Status | Dropdown: Not Applied β†’ Offer | --- ## Environment Variables (`.env`) ``` NVIDIA_API_KEY=nvapi-... # GLM 5.1 + primary key NVIDIA_API_KEY_2=nvapi-... # DeepSeek-v4-Pro, MiniMax NVIDIA_API_KEY_3=nvapi-... # Kimi-K2.6 NVIDIA_API_KEY_4=nvapi-... # Qwen3.5-122b NVIDIA_API_KEY_5=nvapi-... # GPT-OSS-120b NVIDIA_API_KEY_6=nvapi-... # DeepSeek-v4-Flash NVIDIA_API_KEY_7=nvapi-... # Qwen3.5-397b, Qwen3.5-122b-v2 NVIDIA_API_KEY_8=nvapi-... # Step-3.7-Flash GOOGLE_SHEET_ID=1Ehxt3eo... # Your Google Sheet ID RESUME_PATH=data/resume/resume.pdf ``` --- ## Logging & Debugging Every pipeline run writes a timestamped log to `data/logs/run_YYYY-MM-DD_HH-MM-SS.log`. To diagnose failures: 1. Run a search from the UI 2. Switch to the **πŸ“‹ Logs** tab 3. Errors show in red, warnings in yellow 4. Use **Download Full Log File** to share or inspect offline 5. Previous runs are also listed in the selector The log captures: - Every scrape attempt (role, location, raw result count) - Full Python tracebacks on any exception - All `print()` output from scrapers and LLM clients - Playwright browser output --- ## Known Issues / Pending | Issue | Status | Notes | |-------|--------|-------| | Google Drive upload `'Client' object has no attribute 'auth'` | Pending fix | gspread auth method mismatch | | LLM resumes = 0 (GLM timeout during customization) | Pending fix | Switch to Kimi/Step for resume generation | | Naukri blocked by Akamai | Permanent skip | Returns 406 / "Access Denied" with Playwright | | Google OAuth "Access blocked" | Fixed | Add email as test user at GCP console | --- ## Running the UI ```powershell streamlit run ui.py # Opens at http://localhost:8501 # Tabs: # 1. Search β€” configure and run the full pipeline # 2. Results β€” view all assessed jobs with scores # 3. Job Details β€” expand any job for full AI breakdown # 4. Deep Research β€” Odysseus engine to research companies ```