README.md

---
title: JAA ATS Tool
emoji: 🤖
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
license: mit
---

# Job Automation Agent — PM Edition

Automated Product Manager job search, AI-powered assessment, ATS-optimized resume generation, and Google Sheets reporting — all in one pipeline.

---

## What It Does

| Step | What Happens |
|------|--------------|
| 1 | Parses your PDF resume (Kimi-K2.6, ~5s) |
| 2 | Scrapes PM-only jobs from **LinkedIn, Indeed, Glassdoor, Remotive, WeWorkRemotely** — ALL selected roles × locations, last N days |
| 3 | Filters non-PM roles at scrape time; dedup by URL + (title, company) + 30-day SQLite history; platform-level total cap |
| 4 | Fetches FULL job descriptions in bulk (parallel HTTP / single browser session) |
| 5 | Assesses ALL jobs using 7 parallel AI models (10-model pool via NVIDIA API) |
| 6 | Generates ATS-optimized resumes for ALL jobs — DOCX **and PDF**, After ≥ Before guaranteed, target 95% |
| 7 | Writes everything to your Google Sheet with direct job links |
| 8 | Saves local Excel report + resumes in `data/output/resumes/YYYY-MM-DD/` |
| 9 | Saves run to history — reload past runs in one click from the History panel |

---

## Quick Start

```powershell
# 1. Install dependencies
pip install -r requirements.txt
playwright install chromium

# 2. Copy and fill in your API keys
copy .env.example .env
# Edit .env with your NVIDIA_API_KEY, GOOGLE_SHEET_ID, etc.

# 3. Place your resume PDF
# Copy your resume to: data/resume/resume.pdf

# 4. Connect Google (one-time browser login)
python connect_google.py

# 5. Run a test (5 jobs)
# In config.py: ASSESSMENT["test_mode"] = True
python main.py

# 6. Run full production mode (all 100+ PM jobs)
# In config.py: ASSESSMENT["test_mode"] = False
python main.py

# 7. Or use the Streamlit UI
streamlit run ui.py
```

---

## ever-jobs Integration (160+ Platforms)

This project integrates the [ever-jobs](https://github.com/ever-jobs/ever-jobs) NestJS service, which provides REST API access to 160+ job board scrapers.

### Prerequisites

- **Docker Desktop** (preferred): [Install Docker Desktop](https://docs.docker.com/desktop/install/windows-install/)
- OR **Node.js 24.x** for npm subprocess fallback

### Setup (one-time)

```bash
# Clone ever-jobs to vendor/ directory (done automatically during setup)
git clone https://github.com/ever-jobs/ever-jobs.git vendor/ever-jobs --depth=1

# Start via Docker (preferred)
cd vendor/ever-jobs && docker compose up -d

# Verify API is running
curl http://localhost:3001/health
# OR from Python:
python -c "from src.ever_jobs_bridge.server import is_running; print(is_running())"
```

### Automatic Startup

The pipeline calls `ensure_running()` automatically before any ever-jobs platforms are scraped. It tries Docker first, falls back to `npm run start` if Docker is unavailable.

### Platform Selection

The UI has a single **"🌐 Job Platforms"** section with three groups. Selecting LinkedIn, Indeed, Glassdoor, Remotive, WeWorkRemotely, or Naukri uses their dedicated high-quality scrapers; all other platforms go through the ever-jobs REST API.

| Group | Count | Description |
|-------|-------|-------------|
| Search Boards | 94 | General job boards. India-relevant defaults pre-selected. |
| ATS Platforms | 37 | Greenhouse, Lever, Workday etc. — companies post jobs here. NOT resume scoring. |
| Company Pages | 39 | Direct career pages (Flipkart, Swiggy, Amazon, Google, etc.) |

**India default platforms:** linkedin, naukri, indeed, glassdoor, google, bdjobs, internshala, bayt, iimjobs, foundit

### ATS Clarification

> **Important:** "ATS" in ever-jobs means Applicant Tracking System **PLATFORMS**
> (tools companies use to post jobs, like Greenhouse or Lever).
> It does **NOT** mean ATS resume scoring.
>
> Our ATS resume scoring (`src/ats_scorer.py`) uses a 70% JD keyword match +
> 30% resume quality hybrid and is **NOT changed** by this integration.

### Performance Notes

| Selection | Expected Time |
|-----------|---------------|
| Default 10 platforms | ~1–3 minutes |
| 30 platforms | ~3–5 minutes |
| 100+ platforms | 5–10 minutes (warning shown in UI) |

Playwright-based scrapers (some company pages) are inherently slower than API-based boards.

### Cross-Platform Deduplication

In addition to URL-based dedup, a **content fingerprint** (SHA-256 of normalized title+company) catches cross-platform duplicates. For example, the same "Product Manager at Google" posting on LinkedIn AND Greenhouse (different URLs) is detected and deduplicated.

---

## Project Structure

```
Job Automation Agent/
├── main.py                    # Main pipeline (6-step orchestrator)
├── config.py                  # All configuration — models, platforms, ATS settings
├── ui.py                      # Streamlit UI (4 tabs: Search, Results, Job Details, Research)
├── connect_google.py          # One-time Google OAuth setup
├── setup_google.py            # Service account alternative
├── requirements.txt
├── .env                       # API keys (never commit)
├── .env.example               # Template for .env
│
├── src/
│   ├── resume_parser.py       # PDF → plain text (pdfplumber)
│   ├── llm_client.py          # GLM 5.1 wrapper (profile extract, resume customize)
│   ├── model_pool.py          # 10-model parallel AI pool (NVIDIA API)
│   ├── job_assessor.py        # Phase 1 keyword scoring + Phase 2 LLM assessment
│   ├── job_history.py         # SQLite dedup store (data/job_history.db)
│   ├── ats_scorer.py          # Hybrid ATS scoring (70% JD match + 30% quality)
│   ├── resume_customizer.py   # LLM-tailored DOCX resume generator
│   ├── gsheets.py             # Google Sheets + Drive upload
│   ├── excel_reporter.py      # Local Excel report
│   ├── odysseus_llm_core.py   # Deep research LLM core
│   │
│   ├── ever_jobs_bridge/       # ever-jobs NestJS REST API adapter
│   │   ├── __init__.py        # Package init
│   │   ├── server.py          # Docker/npm server lifecycle (start/stop/health)
│   │   ├── client.py          # HTTP client for POST /api/jobs/search
│   │   ├── mapper.py          # IJob JSON → Job dataclass mapper
│   │   └── platforms.py       # 170 platform catalog with group metadata
│   │
│   ├── scrapers/
│   │   ├── base.py            # Job dataclass + BaseScraper + is_pm_role() filter
│   │   ├── linkedin.py        # LinkedIn scraper (requests + BeautifulSoup)
│   │   ├── indeed.py          # Indeed scraper (Playwright for JS rendering)
│   │   ├── glassdoor.py       # Glassdoor scraper (Playwright)
│   │   ├── naukri.py          # Naukri (disabled — blocked by Akamai)
│   │   └── ever_jobs.py       # EverJobsScraper (REST adapter for 160+ platforms)
│   │
│   └── research/
│       ├── deep_researcher.py # Odysseus IterResearch engine (Think→Search→Extract→Synthesize)
│       └── search.py          # DuckDuckGo + Bing fallback, 12h cache
│
└── data/
    ├── resume/resume.pdf      # Your resume (add this)
    ├── job_history.db         # Dedup SQLite DB (auto-created)
    ├── research_cache/        # 12h DuckDuckGo result cache
    └── output/
        ├── resumes/           # Generated DOCX resumes (Company_JobTitle.docx)
        └── reports/           # Excel reports
```

---

## Configuration (`config.py`)

### AI Models (10-model pool via NVIDIA API)

| Model | Speed | Phase 2 | Notes |
|-------|-------|---------|-------|
| Kimi-K2.6 | ~5s/batch | ✅ | Fastest, handles most work |
| Step-3.7-Flash | ~8-35s | ✅ | |
| Qwen3.5-397b | ~9s | ✅ | |
| Qwen3.5-122b-v2 | ~12s | ✅ | |
| GPT-OSS-120b | ~11s | ✅ | |
| Qwen3.5-122b | ~40s | ✅ | |
| DeepSeek-v4-Pro | ~42s | ✅ | |
| DeepSeek-v4-Flash | ~229s | ❌ | Too slow for phase 2 |
| GLM-5.1 | ~234s | ❌ | Used for resume parsing only |
| MiniMax-M2.7 | ~908s | ❌ | Blocked/rate-limited |

### Key Settings

```python
ASSESSMENT = {
    "min_score_for_llm_resume": 6,   # LLM-tailored resume for score >= this
    "generate_all_resumes":     True, # Template resume for ALL PM jobs
    "max_llm_resumes":          30,   # Max LLM resumes per run
    "dedup_days":               30,   # Skip jobs seen in last 30 days
    "test_mode":                True, # ← Set False for full production run
    "test_jobs_limit":          5,    # Max jobs in test mode
}
```

---

## ATS Scoring Method

Hybrid scoring: **70% JD Match + 30% Resume Quality**

- **JD Match (70%)**: Extract keywords FROM the specific job description → match against resume using word-boundary regex (`(?<!\w)kw(?!\w)`) — same approach as Resume-Matcher
- **Resume Quality (30%)**: 6-factor structural analysis (measurable achievements, contact info, education, etc.)
- **Iterative optimization**: Up to 3 LLM attempts to push score up, gap report fed back each iteration
- **Fallback**: `_inject_missing_keywords()` weaves still-missing terms naturally

### Keyword extraction is JD-driven, skills-only, and honest (industry standard)

- Keywords are extracted **from each JD itself** (not a fixed stored list), but
  **skills-only**: a term is kept only if it's a recognised skill/tool/method/
  domain/soft-skill in our gazetteer. Prose nouns (Goals, Authority, Enterprise,
  Productivity…) can never become "keywords" — exactly how real checkers like
  Jobalytics work (they match against a curated skills list, not every word).
- **Dedicated, categorized SKILLS section** (Tools & Analytics / Methodologies /
  Domains / Core Competencies) — the #1 ATS keyword vehicle, placed after the
  summary, parsed and counted by Jobalytics/Resume Worded.
- **Quality over quantity**: cover the JD's real skills (~20–26), each 1–3×,
  across the skills section + summary + woven bullets. Best practice is 15–25
  keywords at 60–80% coverage — NOT 100 stuffed terms.
- **No cap, distributed smartly.** Every meaningful missing keyword is kept —
  woven into relevant bullets first (the ideal, never-penalised place), then any
  remainder spread across several short sentences (each its own paragraph, ≤10
  items). A single long comma-dump is *deliberately not* produced:
  `_strip_keyword_spam` removes any 15+-separator line before scoring, exactly as
  real ATS checkers and recruiters discount stuffing — so distributing keeps them
  all counted without looking like spam.
- **Buzzwords removed.** Vague abstractions (innovation, solutions, tools,
  leadership, leverage, scalable…) are never injected — real checkers like Resume
  Worded penalise them.
- **Honest expectation:** 90%+ is reached on JDs that genuinely fit the
  candidate's background. Out-of-domain JDs score honestly lower — that reflects
  reality (and matches third-party checkers), rather than a faked number.
- The displayed score is a **conservative estimate** (leans low vs our raw
  internal coverage); always verify the final resume on Jobalytics/Simplify.

**Benchmark results** (correctly differentiates — full resume + capable LLM):
- In-domain PM JDs (Airtel, generic PM) → ~83–87
- Out-of-domain JDs (SecOps, lending) → ~60–70, honestly

---

## Google Sheet Columns

| Column | Description |
|--------|-------------|
| Batch Date | When the run happened |
| Rank | Score rank within this batch |
| Job Title / Company / Location | Job details |
| Platform | LinkedIn / Indeed / Glassdoor |
| Relevance Score | AI assessment (1–10) |
| ATS Before (%) | ATS score on original resume |
| ATS After (%) | ATS score on tailored resume |
| ATS Improvement | After − Before |
| Resume Quality | Structural quality score |
| Priority | High / Medium / Low |
| Matching / Missing Skills | Gap analysis |
| AI Recommendation | LLM reasoning |
| Apply Link | Direct job URL (clickable) |
| Resume Link | Google Drive link to tailored resume |
| Application Status | Dropdown: Not Applied → Offer |

---

## Environment Variables (`.env`)

```
NVIDIA_API_KEY=nvapi-...          # GLM 5.1 + primary key
NVIDIA_API_KEY_2=nvapi-...        # DeepSeek-v4-Pro, MiniMax
NVIDIA_API_KEY_3=nvapi-...        # Kimi-K2.6
NVIDIA_API_KEY_4=nvapi-...        # Qwen3.5-122b
NVIDIA_API_KEY_5=nvapi-...        # GPT-OSS-120b
NVIDIA_API_KEY_6=nvapi-...        # DeepSeek-v4-Flash
NVIDIA_API_KEY_7=nvapi-...        # Qwen3.5-397b, Qwen3.5-122b-v2
NVIDIA_API_KEY_8=nvapi-...        # Step-3.7-Flash
GOOGLE_SHEET_ID=1Ehxt3eo...       # Your Google Sheet ID
RESUME_PATH=data/resume/resume.pdf
```

---

## Logging & Debugging

Every pipeline run writes a timestamped log to `data/logs/run_YYYY-MM-DD_HH-MM-SS.log`.

To diagnose failures:
1. Run a search from the UI
2. Switch to the **📋 Logs** tab
3. Errors show in red, warnings in yellow
4. Use **Download Full Log File** to share or inspect offline
5. Previous runs are also listed in the selector

The log captures:
- Every scrape attempt (role, location, raw result count)
- Full Python tracebacks on any exception
- All `print()` output from scrapers and LLM clients
- Playwright browser output

---

## Known Issues / Pending

| Issue | Status | Notes |
|-------|--------|-------|
| Google Drive upload `'Client' object has no attribute 'auth'` | Pending fix | gspread auth method mismatch |
| LLM resumes = 0 (GLM timeout during customization) | Pending fix | Switch to Kimi/Step for resume generation |
| Naukri blocked by Akamai | Permanent skip | Returns 406 / "Access Denied" with Playwright |
| Google OAuth "Access blocked" | Fixed | Add email as test user at GCP console |

---

## Running the UI

```powershell
streamlit run ui.py
# Opens at http://localhost:8501

# Tabs:
# 1. Search     — configure and run the full pipeline
# 2. Results    — view all assessed jobs with scores
# 3. Job Details — expand any job for full AI breakdown
# 4. Deep Research — Odysseus engine to research companies
```