Production readiness: port 7860, Dockerfile, HF Spaces config

.env.example

@@ -0,0 +1,19 @@
+# TasteEngine — Environment Configuration
+# Copy to .env and adjust values
+
+# Flask secret key (generate with: python3 -c "import os; print(os.urandom(24).hex())")
+FLASK_SECRET_KEY=changeme_to_a_random_hex_string
+
+# Set to "1" to enable Flask debug mode (dev only!)
+FLASK_DEBUG=0
+
+# Port for the Flask dev server (ignored when using Gunicorn)
+PORT=5000
+
+# Set to "1" to pre-warm SVD and Slope One caches on startup.
+# First user won't pay the 10s training penalty, but startup takes ~15s longer.
+PREWARM_CACHE=0
+
+# OpenBLAS thread control (leave at 1 for Gunicorn multi-worker setups)
+OPENBLAS_NUM_THREADS=1
+OMP_NUM_THREADS=1

Dockerfile

@@ -0,0 +1,29 @@
+# Use multi-stage build for smaller image
+FROM python:3.11-slim AS builder
+
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+FROM python:3.11-slim
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
+COPY . .
+
+ENV OPENBLAS_NUM_THREADS=1
+ENV OMP_NUM_THREADS=1
+ENV PYTHONUNBUFFERED=1
+
+EXPOSE 7860
+
+CMD ["gunicorn", "app:app", \
+     "--workers", "2", \
+     "--timeout", "120", \
+     "--bind", "0.0.0.0:7860", \
+     "--access-logfile", "-", \
+     "--error-logfile", "-"]

README.md

@@ -1,93 +1,30 @@
-# TasteEngine
+---
+title: TasteEngine
+emoji: 🎯
+colorFrom: indigo
+colorTo: pink
+sdk: docker
+app_port: 7860
+pinned: false
+license: mit
+---
 
-A multi-approach recommender system for e-commerce platforms that generates personalized product recommendations using collaborative filtering, content-based filtering, and knowledge-based techniques, with comprehensive evaluation and comparison.
+# TasteEngine — Multi-Approach Recommender System
 
-## Features
-
-- **Collaborative Filtering** — 5 methods: User-Based, Item-Based, SVD (Matrix Factorization), KNN-Based, and Slope One
-- **Content-Based Filtering** — TF-IDF similarity and feature matching based on user preferences
-- **Knowledge-Based Filtering** — Constraint-based, rule-based, and utility-based recommendation strategies
-- **Evaluation Suite** — RMSE, MAE, Precision@k, Recall@k, F1@k, and Coverage metrics
-- **Explanation Engine** — Human-readable explanations for every recommendation
-- **Interactive Web UI** — Built with Flask, allowing users to select approaches, compare results, and explore products
-- **Synthetic Dataset** — Generates realistic e-commerce data (products, users, ratings)
-
-## Project Structure
-
-```
-TasteEngine/
-├── app.py                      # Flask web application
-├── requirements.txt            # Python dependencies
-├── data/
-│   ├── generate_data.py        # Synthetic dataset generator
-│   ├── products.csv            # Product catalog
-│   ├── users.csv               # User profiles
-│   └── ratings.csv             # User-product ratings
-├── recommender/
-│   ├── collaborative.py        # Collaborative filtering implementations
-│   ├── content_based.py        # Content-based recommendation engine
-│   ├── knowledge_based.py      # Knowledge-based recommendation engine
-│   ├── evaluation.py           # Evaluation metrics and comparison
-│   └── explainer.py            # Recommendation explanation engine
-├── utils/
-│   ├── helpers.py              # Data loading and preprocessing utilities
-│   └── similarity.py           # Similarity metrics (cosine, pearson, adjusted cosine, jaccard)
-├── templates/
-│   ├── index.html              # Landing page
-│   ├── recommend.html          # Recommendation interface
-│   └── evaluation.html         # Evaluation dashboard
-├── static/
-│   └── css/
-│       └── style.css           # Application styles
-└── plan/                       # Design documents
-    ├── 01-overview.md
-    ├── 02-dataset.md
-    ├── 03-collaborative-filtering.md
-    ├── 04-content-based.md
-    ├── 05-knowledge-based.md
-    ├── 06-evaluation.md
-    ├── 07-explanation.md
-    ├── 08-ui-ux.md
-    └── 09-implementation-phases.md
-```
-
-## Installation
-
-```bash
-git clone https://github.com/Abdallah4Z/TasteEngine.git
-cd TasteEngine
-pip install -r requirements.txt
-```
-
-## Usage
+An interactive recommender system comparing **Collaborative Filtering**, **Content-Based**, and **Knowledge-Based** approaches side by side.
 
-```bash
-python app.py
-```
-
-Then open `http://localhost:5000` in your browser.
-
-### Generating Fresh Data
-
-```bash
-python data/generate_data.py
-```
-
-## Recommendation Approaches
-
-| Approach | Methods | Description |
-|---|---|---|
-| Collaborative Filtering | User-Based, Item-Based, SVD, KNN, Slope One | Leverages user-item interaction patterns |
-| Content-Based | TF-IDF, Feature Matching | Uses product attributes and user preferences |
-| Knowledge-Based | Constraint, Rule, Utility | Applies domain rules and constraints |
-
-## Dependencies
+## Features
 
-- Flask 3.0
-- pandas 2.1
-- numpy 1.26
-- scikit-learn 1.3
+- **5 Collaborative Filtering methods**: User-Based, Item-Based, SVD, KNN, Slope One
+- **2 Content-Based methods**: TF-IDF Similarity, Feature Matching
+- **3 Knowledge-Based methods**: Constraint, Rule, Utility
+- **Evaluation Dashboard**: Compare all methods across 6 metrics (RMSE, MAE, Precision@k, Recall@k, F1@k, Coverage)
+- **Interactive UI**: Built with Alpine.js, HTMX, and Chart.js
+- **Incremental Evaluation**: Results appear as each method finishes
+- **Dark/Light Mode**: Toggleable theme
 
-## License
+## Tech Stack
 
-MIT
+- **Backend**: Python, Flask, NumPy, Pandas, scikit-learn
+- **Frontend**: Alpine.js, HTMX, Chart.js, Lucide icons
+- **Deployment**: Docker on Hugging Face Spaces

app.py

@@ -1,7 +1,13 @@
 import os
 import sys
+import logging
+import traceback
+
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 
+os.environ.setdefault("OPENBLAS_NUM_THREADS", "1")
+os.environ.setdefault("OMP_NUM_THREADS", "1")
+
 import json as json_module
 import numpy as np
 import pandas as pd
@@ -13,7 +19,18 @@ from recommender.knowledge_based import KnowledgeBasedRecommender
 from recommender.evaluation import Evaluator
 from recommender.explainer import Explainer
 
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s [%(levelname)s] %(message)s",
+    datefmt="%Y-%m-%d %H:%M:%S",
+)
+logger = logging.getLogger(__name__)
+
 app = Flask(__name__)
+app.secret_key = os.environ.get(
+    "FLASK_SECRET_KEY",
+    os.urandom(24).hex()
+)
 
 products, users, ratings = load_data()
 cf = CollaborativeFiltering(ratings)
@@ -73,6 +90,17 @@ APPROACHES = {
     },
 }
 
+if os.environ.get("PREWARM_CACHE", "0") == "1":
+    import threading
+    logger.info("Pre-warming SVD and Slope One caches...")
+    threading.Thread(
+        target=lambda: (
+            list(cf_train.train_svd_generator()),
+            list(cf_train.compute_slope_one_dev_generator()),
+        ),
+        daemon=True,
+    ).start()
+
 
 def get_product_info(product_id):
     row = products[products["product_id"] == product_id]
@@ -91,6 +119,11 @@ def get_product_info(product_id):
     }
 
 
+@app.route("/health")
+def health():
+    return jsonify({"status": "ok", "users": len(USER_OPTIONS), "products": len(products)})
+
+
 @app.route("/")
 def index():
     return render_template("index.html",
@@ -212,7 +245,8 @@ def api_recommend():
         return jsonify({"recommendations": explanations})
 
     except Exception as e:
-        return jsonify({"error": str(e)}), 500
+        logger.error("Recommendation error: %s\n%s", e, traceback.format_exc())
+        return jsonify({"error": "An internal error occurred while generating recommendations"}), 500
 
 
 CF_METHOD_NAMES = ["user_based", "item_based", "svd", "knn", "slope_one"]
@@ -232,7 +266,8 @@ def api_evaluate_cf(method):
         result = evaluator.evaluate_cf_method(method, cf_train, TEST, k=5)
         return jsonify(result)
     except Exception as e:
-        return jsonify({"method": method, "error": str(e)})
+        logger.error("CF evaluation error [%s]: %s\n%s", method, e, traceback.format_exc())
+        return jsonify({"method": method, "error": "Evaluation failed"})
 
 
 @app.route("/api/evaluate/cf/<method>/stream")
@@ -258,7 +293,8 @@ def api_evaluate_cf_stream(method):
             result = evaluator.evaluate_cf_method(method, cf_train, TEST, k=5)
             yield json_module.dumps({"type": "result", "data": result}) + "\n"
         except Exception as e:
-            yield json_module.dumps({"type": "error", "message": str(e)}) + "\n"
+            logger.error("Stream evaluation error [%s]: %s\n%s", method, e, traceback.format_exc())
+            yield json_module.dumps({"type": "error", "message": "Evaluation failed"}) + "\n"
 
     return Response(stream_with_context(generate()), mimetype="application/x-ndjson")
 
@@ -329,7 +365,8 @@ def api_evaluate_approaches():
         best = max(results, key=lambda a: a.get("Precision@5", 0))["approach"] if results else None
         return jsonify({"approaches": results, "best_approach": best})
     except Exception as e:
-        return jsonify({"error": str(e)}), 500
+        logger.error("Approach evaluation error: %s\n%s", e, traceback.format_exc())
+        return jsonify({"error": "Approach comparison failed"}), 500
 
 
 @app.route("/api/products/filter")
@@ -408,6 +445,7 @@ def api_create_user():
         "budget_min": float(data.get("budget_min", 0)),
         "budget_max": float(data.get("budget_max", 500)),
     })
+    logger.info("Created user %s (ID %d)", new_name, new_id)
     return jsonify({"success": True, "user_id": new_id, "name": new_name})
 
 
@@ -441,7 +479,8 @@ def htmx_recommend():
         else:
             return f'<div class="empty-state"><div class="empty-icon">❌</div><p>Unknown approach: {approach}</p></div>'
     except Exception as e:
-        return f'<div class="empty-state"><div class="empty-icon">❌</div><p>{str(e)}</p></div>'
+        logger.error("HTMX recommend error: %s\n%s", e, traceback.format_exc())
+        return f'<div class="empty-state"><div class="empty-icon">❌</div><p>An error occurred</p></div>'
 
     if not recs:
         return '<div class="empty-state"><div class="empty-icon">📭</div><p>No recommendations found.</p></div>'
@@ -481,4 +520,7 @@ def stars_html(rating):
 
 
 if __name__ == "__main__":
-    app.run(debug=True, host="0.0.0.0", port=5000)
+    debug_mode = os.environ.get("FLASK_DEBUG", "0") == "1"
+    port = int(os.environ.get("PORT", 7860))
+    logger.info("Starting TasteEngine on port %d (debug=%s)", port, debug_mode)
+    app.run(debug=debug_mode, host="0.0.0.0", port=port)

deploy/.dockerignore

@@ -0,0 +1,8 @@
+__pycache__/
+*.pyc
+.env
+.git/
+.gitignore
+discussion/
+plan/
+*.md

deploy/Dockerfile

@@ -0,0 +1,33 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+# Install system dependencies (for numpy/scikit-learn)
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+
+# Copy and install Python dependencies
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy application code
+COPY . .
+
+# Environment variables
+ENV OPENBLAS_NUM_THREADS=1
+ENV OMP_NUM_THREADS=1
+ENV PREWARM_CACHE=1
+ENV PYTHONUNBUFFERED=1
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
+    CMD python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:5000/health')" || exit 1
+
+# Run with Gunicorn
+CMD ["gunicorn", "app:app", \
+     "--workers", "4", \
+     "--timeout", "120", \
+     "--bind", "0.0.0.0:5000", \
+     "--access-logfile", "-", \
+     "--error-logfile", "-"]

deploy/nginx.conf

@@ -0,0 +1,44 @@
+server {
+    listen 80;
+    server_name tasteengine.example.com;
+
+    # Static files — served directly by nginx, Flask never touched
+    location /static/ {
+        alias /home/abdallah/recommender-project/TasteEngine/static/;
+        expires 7d;
+        add_header Cache-Control "public, immutable";
+    }
+
+    # Application proxy
+    location / {
+        proxy_pass http://127.0.0.1:5000;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        # CRITICAL: disable buffering for SSE/streaming endpoints (SVD evaluation)
+        proxy_buffering off;
+        proxy_cache off;
+
+        # Timeouts
+        proxy_connect_timeout 60s;
+        proxy_read_timeout 180s;
+        proxy_send_timeout 60s;
+    }
+
+    # Optional: rate limiting on recommendation API
+    location /api/recommend {
+        limit_req zone=recommend burst=5 nodelay;
+        proxy_pass http://127.0.0.1:5000;
+    }
+
+    # SSL (uncomment after running certbot)
+    # listen 443 ssl;
+    # ssl_certificate /etc/letsencrypt/live/tasteengine.example.com/fullchain.pem;
+    # ssl_certificate_key /etc/letsencrypt/live/tasteengine.example.com/privkey.pem;
+}
+
+# Rate limiting zone definition
+limit_req_zone $binary_remote_addr zone=recommend:10m rate=10r/s;

deploy/tasteengine.service

@@ -0,0 +1,27 @@
+[Unit]
+Description=TasteEngine Recommender System
+After=network.target
+
+[Service]
+Type=simple
+User=abdallah
+WorkingDirectory=/home/abdallah/recommender-project/TasteEngine
+
+Environment=OPENBLAS_NUM_THREADS=1
+Environment=OMP_NUM_THREADS=1
+Environment=PREWARM_CACHE=1
+EnvironmentFile=/home/abdallah/recommender-project/TasteEngine/.env
+
+ExecStart=/home/abdallah/.local/bin/gunicorn app:app \
+    --workers 4 \
+    --timeout 120 \
+    --bind 0.0.0.0:5000 \
+    --access-logfile /var/log/tasteengine/access.log \
+    --error-logfile /var/log/tasteengine/error.log \
+    --log-level info
+
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target

discussion/01-collaborative-filtering.md

@@ -0,0 +1,172 @@
+# Collaborative Filtering — `recommender/collaborative.py`
+
+## Overview
+
+Collaborative Filtering (CF) is the most widely used recommendation approach. It makes predictions based on the collective behavior of users — "users who liked similar items to you also liked these." This implementation provides **5 distinct methods** in a single `CollaborativeFiltering` class.
+
+**Class:** `CollaborativeFiltering`  
+**File:** `recommender/collaborative.py` (248 lines)  
+**Dependencies:** `numpy`, `pandas`, `utils.similarity`, `utils.helpers`, `sklearn.neighbors`
+
+---
+
+## Architecture
+
+### Initialization (`__init__`)
+
+```python
+def __init__(self, ratings_df):
+```
+
+Takes a ratings DataFrame with columns `[user_id, product_id, rating]` and:
+
+1. **Builds a user-item matrix** via `build_user_item_matrix()` — a pivot table where rows = users, columns = items, values = ratings. Missing ratings are `NaN`.
+2. **Extracts metadata:** `n_users`, `n_items`, `user_ids`, `item_ids`, `user_means`, `global_mean`
+3. **Initializes caches:** `_svd_cache = None`, `_slope_one_dev = None` — lazy-computed on first method call, reused thereafter
+
+### Helper Methods
+
+| Method | Purpose |
+|---|---|
+| `_get_user_index(user_id)` | Converts user ID to matrix row index |
+| `_get_item_index(item_id)` | Converts item ID to matrix column index |
+
+---
+
+## The 5 Methods
+
+### 1. User-Based CF (`user_based_cf`)
+
+**Idea:** Find users similar to the target user, aggregate their ratings for unseen items.
+
+**Algorithm:**
+1. Fill missing ratings with the global mean
+2. Compute a full `n_users × n_users` cosine similarity matrix
+3. For each item the target user hasn't rated:
+   - Find the top `k=20` most similar users who DID rate that item
+   - Predict rating = weighted average of their ratings (weighted by similarity)
+
+**Complexity:** O(n_users² × n_items) — scales poorly with users, fast on this dataset (200 users).
+
+**Strengths:** Intuitive, works well with dense data, captures nuanced user preferences.  
+**Weaknesses:** Expensive with many users, performs poorly on sparse data (cold-start users with few ratings).
+
+---
+
+### 2. Item-Based CF (`item_based_cf`)
+
+**Idea:** Find items similar to those the user already liked, recommend them.
+
+**Algorithm:**
+1. Compute an `n_items × n_items` **adjusted cosine similarity** matrix — this normalizes by subtracting each user's mean rating before computing cosine, removing user bias (some users rate everything high/low)
+2. For each unseen item, find the top `k=15` most similar items from the user's rated set
+3. Predict rating = weighted average of the user's ratings on those similar items
+
+**Complexity:** O(n_items² × n_users) — precomputes item-item similarity once, then per-user prediction is O(n_items × k).
+
+**Strengths:** Fast per-user prediction after precomputation, stable results (item similarities change less than user similarities), handles larger user bases better than User-Based.  
+**Weaknesses:** Item-item matrix is O(n_items²) in memory, can miss cross-category recommendations.
+
+---
+
+### 3. SVD (Matrix Factorization) — `svd`
+
+**Idea:** Factorize the user-item matrix into two low-rank matrices (user factors × item factors) that capture latent features like "action vs drama" or "budget vs premium."
+
+**Training (cached via `train_svd_generator`):**
+1. Initialize random factor matrices: `P (n_users × 20)`, `Q (n_items × 20)`, plus bias terms `bu`, `bi`
+2. For each of **100 epochs**, iterate over all observed ratings and update:
+   ```
+   prediction = global_mean + bu[u] + bi[i] + dot(P[u], Q[i])
+   error = actual_rating - prediction
+   bu[u] += learning_rate * (error - regularization * bu[u])
+   bi[i] += learning_rate * (error - regularization * bi[i])
+   P[u] += learning_rate * (error * Q[i] - regularization * P[u])
+   Q[i] += learning_rate * (error * P[u] - regularization * Q[i])
+   ```
+3. Cache `(P, Q, bu, bi)` — first call takes ~10s, subsequent calls are instant
+
+**Prediction:**
+```python
+pred = global_mean + bu[u_idx] + bi[i_idx] + dot(P[u_idx], Q[i_idx])
+```
+
+**Hyperparameters:**
+| Param | Value | Effect |
+|---|---|---|
+| `n_factors` | 20 | More factors = more expressive, risk of overfitting |
+| `n_epochs` | 100 | More epochs = better convergence |
+| `lr` | 0.01 | Learning rate — higher = faster but may overshoot |
+| `reg` | 0.02 | Regularization — higher = prevents overfitting |
+
+**Strengths:** Handles sparsity well, discovers latent relationships, state-of-the-art baseline for many recommendation tasks.  
+**Weaknesses:** Slow to train (100 epochs), less interpretable than memory-based methods, requires careful hyperparameter tuning.
+
+---
+
+### 4. KNN-Based CF (`knn_cf`)
+
+**Idea:** Use sklearn's `NearestNeighbors` to efficiently find the closest users, then aggregate their ratings.
+
+**Algorithm:**
+1. Fill missing ratings with global mean
+2. Fit a `NearestNeighbors(k=10, metric="cosine")` model on the filled matrix
+3. For the target user, find `k` nearest neighbors
+4. For each unseen item, predict = weighted average of neighbor ratings (weights = inverse distances)
+
+**Complexity:** KNN indexing is O(n_users × log(n_users)), per-user prediction is O(k × n_items).
+
+**Strengths:** Very fast nearest-neighbor search via sklearn's optimized KD-tree/ball-tree, robust to outliers.  
+**Weaknesses:** Sensitive to `k` choice, requires the full matrix in memory.
+
+---
+
+### 5. Slope One (`slope_one`)
+
+**Idea:** Predict ratings based on the average difference ("deviation") between items. "If users rate item A 0.5 higher than item B on average, and you rate item B 4.0, you'll likely rate item A 4.5."
+
+**Precomputation (cached via `compute_slope_one_dev_generator`):**
+For every pair of items `(i, j)`:
+```
+dev[i][j] = average(rating_i - rating_j) across all users who rated both
+cnt[i][j] = number of users who rated both
+```
+
+**Prediction:**
+For the target user and each unseen item `i_idx`:
+```
+numerator = sum of (user_rating[j_idx] + dev[i_idx][j_idx]) for all rated items j_idx with cnt > 0
+denominator = number of such rated items
+pred = numerator / denominator
+```
+
+**Strengths:** Simple, fast prediction (O(n_items × n_rated) per user), no hyperparameters, easy to explain, can handle new ratings incrementally.  
+**Weaknesses:** Precomputation is O(n_items² × n_users), memory is O(n_items²), doesn't handle item cold-start.
+
+---
+
+## Routing (`recommend` method)
+
+```python
+def recommend(self, method, user_id, n_recommendations=10, **kwargs):
+```
+
+Maps method name strings to the corresponding function:
+- `"user_based"` → `user_based_cf`
+- `"item_based"` → `item_based_cf`
+- `"svd"` → `svd`
+- `"knn"` → `knn_cf`
+- `"slope_one"` → `slope_one`
+
+All methods return `list[tuple(product_id, predicted_rating)]` sorted by rating descending.
+
+---
+
+## Caching Strategy
+
+| Cache | Trigger | Benefit |
+|---|---|---|
+| `_svd_cache` | First `svd()` call | Avoids re-training 100 epochs per user |
+| `_slope_one_dev` | First `slope_one()` call | Avoids O(n_items² × n_users) recomputation per user |
+
+Both caches use **generator functions** (`train_svd_generator`, `compute_slope_one_dev_generator`) that yield intermediate progress — used by the streaming API to show epoch-level progress in the UI.

discussion/02-content-based.md

@@ -0,0 +1,102 @@
+# Content-Based Filtering — `recommender/content_based.py`
+
+## Overview
+
+Content-Based Filtering recommends items by comparing their **features** (category, brand, name, subcategory) against a user's profile. It doesn't need other users' data — only the target user's history and item attributes.
+
+**Class:** `ContentBasedRecommender`  
+**File:** `recommender/content_based.py` (76 lines)  
+**Dependencies:** `numpy`, `sklearn.feature_extraction.text.TfidfVectorizer`, `sklearn.metrics.pairwise.cosine_similarity`
+
+---
+
+## Architecture
+
+### Initialization (`__init__`)
+
+```python
+def __init__(self, products_df):
+```
+
+1. Copies the products DataFrame
+2. Builds a **text_features** column by concatenating: `category + subcategory + brand + name`
+3. Fits a **TF-IDF vectorizer** (with English stop words removed) on all product text features
+4. Transforms all products into a **TF-IDF matrix** of shape `(n_products × n_features)`
+5. Stores `product_ids` array for index-to-ID mapping
+
+**TF-IDF** (Term Frequency — Inverse Document Frequency) gives higher weight to words that are distinctive to a product ("iPhone" is more important than "device") and lower weight to common words.
+
+---
+
+## The 2 Methods
+
+### 1. TF-IDF Similarity (`tfidf_recommend`)
+
+**Idea:** Build a user profile from items they rated highly, then find similar items by TF-IDF cosine similarity.
+
+**Algorithm:**
+1. Take `user_profile_items` — product IDs the user rated ≥ 3.5
+2. Look up each product's index in the products DataFrame
+3. Average their TF-IDF vectors → **user profile vector** (a single vector representing the user's taste)
+4. Compute cosine similarity between the profile vector and ALL product vectors
+5. Exclude already-rated items, sort by similarity, return top N
+
+**Equation:**
+```
+user_profile = mean(tfidf[item] for item in user_rated_items)
+score[product] = cosine_similarity(user_profile, tfidf[product])
+```
+
+**Complexity:** O(n_products × n_features) — linear in the product catalog size.
+
+**Strengths:** No need for other users' data, works for niche/long-tail items, can recommend new items.  
+**Weaknesses:** Requires sufficient rating history to build a profile, tends to overspecialize (keeps recommending the same category), can't capture quality or popularity.
+
+---
+
+### 2. Feature Matching (`feature_match_recommend`)
+
+**Idea:** Score each product based on how well it matches explicit user preferences (category, brand, price), without needing rating history.
+
+**Algorithm:**
+For each product, compute a score from preference matches:
+
+| Criterion | Weight | Condition |
+|---|---|---|
+| Category match | +40 | Product category is in user's preferred categories |
+| Brand match | +30 | Product brand is in user's favorite brands |
+| Budget fit | +20 | Product price is within user's budget range |
+| Subcategory match | +10 | Product subcategory is in user's preferred set |
+
+```
+total_score = category_score + brand_score + budget_score + subcategory_score
+```
+
+**Complexity:** O(n_products) — iterates over every product once.
+
+**Strengths:** Works with **zero rating history** — needs only preference data (cold-start friendly), transparent and explainable, simple rule-based logic.  
+**Weaknesses:** No learning from past behavior, weights are hardcoded (not learned), can't account for items that don't match stated preferences but the user might still like.
+
+---
+
+## Routing (`recommend` method)
+
+```python
+def recommend(self, method, user_profile_items=None, preferences=None, n_recommendations=10):
+```
+
+- `"tfidf"` → `tfidf_recommend(user_profile_items, n_recommendations)`  
+- `"feature_match"` → `feature_match_recommend(preferences, n_recommendations)`
+
+Both return `list[tuple(product_id, score)]` sorted by score descending.
+
+---
+
+## When to Use Each
+
+| Method | Best For | Needs |
+|---|---|---|
+| **TF-IDF** | Users with 5+ rated items | `user_profile_items` (rating history) |
+| **Feature Match** | New users, cold-start | `preferences` (categories, brands, budget) |
+
+The TF-IDF method captures subtle textual similarities between items (e.g., a "Sneakers" product in Clothing and "Running Shoes" in Footwear), while Feature Matching is explicit and rule-based.

discussion/03-knowledge-based.md

@@ -0,0 +1,161 @@
+# Knowledge-Based Filtering — `recommender/knowledge_based.py`
+
+## Overview
+
+Knowledge-Based Filtering makes recommendations using **explicit domain rules** and **user constraints** rather than past behavior. It's the only approach that works for cold-start users with **zero rating history** — it only needs preferences (categories, brands, budget).
+
+**Class:** `KnowledgeBasedRecommender`  
+**File:** `recommender/knowledge_based.py` (117 lines)  
+**Dependencies:** `numpy`, `pandas`
+
+---
+
+## Architecture
+
+### Initialization (`__init__`)
+
+```python
+def __init__(self, products_df):
+```
+
+Stores a copy of the products DataFrame for filtering and scoring.
+
+No training, no fitting — this is a **memory-based, rule-driven approach** that operates directly on product attributes.
+
+---
+
+## The 3 Methods
+
+### 1. Constraint-Based (`constraint_based`)
+
+**Idea:** Filter the product catalog by hard constraints, then sort by rating.
+
+**Algorithm:**
+1. Start with all products
+2. **Sequentially filter** by (in order):
+   - `budget_max` — remove products exceeding max budget
+   - `budget_min` — remove products below min budget
+   - `category` — keep only preferred categories (if specified)
+   - `brand` — if brand preferences exist AND matches found, restrict to those brands only
+   - `min_rating` — keep only products above minimum rating
+   - `subcategory` — keep matching subcategories (if any found)
+3. Sort remaining products by `avg_rating` descending
+4. Return top N
+
+**Key behavior:** Brand and subcategory filters only activate if there's at least one match — otherwise they're skipped (prevents empty results from overly strict constraints).
+
+**Complexity:** O(n_products) — each filter is a pandas operation on the DataFrame.
+
+**Strengths:** Deterministic, transparent, respects hard constraints perfectly.  
+**Weaknesses:** Can return empty results if constraints are too strict, no ranking beyond rating, doesn't consider user preferences for non-filtered attributes.
+
+---
+
+### 2. Rule-Based (`rule_based`)
+
+**Idea:** Apply cross-selling rules and preference scoring — "laptop buyers also buy accessories."
+
+**Built-in Cross-Sell Rules:**
+
+| If user interacted with... | Recommend accessories like... |
+|---|---|
+| `Laptops` | Laptop Bag, Mouse, Keyboard, Cooling Pad |
+| `Smartphones` | Phone Case, Screen Protector, Charger, Power Bank |
+| `Cameras` | Camera Bag, Tripod, Memory Card, Lens Kit |
+| `Headphones` | Headphone Stand, Cable, Carrying Case, Ear Pads |
+
+**Scoring Formula:**
+
+```
+score = 0
+
+if interacted_category matches and product is a related accessory:
+    score += 50         (cross-sell bonus)
+
+if product category in preferred_categories:
+    score += 20         (category preference)
+
+if price within budget range:
+    score += 15         (budget fit)
+
+if brand in favorite_brands:
+    score += 10         (brand loyalty)
+```
+
+**Complexity:** O(n_products) — scores every product once.
+
+**Strengths:** Captures retail domain knowledge (cross-selling), rewards accessory discovery along with category/brand preferences.  
+**Weaknesses:** Rules are hardcoded for Electronics only, doesn't learn from data, `interacted_category` must be provided by the application.
+
+---
+
+### 3. Utility-Based (`utility_based`)
+
+**Idea:** Compute a weighted multi-attribute utility score for every product, measuring how well it satisfies the user's preferences across multiple dimensions.
+
+**Utility Function:**
+
+```
+U(product) = w_price × price_score
+           + w_category × category_score
+           + w_brand × brand_score
+           + w_rating × rating_score
+```
+
+**Default Weights:**
+
+| Attribute | Weight | Rationale |
+|---|---|---|
+| `category` | 0.3 (30%) | Category preference is strongest signal |
+| `rating` | 0.3 (30%) | Product quality equally important |
+| `brand` | 0.2 (20%) | Brand loyalty is meaningful |
+| `price` | 0.2 (20%) | Budget fit is a soft constraint |
+
+**Attribute Score Functions:**
+
+- **Price score:** Normalized distance from midpoint of user's budget range — penalizes items at the edges, rewards the center
+  ```
+  price_score = 1 - |price - budget_mid| / (budget_range / 2)
+  clamped to [0, 1]
+  ```
+
+- **Category score:** 1.0 if in preferred categories, 0.0 otherwise
+
+- **Brand score:** 1.0 if in favorite brands, 0.0 otherwise
+
+- **Rating score:** Normalized by max rating in catalog
+  ```
+  rating_score = avg_rating / max_rating_in_catalog
+  ```
+
+**Complexity:** O(n_products) — computes utility for every product.
+
+**Strengths:** Continuous scoring (not binary filtering), learnable weights (could be optimized with feedback), handles trade-offs between competing preferences.  
+**Weaknesses:** Weights are fixed but could be tuned, equal treatment within attributes (all categories weighted equally).
+
+---
+
+## Routing (`recommend` method)
+
+```python
+def recommend(self, method, constraints=None, context=None, preferences=None, n_recommendations=10):
+```
+
+- `"constraint"` → `constraint_based(constraints, n_recommendations)`  
+- `"rule"` → `rule_based(context, n_recommendations)`  
+- `"utility"` → `utility_based(preferences, n_recommendations=n_recommendations)`
+
+All return `list[tuple(product_id, score)]` sorted by score descending.
+
+---
+
+## Knowledge-Based vs Other Approaches
+
+| Aspect | Knowledge-Based | Collaborative | Content-Based |
+|---|---|---|---|
+| Needs rating history | **No** | Yes | Yes (TF-IDF) |
+| Needs preferences | Yes | No | Yes (Feature Match) |
+| Cold-start capable | **Yes** | No | Partial |
+| Learns from data | No | Yes | Partial |
+| Deterministic | **Yes** | No (SVD stochastic) | Yes |
+| Explainability | High | Medium | High |

discussion/04-evaluation.md

@@ -0,0 +1,172 @@
+# Evaluation Metrics — `recommender/evaluation.py`
+
+## Overview
+
+The Evaluator measures recommendation quality using **6 standard metrics** across 4 dimensions: prediction accuracy (RMSE, MAE) and ranking quality (Precision, Recall, F1, Coverage).
+
+**Class:** `Evaluator`  
+**File:** `recommender/evaluation.py` (161 lines)  
+**Dependencies:** `numpy`, `pandas`, `sklearn.metrics`
+
+---
+
+## Architecture
+
+### Initialization (`__init__`)
+
+```python
+def __init__(self, ratings_df, predictions_df=None):
+```
+
+Stores the full ratings DataFrame and optionally pre-computed predictions. The evaluator needs a **test set** to compare predicted vs actual ratings.
+
+---
+
+## The 6 Metrics
+
+### 1. RMSE — Root Mean Square Error
+
+**What it measures:** Average magnitude of prediction errors (larger errors penalized more due to squaring).
+
+```python
+def rmse(self, y_true, y_pred):
+    return float(np.sqrt(mean_squared_error(y_true, y_pred)))
+```
+
+**Range:** `0.0` (perfect) to `5.0` (worst, on a 1-5 scale)  
+**Interpretation:** A score of 0.95 means predictions are off by ~0.95 stars on average.  
+**Lower is better.**
+
+**Why squared?** Errors of 2.0 count 4× more than errors of 1.0 — this penalizes large outliers.
+
+---
+
+### 2. MAE — Mean Absolute Error
+
+**What it measures:** Average absolute prediction error (treats all errors equally).
+
+```python
+def mae(self, y_true, y_pred):
+    return float(mean_absolute_error(y_true, y_pred))
+```
+
+**Range:** `0.0` to `5.0`  
+**Interpretation:** Same units as ratings (stars). MAE < RMSE means no large outliers.  
+**Lower is better.**
+
+**RMSE vs MAE:** If RMSE is significantly higher than MAE, the model occasionally makes very bad predictions.
+
+---
+
+### 3. Precision@k
+
+**What it measures:** Of the top-k recommendations, what fraction are actually relevant (rated ≥ 3.5 by the user)?
+
+```python
+def precision_at_k(self, recommended_items, relevant_items, k=5):
+    top_k = recommended_items[:k]
+    hits = len(set(top_k) & set(relevant_items))
+    return hits / k if k > 0 else 0
+```
+
+**Range:** `0.0` to `1.0`  
+**Interpretation:** Precision@5 = 0.05 means 0.25 out of 5 recommendations are relevant (1 item in 20).  
+**Higher is better.**
+
+**Why @k?** Users only see the top-k recommendations — Precision@k measures real-world utility better than global precision.
+
+---
+
+### 4. Recall@k
+
+**What it measures:** Of all the items the user would find relevant, what fraction appears in the top-k?
+
+```python
+def recall_at_k(self, recommended_items, relevant_items, k=5):
+    top_k = recommended_items[:k]
+    hits = len(set(top_k) & set(relevant_items))
+    return hits / len(relevant_items) if len(relevant_items) > 0 else 0
+```
+
+**Range:** `0.0` to `1.0`  
+**Interpretation:** Recall@5 = 0.045 means the top-5 captures 4.5% of all relevant items.  
+**Higher is better.**
+
+**Precision vs Recall:** Precision asks "are the recommendations good?" Recall asks "are we finding all the good items?" There's typically a trade-off.
+
+---
+
+### 5. F1@k
+
+**What it measures:** Harmonic mean of Precision and Recall — a single score balancing both.
+
+```python
+def f1_at_k(self, recommended_items, relevant_items, k=5):
+    p = self.precision_at_k(recommended_items, relevant_items, k)
+    r = self.recall_at_k(recommended_items, relevant_items, k)
+    return 2 * p * r / (p + r) if (p + r) > 0 else 0
+```
+
+**Range:** `0.0` to `1.0`  
+**Interpretation:** F1@5 = 0.036 means balanced precision-recall performance.  
+**Higher is better.** F1 is low if either precision or recall is low.
+
+---
+
+### 6. Coverage
+
+**What it measures:** What fraction of the total product catalog does the recommender ever recommend across all users?
+
+```python
+def coverage(self, recommended_items_list, total_items):
+    recommended_set = set()
+    for items in recommended_items_list:
+        recommended_set.update(items)
+    return len(recommended_set) / total_items if total_items > 0 else 0
+```
+
+**Range:** `0.0` to `1.0`  
+**Interpretation:** Coverage = 0.86 means 86% of all products appear in at least one user's recommendations.  
+**Higher is better.** Low coverage means the system only recommends popular items (the "Harry Potter problem").
+
+---
+
+## Evaluation Process
+
+### Per-Method Evaluation (`evaluate_cf_method`)
+
+For each CF method, the evaluator:
+
+1. **Selects up to 20 test users** from the test set
+2. For each user:
+   - Generates 20 recommendations using the CF method
+   - Collects actual ratings from the test set
+   - Compares predicted vs actual for RMSE/MAE
+   - Computes Precision/Recall/F1@k for ranking quality
+3. Aggregates all metrics across users
+
+**Handling missing predictions:** If a product in the test set wasn't predicted, the evaluator imputes a rating of 2.5 (neutral) — this penalizes methods that can't predict all test items.
+
+### Full Comparison (`compare_cf_methods`)
+
+Iterates over all 5 CF methods, calling `evaluate_cf_method` for each. Returns a list of result dicts.
+
+### Approach Comparison (`compare_approaches`)
+
+Compares the 3 approaches (CF, Content, Knowledge) by:
+1. Using the **best** CF method (Item-Based) for the CF column
+2. Using TF-IDF for Content-Based
+3. Using Constraint-Based for Knowledge-Based
+4. Evaluating on the same 20 test users
+
+---
+
+## How Evaluation is Used
+
+The backend exposes two endpoints:
+
+- **`GET /api/evaluate/cf/<method>`** — evaluates a single CF method (returns metrics in ~2-15s)
+- **`GET /api/evaluate/cf/<method>/stream`** — streaming version for SVD and Slope One (shows epoch/item progress)
+- **`GET /api/evaluate/approaches`** — compares all 3 approaches
+
+The frontend calls these incrementally, showing results as each method finishes.

discussion/05-explainer.md

@@ -0,0 +1,97 @@
+# Explanation Generator — `recommender/explainer.py`
+
+## Overview
+
+The Explainer generates **human-readable explanations** for every recommendation. Each of the 10 recommendation methods has its own explanation template, incorporating product and user details for specificity.
+
+**Class:** `Explainer`  
+**File:** `recommender/explainer.py` (99 lines)  
+**Dependencies:** None (pure Python)
+
+---
+
+## Architecture
+
+### Initialization (`__init__`)
+
+```python
+def __init__(self, products_df, users_df):
+```
+
+Builds two in-memory **caches** (dictionaries) for fast lookup:
+- `_product_cache`: `{product_id: pandas_Series}` — full product row (name, category, brand, price, subcategory)
+- `_user_cache`: `{user_id: pandas_Series}` — full user row (name, age, preferred_categories, favorite_brands, budget)
+
+This avoids pandas DataFrame lookups during real-time explanation generation.
+
+### Helper Methods
+
+| Method | Purpose |
+|---|---|
+| `_get_product(product_id)` | Returns product data dict from cache |
+| `_get_user(user_id)` | Returns user data dict from cache |
+| `_pref_list(user, key)` | Safely extracts comma-separated preferences, handles NaN |
+
+The `_pref_list` method guards against `NaN` values from the pandas cache (a common edge case when user fields are empty in the CSV):
+```python
+def _pref_list(self, user, key):
+    raw = user.get(key, "")
+    if raw is None or isinstance(raw, float):
+        return []
+    if isinstance(raw, str):
+        return [p.strip() for p in raw.split(",") if p.strip()][:2]
+    return list(raw)[:2] if raw else []
+```
+
+---
+
+## Explanation Templates by Method
+
+### Collaborative Filtering (5 methods)
+
+| Method | Template | Example Output |
+|---|---|---|
+| `user_based` | `"Users with {category} taste also liked {name}"` | "Users with Electronics taste also liked X-T5" |
+| `item_based` | `"Matches your {category} preferences — similar to items you've rated highly"` | "Matches your Toys preferences — similar to items you've rated highly" |
+| `svd` | `"Fits your profile: {brand}'s {category} — top latent factor match"` | "Fits your profile: Dyson's Home & Kitchen — top latent factor match" |
+| `knn` | `"Popular among peers who also like {category}"` | "Popular among peers who also like Sports" |
+| `slope_one` | `"Frequently chosen by users who liked the same {category} products"` | "Frequently chosen by users who liked the same Toys products" |
+
+### Content-Based (2 methods)
+
+| Method | Template | Example Output |
+|---|---|---|
+| `tfidf` | `"Content matches: {category} / {subcategory} — strongly aligns with your past likes"` | "Content matches: Home & Kitchen / Storage — strongly aligns with your past likes" |
+| `feature_match` | `"Matches your preferences: {category}, {brand}"` | "Matches your preferences: Home & Kitchen, KitchenAid" |
+
+### Knowledge-Based (3 methods)
+
+| Method | Template | Example Output |
+|---|---|---|
+| `constraint` | `"Fits your criteria: ${price} within budget · {category} · {brand}"` | "Fits your criteria: $241 within budget · Books · Penguin" |
+| `rule` | `"{category} buyers commonly add {name} — cross-sell match"` | "Home & Kitchen buyers commonly add Sofa Set — cross-sell match" |
+| `utility` | `"Top utility score: {category} × {brand} matches your preference weights"` | "Top utility score: Books × Macmillan matches your preference weights" |
+
+---
+
+## Routing (`get_explanation`)
+
+```python
+def get_explanation(self, approach, method, user_id, product_id, details=None):
+```
+
+Dispatches to the correct explainer based on approach:
+- `"cf"` → `explain_cf(method, user_id, product_id, details)`
+- `"content"` → `explain_content(method, user_id, product_id, details)`
+- `"knowledge"` → `explain_knowledge(method, user_id, product_id, details)`
+
+Each sub-explainer receives **details** (a dict from the backend with scores and metadata) and uses `_get_product` / `_get_user` to access full attribute data.
+
+---
+
+## Design Principles
+
+1. **Specific over generic** — Every template includes the product's category, brand, or subcategory. No two explanations read the same way.
+2. **No fake precision** — Scores are not shown as "100% similar" in templates. The score bar (frontend) shows relative strength via a progress bar.
+3. **Domain-aware** — Knowledge templates mention budgets, cross-sell rules, and utility scores.
+4. **Safe defaults** — Fallback templates exist for every method branch in case data is missing.

discussion/06-similarity.md

@@ -0,0 +1,103 @@
+# Similarity Metrics — `utils/similarity.py`
+
+## Overview
+
+Provides 4 similarity/distance metrics used by the Collaborative Filtering module. These measure how "close" two users or two items are based on their rating vectors.
+
+**File:** `utils/similarity.py` (30 lines)  
+**Dependencies:** `numpy`, `sklearn.metrics.pairwise`
+
+---
+
+## The 4 Metrics
+
+### 1. Cosine Similarity
+
+**Used by:** User-Based CF (full similarity matrix)
+
+```python
+def cosine_similarity(matrix):
+    return sklearn_cosine(matrix)
+```
+
+**What it measures:** The angle between two vectors, ignoring magnitude. Two users who rate items in the same proportion (one rates everything 4s, the other 5s) are considered similar.
+
+**Equation:**
+```
+cos(A, B) = (A · B) / (|A| × |B|)
+```
+
+**Range:** `-1` to `1` (for rating vectors, typically `0` to `1` since ratings are positive)
+
+**Why not Pearson?** Cosine similarity with raw ratings is simpler and performs comparably when data isn't heavily biased. The Item-Based method uses **adjusted cosine** which subtracts user means first (equivalent to Pearson normalization).
+
+---
+
+### 2. Pearson Similarity
+
+**Defined but not currently used** by any active method.
+
+```python
+def pearson_similarity(matrix):
+    matrix_centered = matrix - np.nanmean(matrix, axis=1, keepdims=True)
+    matrix_centered = np.nan_to_num(matrix_centered, nan=0.0)
+    mask = ~np.isnan(matrix)
+    norm = np.sqrt(np.nansum(matrix_centered ** 2, axis=1, keepdims=True))
+    norm[norm == 0] = 1
+    matrix_norm = matrix_centered / norm
+    return matrix_norm @ matrix_norm.T
+```
+
+**What it measures:** Linear correlation between rating patterns. Accounts for user bias — a user who rates everything 4-5 and another who rates 2-3 might agree on relative preferences even though absolute ratings differ.
+
+**Why unused:** On this dataset, cosine similarity with mean-imputed missing values gives comparable results with less code complexity.
+
+---
+
+### 3. Adjusted Cosine Similarity
+
+**Used by:** Item-Based CF (item-item similarity matrix)
+
+```python
+def adjusted_cosine_similarity(matrix):
+    user_mean = np.nanmean(matrix, axis=1, keepdims=True)
+    matrix_centered = matrix - user_mean
+    matrix_centered = np.nan_to_num(matrix_centered, nan=0.0)
+    return sklearn_cosine(matrix_centered.T)
+```
+
+**What it measures:** Cosine similarity on **mean-centered ratings** (each user's ratings shifted by their average). Then computes similarity between **items** (transposed matrix), not users.
+
+**Why adjusted?** Raw cosine similarity between items is misleading because users have different rating scales. Alice might give 5 to everything she likes, while Bob gives 4. Adjusted cosine removes this bias by centering.
+
+**Implementation note:** The `.T` transposes the user-item matrix to item-user before computing cosine, giving `n_items × n_items` similarity.
+
+---
+
+### 4. Jaccard Similarity
+
+**Defined but not used** by any active method.
+
+```python
+def jaccard_similarity(set1, set2):
+    intersection = len(set1 & set2)
+    union = len(set1 | set2)
+    return intersection / union if union > 0 else 0
+```
+
+**What it measures:** Overlap between two sets — suitable for binary data (purchased/not purchased) rather than rating data.
+
+**Why unused:** Rating data is continuous (1-5), not binary. Jaccard would waste information by discarding rating values.
+
+---
+
+## Metric Selection Guide
+
+| Metric | Used By | Data Type | Key Property |
+|---|---|---|---|
+| Cosine Similarity | User-Based CF, KNN | Continuous ratings | Simple, fast |
+| Adjusted Cosine | Item-Based CF | Continuous ratings | Removes user bias |
+| Pearson | (available) | Continuous ratings | Correlation-based |
+| Jaccard | (available) | Binary sets | Set overlap |
+
+All metrics are computed using **imputed matrices** — missing values (`NaN`) are filled with the global mean before similarity computation.

discussion/07-helpers.md

@@ -0,0 +1,94 @@
+# Utilities — `utils/helpers.py`
+
+## Overview
+
+Shared utility functions used by the recommender modules and the Flask application. Provides data loading, matrix construction, and user preference extraction.
+
+**File:** `utils/helpers.py` (37 lines)  
+**Dependencies:** `pandas`
+
+---
+
+## Functions
+
+### 1. `load_data()`
+
+```python
+def load_data():
+    products = pd.read_csv(...)
+    users = pd.read_csv(...)
+    ratings = pd.read_csv(...)
+    return products, users, ratings
+```
+
+**Purpose:** Loads the 3 CSV files from the `data/` directory into pandas DataFrames.
+
+**Returns:**
+- `products` — 201 products with columns: `product_id, name, category, subcategory, brand, price, avg_rating, num_reviews`
+- `users` — 200 users with columns: `user_id, name, age, preferred_categories, budget_min, budget_max, favorite_brands`
+- `ratings` — 8000 ratings with columns: `user_id, product_id, rating`
+
+**Called by:** `app.py` at module load time, data generator.
+
+---
+
+### 2. `build_user_item_matrix()`
+
+```python
+def build_user_item_matrix(ratings_df):
+    return ratings_df.pivot_table(
+        index="user_id", columns="product_id", values="rating"
+    )
+```
+
+**Purpose:** Pivots the ratings DataFrame into a **sparse matrix** where:
+- Rows = users (by `user_id`)
+- Columns = items (by `product_id`)
+- Values = ratings (`NaN` where no rating exists)
+
+**Output shape:** `(200 users, 201 products)` — a 200×201 matrix with ~8000 non-NaN entries (~20% density).
+
+**Called by:** `CollaborativeFiltering.__init__()`
+
+---
+
+### 3. `get_user_preferences()`
+
+```python
+def get_user_preferences(users_df, user_id):
+```
+
+**Purpose:** Extracts a user's preference profile from the users DataFrame into a clean Python dict.
+
+**Returns:**
+```python
+{
+    "preferred_categories": {"Electronics", "Sports"},  # set
+    "budget_min": 10.0,                                  # float
+    "budget_max": 500.0,                                 # float
+    "favorite_brands": {"Nike", "Samsung"},              # set
+    "name": "Alice_1",                                   # str
+    "age": 28,                                           # int
+}
+```
+
+**Key details:**
+- `preferred_categories` and `favorite_brands` are stored as comma-separated strings in CSV → converted to Python `set()` for efficient membership testing
+- All numeric types are converted to native Python types (`int`, `float`) for JSON serialization compatibility
+- Returns empty dict `{}` if user not found
+
+**Called by:** `app.py` API endpoints (`/api/recommend`, `/api/user/<id>/preferences`, evaluation).
+
+---
+
+## Data Flow
+
+```
+data/*.csv  →  load_data()  →  products, users, ratings (global DataFrames)
+                                  ↓
+                    build_user_item_matrix(ratings)  →  CF user-item matrix
+                                  ↓
+                    get_user_preferences(users, id)  →  preference dict
+```
+
+The DataFrames are loaded once at module load time and shared across all request handlers.

discussion/08-data-generator.md

@@ -0,0 +1,104 @@
+# Synthetic Data Generator — `data/generate_data.py`
+
+## Overview
+
+Generates a complete synthetic e-commerce dataset with 201 products, 200 users, 8000 ratings, and 2000 purchase interactions. The data has realistic structure: category hierarchies, brand-category associations, and preference-influenced ratings.
+
+**File:** `data/generate_data.py` (240 lines)  
+**Dependencies:** `pandas`, `numpy`
+
+---
+
+## Generated Files
+
+| File | Records | Columns |
+|---|---|---|
+| `products.csv` | 201 | `product_id, name, category, subcategory, brand, price, avg_rating, num_reviews` |
+| `users.csv` | 200 | `user_id, name, age, preferred_categories, budget_min, budget_max, favorite_brands` |
+| `ratings.csv` | 8000 | `user_id, product_id, rating` |
+| `interactions.csv` | 2000 | `user_id, product_id, purchased, quantity` |
+
+---
+
+## Product Generation (`generate_products`)
+
+### Category Hierarchy
+
+8 categories, each with 5 subcategories (40 subcategories total):
+
+| Category | Example Subcategories | Brands |
+|---|---|---|
+| Electronics | Smartphones, Laptops, Headphones, Tablets, Cameras | Samsung, Apple, Sony, LG... |
+| Clothing | Men's, Women's, Kids', Accessories, Footwear | Nike, Adidas, Zara, H&M... |
+| Home & Kitchen | Furniture, Appliances, Cookware, Decor, Storage | IKEA, KitchenAid, Ninja... |
+| Books | Fiction, Non-Fiction, Science, Technology, Self-Help | Penguin, HarperCollins... |
+| Sports | Fitness, Outdoor, Team Sports, Cycling, Swimming | Nike, Wilson, Adidas... |
+| Beauty | Skincare, Makeup, Haircare, Fragrance, Tools | L'Oreal, Clinique, Neutrogena... |
+| Toys | Educational, Action Figures, Board Games, Dolls, Puzzles | LEGO, Mattel, Hasbro... |
+| Automotive | Car Care, Interior, Exterior, Tools, Electronics | Meguiar's, Armor All... |
+
+### Product Generation Logic
+- 8 products per subcategory → 5 subcategories × 8 = 40 products per category → 8 categories × 40 = 320 total... but limited to available names (varies by subcategory)
+- **Price:** Uniform random between \$9.99 and \$1,499.99
+- **Rating:** Uniform random between 3.0 and 5.0 (rounded to 1 decimal)
+- **Reviews:** Random integer between 10 and 500
+
+### Product Name Templates
+Product names are curated per subcategory (e.g., Electronics/Laptops → "ThinkPad X1", "MacBook Pro", "XPS 15"). Each subcategory has 5 unique named products.
+
+---
+
+## User Generation (`generate_users`)
+
+Each user gets:
+1. **Name** — Random from 30 first names + user ID suffix (e.g., "Alice_1", "Bob_2")
+2. **Age** — Random integer 18-65
+3. **Preferred Categories** — 1-3 categories randomly selected from the 8 available
+4. **Budget** — `budget_min` (\$10-\$200) and `budget_max` (budget_min + \$50-\$1,500)
+5. **Favorite Brands** — 0-2 brands randomly chosen from ALL brands across categories
+
+---
+
+## Rating Generation (`generate_ratings`)
+
+### Data Volume
+- **8000 ratings** for 200 users × 201 products = 40,200 possible pairs → ~20% density
+- Each user has ~40 ratings on average (range varies by random selection)
+
+### Rating Formula
+
+```
+base = 3.0  (starting point)
+
+if product.category in user.preferred_categories:
+    base += random(0.5, 1.5)     # category preference bonus
+
+if product.brand in user.favorite_brands:
+    base += random(0.3, 1.0)     # brand loyalty bonus
+
+if product.price within user.budget:
+    base += random(0.0, 0.5)     # budget fit bonus
+else:
+    base -= random(0.0, 0.5)     # budget mismatch penalty
+
+rating = clamp(base + gaussian_noise(0, 0.5), to [1.0, 5.0])
+```
+
+This creates realistic patterns: users rate preferred-category items higher, within-budget items higher, and favorite-brand items higher. The Gaussian noise adds realistic variance.
+
+---
+
+## Interaction Generation (`generate_interactions`)
+
+Samples 2000 rows from the ratings DataFrame and marks them as "purchased" with a random quantity (1-3). This simulates purchase history separate from rating history.
+
+---
+
+## Running the Generator
+
+```bash
+cd TasteEngine
+python data/generate_data.py
+```
+
+This overwrites the 4 CSV files in `data/` with fresh synthetic data. The random seed is fixed (`np.random.seed(42)`) for reproducible output.

discussion/09-app.md

@@ -0,0 +1,171 @@
+# Flask Web Application — `app.py`
+
+## Overview
+
+The Flask application ties together all recommender modules, serving both **HTML pages** (server-rendered with Jinja2) and **JSON API endpoints** (consumed by Alpine.js on the frontend).
+
+**File:** `app.py` (453 lines)  
+**Dependencies:** `Flask`, `pandas`, `numpy`, `recommender/*`, `utils/*`
+
+---
+
+## Architecture
+
+### Module-Level Initialization
+
+```python
+products, users, ratings = load_data()           # Load CSVs once at startup
+cf = CollaborativeFiltering(ratings)              # CF with 5 methods
+cb = ContentBasedRecommender(products)            # Content-Based with 2 methods
+kb = KnowledgeBasedRecommender(products)          # Knowledge-Based with 3 methods
+explainer = Explainer(products, users)            # Explanation generator
+evaluator = Evaluator(ratings)                    # Evaluation metrics
+
+TRAIN, TEST = train_test_split(ratings)           # 80/20 split
+cf_train = CollaborativeFiltering(TRAIN)          # CF trained only on training set
+```
+
+All recommender instances are **global singletons** — created once when the server starts and shared across all requests. This means caches (SVD factors, Slope One deviation matrix) persist across requests.
+
+### Pre-computed Data
+
+| Variable | Type | Purpose |
+|---|---|---|
+| `USER_OPTIONS` | `list[dict]` | User dropdown options (name, age, categories, budget, brands) |
+| `CATEGORIES` | `list[str]` | All product categories |
+| `BRANDS` | `list[str]` | All product brands |
+| `APPROACHES` | `dict` | Approach/method definitions for UI buttons |
+
+---
+
+## Routes
+
+### Page Routes (HTML)
+
+| Route | Template | Description |
+|---|---|---|
+| `GET /` | `index.html` | Landing page with user selection + profile + preference editor |
+| `GET /recommend` | `recommend.html` | Recommendation interface with approach/method selection |
+| `GET /evaluate` | `evaluation.html` | Evaluation dashboard with metrics, charts, analysis |
+
+All page routes inject `active_page`, `users`, `categories`, `brands`, `approaches` as template variables.
+
+### API Routes (JSON)
+
+#### Users
+
+| Method | Route | Purpose |
+|---|---|---|
+| `GET` | `/api/users` | List all users |
+| `GET` | `/api/user/<id>` | Get single user preferences |
+| `POST` | `/api/users` | Create a new user |
+
+**Create User** (`POST /api/users`):
+```python
+new_id = max(user_ids) + 1
+# Creates DataFrame row, concats to global users
+# Appends to USER_OPTIONS for immediate dropdown availability
+```
+
+#### Products
+
+| Method | Route | Purpose |
+|---|---|---|
+| `GET` | `/api/products?category=X` | Get products (filtered by category) |
+| `GET` | `/api/products/filter?category=X&brand=Y&price_max=Z&q=search` | Filter products by multiple criteria |
+
+#### Recommendations
+
+| Method | Route | Purpose |
+|---|---|---|
+| `POST` | `/api/recommend` | Generate recommendations |
+
+**Request body:**
+```json
+{
+    "user_id": 1,
+    "approach": "cf",
+    "method": "user_based",
+    "n": 10
+}
+```
+
+**Flow:**
+1. Load user's rated items (`rating >= 3.5`) and preferences
+2. Dispatch to correct recommender based on `approach` + `method`
+3. Generate explanations for each recommendation
+4. Return list with product info, score, and explanation
+
+**Response:**
+```json
+{
+    "recommendations": [
+        {
+            "id": 58, "name": "X-T5", "category": "Electronics",
+            "price": 999.99, "avg_rating": 4.5,
+            "score": 0.92, "explanation": "Users with Electronics taste also liked X-T5"
+        }
+    ]
+}
+```
+
+#### Evaluation
+
+| Method | Route | Purpose |
+|---|---|---|
+| `GET` | `/api/evaluate/cf/<method>` | Evaluate a single CF method |
+| `GET` | `/api/evaluate/cf/<method>/stream` | Stream epoch/item progress during evaluation |
+| `GET` | `/api/evaluate/approaches` | Compare all 3 approaches |
+
+**Streaming endpoint** (`/stream`):
+Returns NDJSON (newline-delimited JSON):
+```
+{"type":"progress","current":47,"total":100}    ← SVD epoch 47/100
+{"type":"progress","current":48,"total":100}
+...
+{"type":"phase","label":"Evaluating users..."}   ← Training done, now evaluating
+{"type":"result","data":{"RMSE":0.9675,...}}     ← Final metrics
+```
+
+#### Preferences
+
+| Method | Route | Purpose |
+|---|---|---|
+| `PUT` | `/api/user/<id>/preferences` | Update user budget preferences |
+
+#### HTMX
+
+| Method | Route | Purpose |
+|---|---|---|
+| `POST` | `/htmx/recommend` | Return recommendation HTML fragment (for HTMX-powered requests) |
+
+---
+
+## UI Integration
+
+The Flask app integrates with a **3-tier frontend**:
+
+1. **Alpine.js** — manages interactive state (approach selection, results display, compare view, evaluation progress) via `x-data` components
+2. **HTMX** — provides AJAX navigation patterns (loaded but used minimally)
+3. **Lucide** — SVG icon library (loaded but used for brand icon only)
+
+### Key Frontend-Backend Interactions
+
+| Feature | Frontend | Backend |
+|---|---|---|
+| User selection | Alpine reads `userData` from template variable | Injects `users` as JSON |
+| Recommendations | Alpine `fetch()` → `POST /api/recommend` | Returns JSON with product list |
+| Compare all | Parallel `fetch()` → multiple `POST /api/recommend` | Each returns recommendations |
+| Evaluation | Incremental `fetch()` → per-method endpoints | SVD/Slope One use streaming |
+| Preferences | Alpine `fetch()` → `PUT /api/user/<id>/preferences` | Updates in-memory DataFrame |
+
+---
+
+## Error Handling
+
+All API endpoints return JSON errors with appropriate HTTP status codes:
+- `400` — Missing or invalid parameters
+- `404` — User/product not found
+- `500` — Internal server error with error message
+
+Frontend displays errors via the `toast()` notification system.

discussion/10-mathematical-details.md

@@ -0,0 +1,723 @@
+# Mathematical Foundations of Recommender Systems
+
+## Table of Contents
+
+1. [Collaborative Filtering](#1-collaborative-filtering)
+   - [1.1 User-Based CF](#11-user-based-cf)
+   - [1.2 Item-Based CF](#12-item-based-cf)
+   - [1.3 SVD (Matrix Factorization)](#13-svd-matrix-factorization)
+   - [1.4 KNN-Based CF](#14-knn-based-cf)
+   - [1.5 Slope One](#15-slope-one)
+2. [Content-Based Filtering](#2-content-based-filtering)
+   - [2.1 TF-IDF Similarity](#21-tf-idf-similarity)
+   - [2.2 Feature Matching](#22-feature-matching)
+3. [Knowledge-Based Filtering](#3-knowledge-based-filtering)
+   - [3.1 Constraint-Based](#31-constraint-based)
+   - [3.2 Rule-Based](#32-rule-based)
+   - [3.3 Utility-Based](#33-utility-based)
+4. [Similarity Metrics](#4-similarity-metrics)
+   - [4.1 Cosine Similarity](#41-cosine-similarity)
+   - [4.2 Pearson Correlation](#42-pearson-correlation)
+   - [4.3 Adjusted Cosine Similarity](#43-adjusted-cosine-similarity)
+   - [4.4 Jaccard Similarity](#44-jaccard-similarity)
+5. [Evaluation Metrics](#5-evaluation-metrics)
+   - [5.1 RMSE](#51-rmse)
+   - [5.2 MAE](#52-mae)
+   - [5.3 Precision@k](#53-precisionk)
+   - [5.4 Recall@k](#54-recallk)
+   - [5.5 F1@k](#55-f1k)
+   - [5.6 Coverage](#56-coverage)
+
+---
+
+# 1. Collaborative Filtering
+
+## Notation
+
+Let:
+
+- $U = \{1, 2, \dots, n_u\}$ be the set of $n_u$ users
+- $I = \{1, 2, \dots, n_i\}$ be the set of $n_i$ items
+- $\mathbf{R} \in \mathbb{R}^{n_u \times n_i}$ be the user-item rating matrix, where $r_{u,i}$ is the rating of user $u$ for item $i$
+- $\mathbf{R}_{u,i} = \text{NaN}$ if user $u$ has not rated item $i$
+- $\mu$ be the global mean rating across all observed ratings
+- $\mu_u$ be the mean rating of user $u$
+
+---
+
+## 1.1 User-Based CF
+
+### Concept
+
+Predict ratings by aggregating ratings from the **most similar users** (neighbors) to the target user.
+
+### Step 1: Compute User Similarity Matrix
+
+Let $\mathbf{R}^\text{imp}$ be the imputed rating matrix where all NaN values are replaced by the global mean $\mu$:
+
+$$\hat{r}_{u,i} = \begin{cases} r_{u,i} & \text{if } r_{u,i} \text{ is observed} \\ \mu & \text{otherwise} \end{cases}$$
+
+Compute the pair-wise cosine similarity matrix $\mathbf{S} \in \mathbb{R}^{n_u \times n_u}$:
+
+$$s_{u,v} = \cos(\mathbf{R}^\text{imp}_{u,:}, \mathbf{R}^\text{imp}_{v,:}) = \frac{\mathbf{R}^\text{imp}_{u,:} \cdot \mathbf{R}^\text{imp}_{v,:}}{\|\mathbf{R}^\text{imp}_{u,:}\| \cdot \|\mathbf{R}^\text{imp}_{v,:}\|}$$
+
+Where $\mathbf{R}^\text{imp}_{u,:}$ is the row vector of user $u$'s imputed ratings.
+
+### Step 2: Predict Rating for Unseen Item
+
+For each unseen item $i$ (where $\mathbf{R}_{u,i} = \text{NaN}$):
+
+1. Find the $k$ most similar users who have rated item $i$ (similarity $s_{u,v} > 0$)
+2. Let $N_{u,i} = \{v_1, v_2, \dots, v_k\}$ be the set of valid neighbors
+
+The predicted rating is the **weighted average**:
+
+$$\hat{r}_{u,i} = \frac{\sum_{v \in N_{u,i}} s_{u,v} \cdot r_{v,i}}{\sum_{v \in N_{u,i}} s_{u,v}}$$
+
+### Step 3: Generate Top-N Recommendations
+
+$$\text{Rec}(u) = \text{top-}n \text{ items with highest } \hat{r}_{u,i} \text{ among unseen items}$$
+
+### Complexity
+
+- Similarity matrix: $O(n_u^2 \cdot n_i)$
+- Per-user prediction: $O(k \cdot n_i)$
+
+### Free parameters
+
+- $k = 20$ (number of neighbors)
+- Imputation strategy: global mean
+
+---
+
+## 1.2 Item-Based CF
+
+### Concept
+
+Predict ratings by finding items **similar to items the user already rated highly**. Item similarities are computed using **adjusted cosine similarity** to remove user bias.
+
+### Step 1: Compute Item-Adjusted Cosine Similarity
+
+For each pair of items $(i, j)$, the adjusted cosine similarity is:
+
+$$s_{i,j}^\text{adj} = \frac{\sum_{u \in U_{ij}} (r_{u,i} - \mu_u)(r_{u,j} - \mu_u)}{\sqrt{\sum_{u \in U_{ij}} (r_{u,i} - \mu_u)^2} \sqrt{\sum_{u \in U_{ij}} (r_{u,j} - \mu_u)^2}}$$
+
+Where:
+- $U_{ij}$ is the set of users who rated both items $i$ and $j$
+- $\mu_u$ is user $u$'s mean rating
+
+This normalizes each user's ratings by their mean, removing the bias of users who rate everything high or low.
+
+### Step 2: Predict Rating
+
+Let $I_u^+ = \{i \in I : r_{u,i} \text{ is observed}\}$ be the set of items user $u$ has rated.
+
+For each unseen item $i$, find the top $k$ most similar items among $I_u^+$:
+
+$$N_{i,u} = \text{top-}k \left\{ j \in I_u^+ : s_{i,j}^\text{adj} > 0 \right\}$$
+
+$$\hat{r}_{u,i} = \frac{\sum_{j \in N_{i,u}} s_{i,j}^\text{adj} \cdot r_{u,j}}{\sum_{j \in N_{i,u}} s_{i,j}^\text{adj}}$$
+
+### Step 3: Generate Top-N Recommendations
+
+Same as User-Based CF — sort by predicted rating.
+
+### Complexity
+
+- Item similarity matrix: $O(n_i^2 \cdot n_u)$ (precomputed once)
+- Per-user prediction: $O(n_i \cdot |I_u^+|)$
+
+### Free parameters
+
+- $k = 15$ (number of item neighbors)
+- Similarity metric: adjusted cosine
+
+---
+
+## 1.3 SVD (Matrix Factorization)
+
+### Concept
+
+Decompose the user-item rating matrix into two low-rank factor matrices that capture **latent features** (e.g., genre affinity, price sensitivity, brand loyalty).
+
+### Problem Statement
+
+Approximate the rating matrix $\mathbf{R}$ as a product of two lower-dimensional matrices:
+
+$$\mathbf{R} \approx \mathbf{P} \cdot \mathbf{Q}^T$$
+
+Where:
+- $\mathbf{P} \in \mathbb{R}^{n_u \times f}$ — user factor matrix (each row = user's latent preferences)
+- $\mathbf{Q} \in \mathbb{R}^{n_i \times f}$ — item factor matrix (each row = item's latent attributes)
+- $f$ = number of latent factors (typically 20-100)
+
+### Bias-Enhanced Prediction
+
+The predicted rating includes user/item bias terms:
+
+$$\hat{r}_{u,i} = \mu + b_u + b_i + \mathbf{p}_u \cdot \mathbf{q}_i^T$$
+
+Where:
+- $\mu$ = global mean rating
+- $b_u$ = bias of user $u$ (tendency to rate higher/lower than average)
+- $b_i$ = bias of item $i$ (tendency to be rated higher/lower than average)
+- $\mathbf{p}_u$ = user $u$'s latent factor vector (row of $\mathbf{P}$)
+- $\mathbf{q}_i$ = item $i$'s latent factor vector (row of $\mathbf{Q}$)
+
+### Objective Function (to minimize)
+
+$$L = \sum_{(u,i) \in \mathcal{K}} \left( r_{u,i} - \hat{r}_{u,i} \right)^2 + \lambda \left( b_u^2 + b_i^2 + \|\mathbf{p}_u\|^2 + \|\mathbf{q}_i\|^2 \right)$$
+
+Where:
+- $\mathcal{K}$ = set of all observed ratings
+- $\lambda$ = regularization parameter (prevents overfitting)
+
+### Optimization — Stochastic Gradient Descent
+
+For each observed rating $(u, i)$, compute the error:
+
+$$e_{u,i} = r_{u,i} - \hat{r}_{u,i}$$
+
+Update parameters by moving in the direction that reduces error:
+
+$$
+\begin{aligned}
+b_u &\leftarrow b_u + \eta \cdot (e_{u,i} - \lambda \cdot b_u) \\
+b_i &\leftarrow b_i + \eta \cdot (e_{u,i} - \lambda \cdot b_i) \\
+\mathbf{p}_u &\leftarrow \mathbf{p}_u + \eta \cdot (e_{u,i} \cdot \mathbf{q}_i - \lambda \cdot \mathbf{p}_u) \\
+\mathbf{q}_i &\leftarrow \mathbf{q}_i + \eta \cdot (e_{u,i} \cdot \mathbf{p}_u - \lambda \cdot \mathbf{q}_i)
+\end{aligned}
+$$
+
+Where $\eta$ is the learning rate.
+
+### Algorithm
+
+```
+1. Initialize P, Q with small random values N(0, 0.1)
+2. Initialize bu, bi = 0
+3. For epoch = 1 to n_epochs:
+   a. Shuffle observed rating pairs
+   b. For each (u, i) in observed:
+      - Compute prediction error e_ui = r_ui - (mu + bu[u] + bi[i] + P[u]·Q[i])
+      - Update bu[u], bi[i], P[u], Q[i]
+4. Return (P, Q, bu, bi) — cached for future predictions
+```
+
+### Prediction (after training)
+
+For an unseen item $i$ for user $u$:
+
+$$\hat{r}_{u,i} = \mu + b_u + b_i + \mathbf{p}_u \cdot \mathbf{q}_i^T$$
+
+### Caching Strategy
+
+The trained factors $(P, Q, b_u, b_i)$ are cached. The first call triggers 100 epochs of training. All subsequent calls use the cached factors — prediction is $O(n_i \cdot f)$ per user.
+
+### Hyperparameters
+
+| Parameter | Value | Effect |
+|---|---|---|
+| $f$ (factors) | 20 | Higher = more expressive, risk of overfitting |
+| $n_\text{epochs}$ | 100 | Higher = better convergence |
+| $\eta$ (lr) | 0.01 | Higher = faster convergence, may overshoot |
+| $\lambda$ (reg) | 0.02 | Higher = stronger regularization |
+
+---
+
+## 1.4 KNN-Based CF
+
+### Concept
+
+A variant of User-Based CF that uses **sklearn's optimized NearestNeighbors** to efficiently find similar users, then weights their ratings by inverse distance.
+
+### Step 1: Find Nearest Neighbors
+
+Let $\mathbf{R}^\text{imp}$ be the mean-imputed matrix (same as User-Based CF).
+
+For target user $u$, find $k+1$ nearest neighbors using cosine distance:
+
+$$\text{dist}(\mathbf{R}^\text{imp}_{u,:}, \mathbf{R}^\text{imp}_{v,:}) = 1 - \cos(\mathbf{R}^\text{imp}_{u,:}, \mathbf{R}^\text{imp}_{v,:})$$
+
+The nearest neighbor algorithm returns:
+- $\text{indices}[0] = [u, v_1, v_2, \dots, v_k]$ — indice 0 is the user themselves
+- $\text{distances}[0] = [0, d_1, d_2, \dots, d_k]$ — distances to each neighbor
+
+Neighbors = $\text{indices}[0][1:]$ (exclude self).
+
+### Step 2: Predict Rating
+
+For each unseen item $i$:
+
+$$\hat{r}_{u,i} = \frac{\sum_{v \in N_u} w_v \cdot r_{v,i}}{\sum_{v \in N_u} w_v}$$
+
+Where weights are **inverse distances** (closer = higher weight):
+
+$$w_v = \frac{1}{d_v + \epsilon}, \quad \epsilon = 10^{-6}$$
+
+Only neighbors who rated item $i$ contribute ($r_{v,i} \neq \text{NaN}$).
+
+### Complexity
+
+- KNN index building: $O(n_u \cdot f_\text{tree})$ via sklearn's KD-tree/ball-tree
+- Per-user neighbor search: $O(\log n_u)$
+- Per-user prediction: $O(k \cdot n_i)$
+
+### Free parameters
+
+- $k = 10$ (number of neighbors)
+- Metric: cosine distance
+- Weighting: inverse distance
+
+### Key Difference from User-Based CF
+
+User-Based CF computes all pair-wise similarities explicitly (dense $n_u \times n_u$ matrix). KNN-CF uses a tree structure for approximate nearest neighbor search, which is **more efficient for large user bases** but slightly less precise.
+
+---
+
+## 1.5 Slope One
+
+### Concept
+
+A simple yet effective collaborative filtering algorithm based on the **average rating difference** between items. "If on average item A is rated 0.5 stars higher than item B, and you rate item B 4.0, you'll likely rate item A 4.5."
+
+### Step 1: Precompute Deviation Matrix
+
+For every pair of items $(i, j)$, compute:
+
+$$d_{i,j} = \frac{\sum_{u \in U_{ij}} (r_{u,i} - r_{u,j})}{|U_{ij}|}$$
+
+$$c_{i,j} = |U_{ij}|$$
+
+Where:
+- $U_{ij}$ = set of users who rated both items $i$ and $j$
+- $d_{i,j}$ = average deviation (how much higher/lower item $i$ is rated compared to item $j$)
+- $c_{i,j}$ = number of users who contributed to this deviation (confidence)
+
+This produces two $n_i \times n_i$ matrices: $\mathbf{D}$ (deviations) and $\mathbf{C}$ (counts).
+
+### Step 2: Predict Rating
+
+For user $u$ and unseen item $i$:
+
+Let $I_u^\text{rated} = \{j : r_{u,j} \text{ is observed}\}$.
+
+$$\hat{r}_{u,i} = \frac{\sum_{j \in I_u^\text{rated}} (r_{u,j} + d_{i,j}) \cdot \mathbf{1}[c_{i,j} > 0]}{\sum_{j \in I_u^\text{rated}} \mathbf{1}[c_{i,j} > 0]}$$
+
+Where $\mathbf{1}[c_{i,j} > 0]$ is an indicator function that is 1 when the deviation is well-defined (at least one user rated both items).
+
+Only items $j$ with $c_{i,j} > 0$ contribute to the prediction.
+
+### Complexity
+
+- Precomputation: $O(n_i^2 \cdot n_u)$ (done once, cached)
+- Per-user prediction: $O(n_i \cdot |I_u^\text{rated}|)$
+
+### Key Properties
+
+- **No hyperparameters** — fully deterministic
+- **Incremental** — new ratings can be incorporated by updating $d_{i,j}$ and $c_{i,j}$ without retraining
+- **Interpretable** — each prediction is the average of multiple "if you liked X, you'd probably rate Y this much" estimates
+
+---
+
+# 2. Content-Based Filtering
+
+## 2.1 TF-IDF Similarity
+
+### Concept
+
+Build a **text profile** for each product from its attributes (category, subcategory, brand, name), then recommend products whose TF-IDF vectors are most similar to the user's profile (average of items they liked).
+
+### Step 1: Build Text Features
+
+For each product $p$, construct a text document:
+
+$$\text{text}_p = \text{category}_p \oplus \text{subcategory}_p \oplus \text{brand}_p \oplus \text{name}_p$$
+
+### Step 2: Compute TF-IDF Matrix
+
+The **Term Frequency — Inverse Document Frequency** (TF-IDF) weight for term $t$ in product $p$:
+
+$$\text{tf-idf}_{t,p} = \text{tf}_{t,p} \times \text{idf}_t$$
+
+Where:
+- $\text{tf}_{t,p}$ = raw count of term $t$ in product $p$'s text (normalized by document length)
+- $\text{idf}_t = \log\left(\frac{n_i}{\text{df}_t}\right) + 1$ (where $\text{df}_t$ = number of products containing term $t$)
+
+This produces a matrix $\mathbf{T} \in \mathbb{R}^{n_i \times m}$ where $m$ is the vocabulary size (number of unique words across all product texts).
+
+### Step 3: Build User Profile
+
+Let $I_u^+ = \{p_1, p_2, \dots, p_k\}$ be the set of products user $u$ rated $\geq 3.5$.
+
+The user's content profile is the **centroid** (average) of their liked products' TF-IDF vectors:
+
+$$\mathbf{c}_u = \frac{1}{|I_u^+|} \sum_{p \in I_u^+} \mathbf{T}_{p,:}$$
+
+### Step 4: Score Products
+
+For each product $q \notin I_u^+$, compute the cosine similarity between the user profile and the product's TF-IDF vector:
+
+$$\text{score}_{u,q} = \cos(\mathbf{c}_u, \mathbf{T}_{q,:}) = \frac{\mathbf{c}_u \cdot \mathbf{T}_{q,:}}{\|\mathbf{c}_u\| \cdot \|\mathbf{T}_{q,:}\|}$$
+
+### Step 5: Generate Recommendations
+
+$$\text{Rec}(u) = \text{top-}n \text{ products with highest } \text{score}_{u,q} \text{ among products not in } I_u^+$$
+
+### Complexity
+
+- Vectorization: $O(n_i \cdot m)$
+- Profile building: $O(|I_u^+| \cdot m)$
+- Scoring: $O(n_i \cdot m)$
+
+---
+
+## 2.2 Feature Matching
+
+### Concept
+
+Score products based on explicit preference matches without requiring rating history. A **rule-based scoring** function assigns points for each attribute match.
+
+### Scoring Function
+
+For each product $p$, the total score is:
+
+$$S_p = w_{\text{cat}} \cdot \mathbf{1}[\text{cat}_p \in \text{pref}_\text{cat}] + w_{\text{brand}} \cdot \mathbf{1}[\text{brand}_p \in \text{pref}_\text{brand}] + w_{\text{budget}} \cdot \mathbf{1}[\text{budget}_{\min} \leq \text{price}_p \leq \text{budget}_{\max}] + w_{\text{sub}} \cdot \mathbf{1}[\text{subcat}_p \in \text{pref}_\text{subcat}]$$
+
+Where:
+
+| Component | Weight | Condition |
+|---|---|---|
+| Category match | $w_{\text{cat}} = 40$ | Product category $\in$ user's preferred categories |
+| Brand match | $w_{\text{brand}} = 30$ | Product brand $\in$ user's favorite brands |
+| Budget fit | $w_{\text{budget}} = 20$ | Product price within user's budget range |
+| Subcategory match | $w_{\text{sub}} = 10$ | Product subcategory matches (complex condition) |
+
+### Complexity
+
+$O(n_i)$ — a single pass over the product catalog.
+
+### Cold-Start Advantage
+
+Unlike TF-IDF, Feature Matching requires **zero rating history**. It only needs preference data (categories, brands, budget), making it suitable for new users.
+
+---
+
+# 3. Knowledge-Based Filtering
+
+## 3.1 Constraint-Based
+
+### Concept
+
+Sequentially filter the product catalog by hard constraints, then sort by rating.
+
+### Filtering Pipeline
+
+Starting with the full product set $\mathcal{P}_0 = \mathcal{P}$:
+
+| Step | Filter | Result |
+|---|---|---|
+| 1 | Budget max | $\mathcal{P}_1 = \{p \in \mathcal{P}_0 : \text{price}_p \leq B_{\max}\}$ |
+| 2 | Budget min | $\mathcal{P}_2 = \{p \in \mathcal{P}_1 : \text{price}_p \geq B_{\min}\}$ |
+| 3 | Category | $\mathcal{P}_3 = \{p \in \mathcal{P}_2 : \text{cat}_p \in C_{\text{pref}}\}$ (if $C_{\text{pref}}$ non-empty) |
+| 4 | Brand | $\mathcal{P}_4 = \{p \in \mathcal{P}_3 : \text{brand}_p \in B_{\text{pref}}\}$ (only if match exists) |
+| 5 | Min rating | $\mathcal{P}_5 = \{p \in \mathcal{P}_4 : \text{rating}_p \geq R_{\min}\}$ |
+
+### Safety Mechanism
+
+Steps 4 and 5 use a **soft filter**: the filter only activates if there's at least one matching product. Otherwise, the constraint is skipped to prevent empty results.
+
+### Ranking
+
+$$\text{Rec}(u) = \text{top-}n \text{ products from } \mathcal{P}_5 \text{ sorted by } \text{avg\_rating}_p \text{ descending}$$
+
+### Complexity
+
+$O(n_i)$ — each filter is a linear pass.
+
+---
+
+## 3.2 Rule-Based
+
+### Concept
+
+Score products using domain-specific **cross-selling rules** combined with preference signals.
+
+### Cross-Selling Rules
+
+The system defines rules mapping interacted categories to accessory recommendations:
+
+$$\mathcal{R} = \{(c_{\text{base}}, \{a_1, a_2, \dots\}) : \text{ buying } c_{\text{base}} \text{ triggers recommending accessories}\}$$
+
+Example: `Laptops → {Laptop Bag, Mouse, Keyboard, Cooling Pad}`
+
+### Scoring Function
+
+For product $p$ with category $\text{cat}_p$, subcategory $\text{sub}_p$, brand $\text{brand}_p$, and price $\text{price}_p$:
+
+$$
+S_p = 
+\begin{aligned}
+& w_{\text{cross}} \cdot \mathbf{1}[\text{sub}_p \in \text{accessories}(\text{interacted\_cat})] \\
+&+ w_{\text{cat}} \cdot \mathbf{1}[\text{cat}_p \in \text{pref}_\text{cat}] \\
+&+ w_{\text{budget}} \cdot \mathbf{1}[\text{budget}_{\min} \leq \text{price}_p \leq \text{budget}_{\max}] \\
+&+ w_{\text{brand}} \cdot \mathbf{1}[\text{brand}_p \in \text{pref}_\text{brand}]
+\end{aligned}
+$$
+
+With weights:
+- $w_{\text{cross}} = 50$ — strongest signal (direct cross-sell)
+- $w_{\text{cat}} = 20$ — category preference
+- $w_{\text{budget}} = 15$ — budget fit
+- $w_{\text{brand}} = 10$ — brand loyalty
+
+### Complexity
+
+$O(n_i)$ — scores every product once.
+
+---
+
+## 3.3 Utility-Based
+
+### Concept
+
+Compute a **weighted multi-attribute utility** for each product, measuring overall satisfaction across four dimensions.
+
+### Utility Function
+
+$$U(p) = w_{\text{price}} \cdot u_{\text{price}}(p) + w_{\text{cat}} \cdot u_{\text{cat}}(p) + w_{\text{brand}} \cdot u_{\text{brand}}(p) + w_{\text{rating}} \cdot u_{\text{rating}}(p)$$
+
+With default weights $\mathbf{w} = [0.2, 0.3, 0.2, 0.3]$ (summing to 1.0).
+
+### Component Utility Functions
+
+**Price utility** — Measures how close the price is to the center of the user's budget range:
+
+$$u_{\text{price}}(p) = 1 - \frac{|\text{price}_p - \frac{B_{\min} + B_{\max}}{2}|}{\frac{B_{\max} - B_{\min}}{2}}$$
+
+Clipped to $[0, 1]$. Scores 1.0 at the budget midpoint, 0.0 at the edges.
+
+**Category utility** — Binary:
+
+$$u_{\text{cat}}(p) = \begin{cases} 1 & \text{if } \text{cat}_p \in \text{pref}_\text{cat} \\ 0 & \text{otherwise} \end{cases}$$
+
+**Brand utility** — Binary:
+
+$$u_{\text{brand}}(p) = \begin{cases} 1 & \text{if } \text{brand}_p \in \text{pref}_\text{brand} \\ 0 & \text{otherwise} \end{cases}$$
+
+**Rating utility** — Normalized product quality:
+
+$$u_{\text{rating}}(p) = \frac{\text{avg\_rating}_p}{\max(\text{avg\_rating})}$$
+
+### Final Score
+
+$$U(p) \in [0, 1] \text{ for each product}$$
+
+$$\text{Rec}(u) = \text{top-}n \text{ products with highest } U(p)$$
+
+### Complexity
+
+$O(n_i)$ — computes utility for every product.
+
+### Advantages Over Constraint-Based
+
+Unlike hard filtering (Constraint-Based), Utility-Based uses **soft scoring** — a product that slightly exceeds the budget but is in a preferred category and has high ratings can still rank highly. This produces more diverse recommendations.
+
+---
+
+# 4. Similarity Metrics
+
+## 4.1 Cosine Similarity
+
+### Definition
+
+For two vectors $\mathbf{a}, \mathbf{b} \in \mathbb{R}^n$:
+
+$$\cos(\mathbf{a}, \mathbf{b}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\| \cdot \|\mathbf{b}\|} = \frac{\sum_{i=1}^n a_i b_i}{\sqrt{\sum_{i=1}^n a_i^2} \sqrt{\sum_{i=1}^n b_i^2}}$$
+
+### Properties
+
+- Range: $[-1, 1]$ (for non-negative rating vectors: $[0, 1]$)
+- Measures the **angle** between vectors, ignoring magnitude
+- Two users who rate things proportionally (Alice: [4, 4, 4], Bob: [5, 5, 5]) have $\cos = 1$
+
+### Used For
+- User-Based CF: user-user similarity
+- Content-Based: user profile vs product vector similarity
+
+---
+
+## 4.2 Pearson Correlation
+
+### Definition
+
+Measures linear correlation between two vectors by first **centering** each by its mean:
+
+$$\text{pearson}(\mathbf{a}, \mathbf{b}) = \frac{\sum_{i=1}^n (a_i - \bar{a})(b_i - \bar{b})}{\sqrt{\sum_{i=1}^n (a_i - \bar{a})^2} \sqrt{\sum_{i=1}^n (b_i - \bar{b})^2}}$$
+
+### Relationship to Cosine
+
+Pearson is equivalent to cosine similarity on **mean-centered** vectors:
+
+$$\text{pearson}(\mathbf{a}, \mathbf{b}) = \cos(\mathbf{a} - \bar{a}, \mathbf{b} - \bar{b})$$
+
+### Why It Matters
+
+Removes user bias: Alice rates everything 4-5, Bob rates 2-3, but they agree on relative preferences → Pearson captures this.
+
+---
+
+## 4.3 Adjusted Cosine Similarity
+
+### Definition
+
+Applied to **item-item** similarity. For items $i$ and $j$:
+
+$$\text{adj\_cos}(i, j) = \frac{\sum_{u \in U_{ij}} (r_{u,i} - \mu_u)(r_{u,j} - \mu_u)}{\sqrt{\sum_{u \in U_{ij}} (r_{u,i} - \mu_u)^2} \sqrt{\sum_{u \in U_{ij}} (r_{u,j} - \mu_u)^2}}$$
+
+Where:
+- $U_{ij}$ = users who rated both items
+- $\mu_u$ = user $u$'s mean rating
+
+### Key Insight
+
+This is identical to the Pearson formula but applied **column-wise** (item space) instead of row-wise (user space). The centering by user mean removes the same user-bias issue.
+
+### Implementation
+
+In code, this is computed by:
+1. Centering each user's ratings: $\mathbf{R}^\text{centered}_{u,i} = r_{u,i} - \mu_u$ (NaN if unrated)
+2. Imputing NaN with 0
+3. Computing cosine similarity on the **transposed** matrix: $\text{sklearn\_cosine}((\mathbf{R}^\text{centered})^T)$
+
+---
+
+## 4.4 Jaccard Similarity
+
+### Definition
+
+For two sets $A$ and $B$:
+
+$$J(A, B) = \frac{|A \cap B|}{|A \cup B|}$$
+
+### Example
+
+$A = \{\text{Electronics, Sports}\}$, $B = \{\text{Electronics, Books}\}$
+$$J(A, B) = \frac{1}{3} \approx 0.33$$
+
+### Why Not Used Here
+
+Jaccard is designed for **binary data** (purchased/not purchased). Rating data is continuous (1-5 stars). Using Jaccard would discard valuable rating magnitude information.
+
+---
+
+# 5. Evaluation Metrics
+
+## Notation
+
+For evaluation, define:
+
+- $\mathcal{Y}_\text{true}$ = set of actual ratings in test set
+- $\mathcal{Y}_\text{pred}$ = set of corresponding predicted ratings
+- $\text{Rec}_k(u)$ = top-$k$ recommended items for user $u$
+- $\text{Rel}(u)$ = relevant items for user $u$ (rated $\geq 3.5$ in test set)
+- $\mathcal{U}_\text{test}$ = test set users
+
+---
+
+## 5.1 RMSE
+
+Root Mean Square Error:
+
+$$\text{RMSE} = \sqrt{\frac{1}{n} \sum_{(u,i) \in \text{test}} (r_{u,i} - \hat{r}_{u,i})^2}$$
+
+### Properties
+- **Scale:** Same units as ratings (0-5 for a 5-star system)
+- **Interpretation:** Average prediction error, **larger errors penalized quadratically**
+- **Range:** $[0, 5]$ (for 1-5 ratings)
+- **Lower is better**
+
+### Why Square Errors?
+
+An error of 2.0 contributes 4× as much as an error of 1.0 ($2^2 = 4, 1^2 = 1$). This heavily penalizes large mistakes — important because a recommendation with a wildly wrong predicted rating is worse than one slightly off.
+
+---
+
+## 5.2 MAE
+
+Mean Absolute Error:
+
+$$\text{MAE} = \frac{1}{n} \sum_{(u,i) \in \text{test}} |r_{u,i} - \hat{r}_{u,i}|$$
+
+### Properties
+- **Scale:** Same units as ratings
+- **Interpretation:** Average absolute prediction error, **all errors weighted equally**
+- **Range:** $[0, 5]$
+- **Lower is better**
+
+### RMSE vs MAE
+
+If $\text{RMSE} \approx \text{MAE}$, prediction errors are evenly distributed. If $\text{RMSE} > \text{MAE}$ significantly, the model occasionally makes very large errors (outliers).
+
+---
+
+## 5.3 Precision@k
+
+$$\text{Precision@k} = \frac{1}{|\mathcal{U}_\text{test}|} \sum_{u \in \mathcal{U}_\text{test}} \frac{|\text{Rec}_k(u) \cap \text{Rel}(u)|}{k}$$
+
+### Properties
+- **Range:** $[0, 1]$
+- **Interpretation:** Of the top-$k$ recommendations, what fraction are relevant?
+- **Higher is better**
+
+### Example
+
+If a user's top-5 recommendations contain 2 relevant items: $\text{Precision@5} = 2/5 = 0.4$
+
+---
+
+## 5.4 Recall@k
+
+$$\text{Recall@k} = \frac{1}{|\mathcal{U}_\text{test}|} \sum_{u \in \mathcal{U}_\text{test}} \frac{|\text{Rec}_k(u) \cap \text{Rel}(u)|}{|\text{Rel}(u)|}$$
+
+### Properties
+- **Range:** $[0, 1]$
+- **Interpretation:** Of all relevant items, what fraction appears in the top-$k$?
+- **Higher is better**
+
+### Example
+
+If a user has 10 relevant items and 2 appear in the top-5: $\text{Recall@5} = 2/10 = 0.2$
+
+---
+
+## 5.5 F1@k
+
+Harmonic mean of Precision and Recall:
+
+$$\text{F1@k} = \frac{1}{|\mathcal{U}_\text{test}|} \sum_{u \in \mathcal{U}_\text{test}} \frac{2 \cdot \text{Precision@k}(u) \cdot \text{Recall@k}(u)}{\text{Precision@k}(u) + \text{Recall@k}(u)}$$
+
+### Properties
+- **Range:** $[0, 1]$
+- **Interpretation:** Balanced measure — only high if BOTH precision and recall are high
+- **Higher is better**
+
+### Why Harmonic Mean?
+
+The harmonic mean is more conservative than the arithmetic mean. If $\text{Precision} = 0.9$ but $\text{Recall} = 0.1$, the F1 score is $2 \cdot 0.9 \cdot 0.1 / (0.9 + 0.1) = 0.18$ — correctly reflecting that one metric is poor. Arithmetic mean would give $0.5$, misleadingly optimistic.
+
+---
+
+## 5.6 Coverage
+
+$$\text{Coverage} = \frac{|\bigcup_{u \in \mathcal{U}_\text{test}} \text{Rec}_k(u)|}{n_i}$$
+
+### Properties
+- **Range:** $[0, 1]$
+- **Interpretation:** What fraction of the total product catalog ever appears in anyone's top-$k$?
+- **Higher is better**
+
+### Significance
+
+Low coverage means the system only recommends popular items (the "Harry Potter problem" — everyone is recommended the same blockbusters). High coverage means niche/long-tail items also have a chance to be recommended.

requirements.txt

@@ -2,3 +2,5 @@ Flask==3.0.0
 pandas==2.1.4
 numpy==1.26.2
 scikit-learn==1.3.2
+gunicorn==22.0.0
+python-dotenv==1.0.1