---
license: mit
tags:
  - tabular
  - ensemble
  - stacking
  - regression
  - scikit-learn
  - fertilizer
library_name: scikit-learn
model_name: Swandip_optimal_fertilizer
datasets:
  - custom
inference: false
---

# Swandip's Optimal Fertilizer Prediction Ensemble 🌱

## 🧠 Overview
This repository contains a powerful **Level-3 stacked ensemble model** designed for optimal fertilizer prediction. The model architecture uses **over 80 diverse base learners**, multiple advanced ensemble strategies, and robust feature engineering techniques to achieve **state-of-the-art log loss performance**.

> **Best LogLoss: `1.86257` (Ridge Ensemble)**  
> **Hill Climbing LogLoss: `1.86554`**

---

## 📊 Evaluation Metric
**`LogLoss`** was used as the primary evaluation metric to assess model performance.

---

## 🧱 Architecture

The model follows a 3-tier ensemble stacking structure:

### Level 1 - Diverse Base Models

Includes over **80+ models** from the following families:
- **AutoGluon (27 models)**
- **MLP (x2)**
- **XGBoost (x20)** – tuned with bagging and early stopping
- **LGBM GBDT (x4)** and **LGBM GOSS**
- **TabTransformer (x3)**
- **Neural Networks (NNx15)** – including deep tabular variants
- **CatBoost (x2)**
- **HistGradientBoost (HGBx2)**
- **YDF (Yandex Decision Forest)**

### Feature Engineering Highlights:
- ✅ **Binned Features** – Numerical columns transformed into bins to capture non-linear effects.
- ✅ **All Numerical → Categorical** – Applied label encoding or one-hot encoding to convert features.
- ✅ **Data Augmentation** using `Train+Orig` blending.

---

### Level 2 - Intermediate Ensembles

Various ensemble strategies were applied to Level-1 predictions:
- 🔁 **Logistic Regression (LR)**
- 🧱 **Voting Classifier**
- 🎯 **Stacking Classifier**
- ⚖️ **Weighted Ensemble**
- 📊 **Cluster Averaging**
- 🗳️ **Weighted Voting Classifier**

---

### Level 3 - Final Meta-Ensemble

- 🔼 **Hill Climbing**  
- 🧠 **Ridge Ensemble (Best Performer)**

---

## 🏆 Performance Summary

| Level | Description                  | Score (LogLoss)   |
|-------|------------------------------|-------------------|
| 1     | Base models                  | 1.94 – 1.88       |
| 2     | Intermediate ensembles       | 1.88 – 1.87       |
| 3     | Ridge & Hill Climbing        | **1.86257** (best) |

---

## 🖼️ Model Architecture

![Model_Architecture (1).jpg](https://cdn-uploads.huggingface.co/production/uploads/6863e66ef88659d0f4a3d90d/oscbFPQrWb8-SBLKfnRef.jpeg)

---


📁 Files Included
Swandip_optimal_fertilizer_model.joblib – Trained ensemble model

README.md – This file

architecture.jpg – Visual representation of the architecture


👨‍🔬 Author
Swandip Singha
📬 Kaggle: [@SwandipSingha](https://www.kaggle.com/swandipsingha)

📜 License
MIT License – You are free to use, modify, and distribute this work with proper credit.