# CNN Architecture (Mathematical Formulation)

We consider an input image  

$$
x \in \mathbb{R}^{3 \times 256 \times 256}
$$

with 3 color channels (RGB). Shapes are written as (channels × height × width).

---

## 1. Convolution Block 1 (16 channels)

**Conv1 (3→16, kernel=3, padding=1, stride=1):**

$$
y^{(1)}_{k,i,j}
=\sum_{c=1}^{3}\sum_{u=-1}^{1}\sum_{v=-1}^{1}
W^{(1)}_{k,c,u,v}\; x_{c,\,i+u,\,j+v} + b^{(1)}_{k}
$$

Output:

$$
Y^{(1)} \in \mathbb{R}^{16 \times 256 \times 256}
$$

**BatchNorm1:**

$$
\hat{y}^{(1)}_{k,i,j}=\frac{y^{(1)}_{k,i,j}-\mu_k}{\sqrt{\sigma_k^2+\varepsilon}},\qquad
z^{(1)}_{k,i,j}=\gamma_k \hat{y}^{(1)}_{k,i,j}+\beta_k
$$

**ReLU1:**

$$
a^{(1)}_{k,i,j} = \max(0, z^{(1)}_{k,i,j})
$$

**MaxPool1 (2×2, stride=2):**

$$
p^{(1)}_{k,i,j} = \max_{0 \le u,v < 2} a^{(1)}_{k,\,2i+u,\,2j+v}
$$

Shape: $16 \times 128 \times 128$

---

## 2. Convolution Block 2 (32 channels)

**Conv2 (16→32):**

$$
y^{(2)}_{k,i,j}
=\sum_{c=1}^{16}\sum_{u=-1}^{1}\sum_{v=-1}^{1}
W^{(2)}_{k,c,u,v}\; p^{(1)}_{c,\,i+u,\,j+v} + b^{(2)}_{k}
$$

Then BN2 → ReLU2 → MaxPool2.  

Shape after pooling: $32 \times 64 \times 64$.

---

## 3. Convolution Block 3 (64 channels)

**Conv3 (32→64):**

$$
y^{(3)}_{k,i,j}
=\sum_{c=1}^{32}\sum_{u=-1}^{1}\sum_{v=-1}^{1}
W^{(3)}_{k,c,u,v}\; p^{(2)}_{c,\,i+u,\,j+v} + b^{(3)}_{k}
$$

Then BN3 → ReLU3 → MaxPool3.  

Shape after pooling: $64 \times 32 \times 32$.

---

## 4. Flatten

$$
h = \mathrm{vec}\bigl(p^{(3)}\bigr) \in \mathbb{R}^{64 \cdot 32 \cdot 32} = \mathbb{R}^{65536}
$$

---

## 5. Fully Connected Head

**FC1 (65536→256) + ReLU:**

$$
u_1 = W_1 h + b_1,\quad
a_1 = \max(0, u_1)
$$

**Dropout (p=0.5):**

$$
m \sim \text{Bernoulli}(0.5)^{256},\quad
\tilde{a}_1 = \frac{m \odot a_1}{0.5}
$$

**FC2 (256→64) + ReLU:**

$$
u_2 = W_2 \tilde{a}_1 + b_2,\quad
a_2 = \max(0, u_2)
$$

**FC3 (64→4) logits:**

$$
z = W_3 a_2 + b_3 \in \mathbb{R}^4
$$

---

## 6. Prediction & Loss

**Softmax (conceptual):**

$$
p_c = \frac{e^{z_c}}{\sum_{j=1}^4 e^{z_j}}, \quad c = 1,\dots,4
$$

**Predicted class:**

$$
\hat{y} = \arg\max_c z_c
$$

**Cross-Entropy Loss** for true class $y$:

$$
\mathcal{L}(z,y) = -\log p_y
= -z_y + \log\!\Bigl(\sum_{j=1}^{4} e^{z_j}\Bigr)
$$

---

## 7. Why These Pieces Matter

- **Deeper convs (16→32→64):** extract features hierarchically (edges → textures → scenes).  
- **ReLU:** avoids vanishing gradients, speeds training.  
- **BatchNorm:** normalizes activations, stabilizes training, regularizes.  
- **MaxPool:** adds translation invariance, reduces computation.  
- **Dropout:** prevents overfitting by randomly dropping neurons.  
- **FC head:** compresses learned features into logits for the 4 classes.  


# CNN Architecture Diagram


```text
Input (3 × 256 × 256)
        │
        ▼
[Conv1: 3→16, 3×3 + BN + ReLU]
        │
        ▼
MaxPool 2×2 → (16 × 128 × 128)
        │
        ▼
[Conv2: 16→32, 3×3 + BN + ReLU]
        │
        ▼
MaxPool 2×2 → (32 × 64 × 64)
        │
        ▼
[Conv3: 32→64, 3×3 + BN + ReLU]
        │
        ▼
MaxPool 2×2 → (64 × 32 × 32)
        │
        ▼
Flatten → 65536
        │
        ▼
[FC1: 65536→256 + ReLU + Dropout]
        │
        ▼
[FC2: 256→64 + ReLU]
        │
        ▼
[FC3: 64→4 logits]
        │
        ▼
Softmax → {Sea, Forest, Urban, Field}