model.md

CNN Architecture (Mathematical Formulation)

We consider an input image

$$x\in {\mathbb{R}}^{3\times 256\times 256}x\; \backslash in\; \backslash mathbb\{R\}^\{3\; \backslash times\; 256\; \backslash times\; 256\}$$

with 3 color channels (RGB). Shapes are written as (channels × height × width).

---

1. Convolution Block 1 (16 channels)

Conv1 (3→16, kernel=3, padding=1, stride=1):

$$y_{k,i,j}^{(1)}=\sum _{c=1}^3\sum _{u=-1}^1\sum _{v=-1}^1{W}_{k,c,u,v}^{(1)}{x}_{c,i+u,j+v}+b_k^{(1)}y^\{(1)\}\_\{k,i,j\}\; =\backslash sum\_\{c=1\}^\{3\}\backslash sum\_\{u=-1\}^\{1\}\backslash sum\_\{v=-1\}^\{1\}\; W^\{(1)\}\_\{k,c,u,v\}\backslash ;\; x\_\{c,\backslash ,i+u,\backslash ,j+v\}\; +\; b^\{(1)\}\_\{k\}$$

Output:

$$Y^{(1)}\in {\mathbb{R}}^{16\times 256\times 256}Y^\{(1)\}\; \backslash in\; \backslash mathbb\{R\}^\{16\; \backslash times\; 256\; \backslash times\; 256\}$$

BatchNorm1:

$${\widehat{y}}_{k,i,j}^{(1)}=\frac{y_{k,i,j}^{(1)}-{\mu }_k}{\sqrt{{\sigma }_k^2+\epsilon }},z_{k,i,j}^{(1)}={\gamma }_k{\widehat{y}}_{k,i,j}^{(1)}+{\beta }_k\backslash hat\{y\}^\{(1)\}\_\{k,i,j\}=\backslash frac\{y^\{(1)\}\_\{k,i,j\}-\backslash mu\_k\}\{\backslash sqrt\{\backslash sigma\_k^2+\backslash varepsilon\}\},\backslash qquad\; z^\{(1)\}\_\{k,i,j\}=\backslash gamma\_k\; \backslash hat\{y\}^\{(1)\}\_\{k,i,j\}+\backslash beta\_k$$

ReLU1:

$$a_{k,i,j}^{(1)}=\max (0,z_{k,i,j}^{(1)})a^\{(1)\}\_\{k,i,j\}\; =\; \backslash max(0,\; z^\{(1)\}\_\{k,i,j\})$$

MaxPool1 (2×2, stride=2):

Shape: $16 \times 128 \times 128$

---

2. Convolution Block 2 (32 channels)

Conv2 (16→32):

$$y_{k,i,j}^{(2)}=\sum _{c=1}^{16}\sum _{u=-1}^1\sum _{v=-1}^1{W}_{k,c,u,v}^{(2)}{p}_{c,i+u,j+v}^{(1)}+b_k^{(2)}y^\{(2)\}\_\{k,i,j\}\; =\backslash sum\_\{c=1\}^\{16\}\backslash sum\_\{u=-1\}^\{1\}\backslash sum\_\{v=-1\}^\{1\}\; W^\{(2)\}\_\{k,c,u,v\}\backslash ;\; p^\{(1)\}\_\{c,\backslash ,i+u,\backslash ,j+v\}\; +\; b^\{(2)\}\_\{k\}$$

Then BN2 → ReLU2 → MaxPool2.

Shape after pooling: $32 \times 64 \times 64$.

---

3. Convolution Block 3 (64 channels)

Conv3 (32→64):

$$y_{k,i,j}^{(3)}=\sum _{c=1}^{32}\sum _{u=-1}^1\sum _{v=-1}^1{W}_{k,c,u,v}^{(3)}{p}_{c,i+u,j+v}^{(2)}+b_k^{(3)}y^\{(3)\}\_\{k,i,j\}\; =\backslash sum\_\{c=1\}^\{32\}\backslash sum\_\{u=-1\}^\{1\}\backslash sum\_\{v=-1\}^\{1\}\; W^\{(3)\}\_\{k,c,u,v\}\backslash ;\; p^\{(2)\}\_\{c,\backslash ,i+u,\backslash ,j+v\}\; +\; b^\{(3)\}\_\{k\}$$

Then BN3 → ReLU3 → MaxPool3.

Shape after pooling: $64 \times 32 \times 32$.

---

4. Flatten

$$h=\mathrm{v}\mathrm{e}\mathrm{c}\left(p^{(3)}\right)\in {\mathbb{R}}^{64\cdot 32\cdot 32}={\mathbb{R}}^{65536}h\; =\; \backslash mathrm\{vec\}\backslash bigl(p^\{(3)\}\backslash bigr)\; \backslash in\; \backslash mathbb\{R\}^\{64\; \backslash cdot\; 32\; \backslash cdot\; 32\}\; =\; \backslash mathbb\{R\}^\{65536\}$$

---

5. Fully Connected Head

FC1 (65536→256) + ReLU:

$$u_1=W_{1}h+b_1,a_1=\max (0,u_1)u\_1\; =\; W\_1\; h\; +\; b\_1,\backslash quad\; a\_1\; =\; \backslash max(0,\; u\_1)$$

Dropout (p=0.5):

$$m\sim \text{Bernoulli}(0.5{)}^{256},{\tilde{a}}_1=\frac{m\odot a_1}{0.5}m\; \backslash sim\; \backslash text\{Bernoulli\}(0.5)^\{256\},\backslash quad\; \backslash tilde\{a\}\_1\; =\; \backslash frac\{m\; \backslash odot\; a\_1\}\{0.5\}$$

FC2 (256→64) + ReLU:

$$u_2=W_2{\tilde{a}}_1+b_2,a_2=\max (0,u_2)u\_2\; =\; W\_2\; \backslash tilde\{a\}\_1\; +\; b\_2,\backslash quad\; a\_2\; =\; \backslash max(0,\; u\_2)$$

FC3 (64→4) logits:

$$z=W_3{a}_2+b_3\in {\mathbb{R}}^{4}z\; =\; W\_3\; a\_2\; +\; b\_3\; \backslash in\; \backslash mathbb\{R\}^4$$

---

6. Prediction & Loss

Softmax (conceptual):

$$p_c=\frac{e^{z_c}}{\sum _{j=1}^4{e}^{z_j}},c=1,\dots ,4p\_c\; =\; \backslash frac\{e^\{z\_c\}\}\{\backslash sum\_\{j=1\}^4\; e^\{z\_j\}\},\; \backslash quad\; c\; =\; 1,\backslash dots,4$$

Predicted class:

$$\widehat{y}=\arg \underset{c}{\max }{z}_c\backslash hat\{y\}\; =\; \backslash arg\backslash max\_c\; z\_c$$

Cross-Entropy Loss for true class $y$:

$$\mathcal{L}(z,y)=-\log p_y=-z_y+\log \text{\hspace{0.17em}⁣}\left(\sum _{j=1}^4{e}^{z_j}\right)\backslash mathcal\{L\}(z,y)\; =\; -\backslash log\; p\_y\; =\; -z\_y\; +\; \backslash log\backslash !\backslash Bigl(\backslash sum\_\{j=1\}^\{4\}\; e^\{z\_j\}\backslash Bigr)$$

---

7. Why These Pieces Matter

• Deeper convs (16→32→64): extract features hierarchically (edges → textures → scenes).
• ReLU: avoids vanishing gradients, speeds training.
• BatchNorm: normalizes activations, stabilizes training, regularizes.
• MaxPool: adds translation invariance, reduces computation.
• Dropout: prevents overfitting by randomly dropping neurons.
• FC head: compresses learned features into logits for the 4 classes.

CNN Architecture Diagram

Input (3 × 256 × 256)
        │
        ▼
[Conv1: 3→16, 3×3 + BN + ReLU]
        │
        ▼
MaxPool 2×2 → (16 × 128 × 128)
        │
        ▼
[Conv2: 16→32, 3×3 + BN + ReLU]
        │
        ▼
MaxPool 2×2 → (32 × 64 × 64)
        │
        ▼
[Conv3: 32→64, 3×3 + BN + ReLU]
        │
        ▼
MaxPool 2×2 → (64 × 32 × 32)
        │
        ▼
Flatten → 65536
        │
        ▼
[FC1: 65536→256 + ReLU + Dropout]
        │
        ▼
[FC2: 256→64 + ReLU]
        │
        ▼
[FC3: 64→4 logits]
        │
        ▼
Softmax → {Sea, Forest, Urban, Field}