Update README.md

README.md

@@ -1,3 +1,109 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+- en
+library_name: transformers
+tags:
+- LLM
+- CasualLLM
+- StoryLLM
+- SLM
+- NOVA
+---
+# Nova-Casual-LLM
+
+A lightweight and efficient transformer-based language model built with custom architecture and integrated seamlessly into the Hugging Face ecosystem.
+
+## Model Summary
+
+- **Model Name**: NovaForCausalLM
+- **Architecture**: Custom decoder-only transformer (`NOVA`)
+- **Model Type**: `nova`
+- **Use Case**: Causal Language Modeling (text generation, auto-completion)
+- **Parameters**: 14,412,400 trainable parameters
+- **Pretrained Tokenizer**: `PreTrainedTokenizerFast`
+- **Framework**: PyTorch
+- **Hugging Face Integration**: Compatible with `transformers` via custom `AutoModel` and `AutoConfig` registration.
+
+---
+
+## Files Included
+
+| File                       | Description                                       |
+|---------------------------|----------------------------------------------------|
+| `config.json`             | Configuration of model hyperparameters             |
+| `model.safetensors`       | Serialized model weights (efficient format)        |
+| `nova_modelling.py`       | Custom model and config class definitions          |
+| `tokenizer.json`          | Serialized tokenizer                               |
+| `tokenizer_config.json`   | Tokenizer configuration metadata                   |
+| `special_tokens_map.json` | Mapping for special tokens (e.g., BOS, EOS)        |
+| `README.md`               | Model card (you’re reading it!)                    |
+
+---
+
+## Model Architecture
+
+### `NovaForCausalLM`
+
+The model consists of:
+- Embedding layers: token + positional
+- Stack of transformer decoder blocks
+- Multi-head attention with 640 individual heads
+- Layer normalization
+- Final linear head for vocabulary logits
+
+### Configuration (`NovaConfig`)
+
+```json
+{
+  "model_type": "nova",
+  "vocab_size": 6000,
+  "block_size": 256,
+  "n_embd": 640,
+  "n_layer": 4,
+  "n_head": 8
+}
+```
+
+## 🚀 Usage
+
+### Step 1: Clone the repo (to get the `nova_modelling.py`)
+
+```bash
+git clone https://huggingface.co/harshit36/Nova-Casual-LLM
+cd Nova-Casual-LLM
+```
+
+```python
+import sys
+sys.path.append("./Nova-Casual-LLM/")  # add current dir to path
+
+from transformers import PreTrainedTokenizerFast
+from nova_modelling import NovaConfig, NovaForCausalLM
+
+# Load tokenizer
+tokenizer = PreTrainedTokenizerFast.from_pretrained("harshit36/Nova-Casual-LLM")
+
+# Load config
+config = NovaConfig.from_pretrained("harshit36/Nova-Casual-LLM")
+
+# Instantiate model using your custom class
+model = NovaForCausalLM(config)
+model = model.from_pretrained("harshit36/Nova-Casual-LLM")
+
+# Use the model
+input_ids = tokenizer("Hello world", return_tensors="pt").input_ids
+output = model.generate(input_ids)
+print(tokenizer.decode(output[0], skip_special_tokens=True).replace(" ","").replace("Ġ"," ").replace("Ċ","\n"))
+
+```
+
+## Intended Use
+
+General text generation
+
+Hybrid Positional Encoding Research model (Combination of Sinusoidal and learnable encodings)
+
+Educational demonstrations of custom HF model integration
+
+Rapid prototyping of transformer models