VADRK155 commited on
Commit
a1c6add
·
verified ·
1 Parent(s): f1890d3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -1,3 +1,39 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language:
4
+ - en
5
+ tags:
6
+ - text-generation
7
+ - gpt
8
+ - from-scratch
9
+ - conversational
10
  ---
11
+
12
+ # Cortex 2 chat preview
13
+
14
+ A small GPT model for conversational chat, trained completely from scratch (custom PyTorch architecture, no `transformers` library involved).
15
+
16
+ > ⚠️ **Note:** this is a **preview** release and may communicate worse than [Cortex 1 Chat](https://huggingface.co/VADRK155/Cortex-1-Chat).
17
+
18
+ - **Parameters**: 346m
19
+ - **Weights format**: fp16
20
+ - **Architecture**: Decoder-only Transformer (custom implementation)
21
+ - **Training**: from scratch, no pretrained weights used
22
+
23
+ > ⚠️ **Note:** the context window is only **256 tokens**, so the model will forget earlier parts of the conversation quickly.
24
+
25
+ ## How to run
26
+
27
+ 2. Install dependencies:
28
+
29
+ pip install -r requirements.txt
30
+
31
+
32
+ 3. Run:
33
+
34
+ python chat.py
35
+
36
+
37
+ ## Limitations
38
+
39
+ This is a small model, so it may lose coherence, mix up facts, or cut off mid-thought — expected behavior for this parameter count.