Arush kumar commited on
Commit
470e2bf
·
1 Parent(s): fd6ac4c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +54 -8
README.md CHANGED
@@ -1,11 +1,57 @@
1
  ---
2
- title: Veylon
3
- emoji: 💻
4
- colorFrom: pink
5
- colorTo: red
6
- sdk: docker
 
 
7
  pinned: false
8
- short_description: An very lightweight LLM
9
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
1
  ---
2
+ title: Veylon Alpha 1 Preview
3
+ emoji: 🚀
4
+ colorFrom: blue
5
+ colorTo: purple
6
+ sdk: gradio
7
+ sdk_version: 5.0.0
8
+ app_file: app.py
9
  pinned: false
10
+ tags:
11
+ - llm
12
+ - language-model
13
+ - keras
14
+ - jax
15
+ - transformer
16
+ - ai
17
+ - text-generation
18
+ --- Veylon Alpha 1 Preview
19
+
20
+ Veylon Alpha 1 Preview is a lightweight experimental language model designed for efficient training and fast inference.
21
+
22
+ Highlights
23
+
24
+ - 7M parameter prototype
25
+ - Trained with Keras 3 + JAX
26
+ - Optimized for TPU v5e training
27
+ - Sliding Window Attention (SWA)
28
+ - Grouped Query Attention (GQA)
29
+ - Fast training throughput
30
+ - Research-focused architecture
31
+
32
+ Performance
33
+
34
+ - Context Length: 1024 tokens
35
+ - Vocabulary Size: 8000
36
+ - TPU Throughput: 200K+ tokens/sec during training
37
+ - Lightweight checkpoint size
38
+
39
+ Try It
40
+
41
+ Type a prompt and generate text directly in the demo.
42
+
43
+ Roadmap
44
+
45
+ - Veylon Alpha 10M
46
+ - Veylon Alpha 30M
47
+ - Veylon Alpha 100M
48
+ - Advanced memory systems
49
+ - Longer context support
50
+
51
+ Author
52
+
53
+ Created by IconicDev.
54
+
55
+ Disclaimer
56
 
57
+ This is a research preview and may generate inaccurate or nonsensical outputs.