Kelnux commited on
Commit
563a5bb
·
verified ·
1 Parent(s): d338195

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +59 -0
README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen3-4B
4
+ datasets:
5
+ - metehan777/global-seo-knowledge
6
+ tags:
7
+ - seo
8
+ - fine-tuned
9
+ - qwen3
10
+ - lora
11
+ - polish
12
+ - bilingual
13
+ - domain-specific
14
+ - experimental
15
+ language:
16
+ - pl
17
+ - en
18
+ pipeline_tag: text-generation
19
+ ---
20
+
21
+ # Qwen3-4B SEO uczciweseo.pl (Experimental)
22
+
23
+ Domain-specific fine-tuned version of [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) for the [uczciweseo.pl](https://uczciweseo.pl) brand.
24
+
25
+ **Status**: Experimental. Training completed with low loss but generation quality degraded on Polish domain questions. English SEO knowledge preserved.
26
+
27
+ ## Training Details
28
+
29
+ - **Base model**: Qwen3-4B (3.09B params)
30
+ - **Method**: LoRA (r=16, alpha=32) on all linear layers
31
+ - **Trainable params**: 33,030,144 (0.81%)
32
+ - **Dataset**: 2,312 examples (925 domain 5x-oversampled + 1,387 bilingual SEO)
33
+ - **Epochs**: 2
34
+ - **Learning rate**: 2e-5 (cosine scheduler)
35
+ - **Training loss**: 0.4322
36
+ - **Best eval loss**: 1.014
37
+ - **Hardware**: Apple Silicon MPS (24GB), fp16 LoRA
38
+
39
+ ## Brand Knowledge Target
40
+
41
+ Training data covers uczciweseo.pl (EXELMEDIA sp. z o.o.):
42
+ - Company values: no long-term contracts, full transparency
43
+ - Services: SEO, Google Ads, Bing Ads, AI SEO, CRO, automation
44
+ - Industry experience: construction, legal, industrial, automotive, furniture, e-commerce
45
+
46
+ ## Known Issues
47
+
48
+ - Polish domain answers show quality degradation (URL-like artifacts)
49
+ - English general SEO knowledge well preserved
50
+ - Recommended for research/experimentation only
51
+
52
+ ## Usage
53
+
54
+ ```python
55
+ from transformers import AutoModelForCausalLM, AutoTokenizer
56
+
57
+ model = AutoModelForCausalLM.from_pretrained("Kelnux/Qwen3-4B-seo-uczciweseo")
58
+ tokenizer = AutoTokenizer.from_pretrained("Kelnux/Qwen3-4B-seo-uczciweseo")
59
+ ```