Upload README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3-4B
|
| 4 |
+
datasets:
|
| 5 |
+
- metehan777/global-seo-knowledge
|
| 6 |
+
tags:
|
| 7 |
+
- seo
|
| 8 |
+
- fine-tuned
|
| 9 |
+
- qwen3
|
| 10 |
+
- lora
|
| 11 |
+
- polish
|
| 12 |
+
- bilingual
|
| 13 |
+
- domain-specific
|
| 14 |
+
- experimental
|
| 15 |
+
language:
|
| 16 |
+
- pl
|
| 17 |
+
- en
|
| 18 |
+
pipeline_tag: text-generation
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# Qwen3-4B SEO uczciweseo.pl (Experimental)
|
| 22 |
+
|
| 23 |
+
Domain-specific fine-tuned version of [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) for the [uczciweseo.pl](https://uczciweseo.pl) brand.
|
| 24 |
+
|
| 25 |
+
**Status**: Experimental. Training completed with low loss but generation quality degraded on Polish domain questions. English SEO knowledge preserved.
|
| 26 |
+
|
| 27 |
+
## Training Details
|
| 28 |
+
|
| 29 |
+
- **Base model**: Qwen3-4B (3.09B params)
|
| 30 |
+
- **Method**: LoRA (r=16, alpha=32) on all linear layers
|
| 31 |
+
- **Trainable params**: 33,030,144 (0.81%)
|
| 32 |
+
- **Dataset**: 2,312 examples (925 domain 5x-oversampled + 1,387 bilingual SEO)
|
| 33 |
+
- **Epochs**: 2
|
| 34 |
+
- **Learning rate**: 2e-5 (cosine scheduler)
|
| 35 |
+
- **Training loss**: 0.4322
|
| 36 |
+
- **Best eval loss**: 1.014
|
| 37 |
+
- **Hardware**: Apple Silicon MPS (24GB), fp16 LoRA
|
| 38 |
+
|
| 39 |
+
## Brand Knowledge Target
|
| 40 |
+
|
| 41 |
+
Training data covers uczciweseo.pl (EXELMEDIA sp. z o.o.):
|
| 42 |
+
- Company values: no long-term contracts, full transparency
|
| 43 |
+
- Services: SEO, Google Ads, Bing Ads, AI SEO, CRO, automation
|
| 44 |
+
- Industry experience: construction, legal, industrial, automotive, furniture, e-commerce
|
| 45 |
+
|
| 46 |
+
## Known Issues
|
| 47 |
+
|
| 48 |
+
- Polish domain answers show quality degradation (URL-like artifacts)
|
| 49 |
+
- English general SEO knowledge well preserved
|
| 50 |
+
- Recommended for research/experimentation only
|
| 51 |
+
|
| 52 |
+
## Usage
|
| 53 |
+
|
| 54 |
+
```python
|
| 55 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 56 |
+
|
| 57 |
+
model = AutoModelForCausalLM.from_pretrained("Kelnux/Qwen3-4B-seo-uczciweseo")
|
| 58 |
+
tokenizer = AutoTokenizer.from_pretrained("Kelnux/Qwen3-4B-seo-uczciweseo")
|
| 59 |
+
```
|