Instructions to use DopeorNope/COKAL_merged_test-v1-13B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DopeorNope/COKAL_merged_test-v1-13B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DopeorNope/COKAL_merged_test-v1-13B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DopeorNope/COKAL_merged_test-v1-13B")
model = AutoModelForCausalLM.from_pretrained("DopeorNope/COKAL_merged_test-v1-13B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DopeorNope/COKAL_merged_test-v1-13B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DopeorNope/COKAL_merged_test-v1-13B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DopeorNope/COKAL_merged_test-v1-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DopeorNope/COKAL_merged_test-v1-13B

SGLang

How to use DopeorNope/COKAL_merged_test-v1-13B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DopeorNope/COKAL_merged_test-v1-13B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DopeorNope/COKAL_merged_test-v1-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DopeorNope/COKAL_merged_test-v1-13B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DopeorNope/COKAL_merged_test-v1-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DopeorNope/COKAL_merged_test-v1-13B with Docker Model Runner:
```
docker model run hf.co/DopeorNope/COKAL_merged_test-v1-13B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

The license is cc-by-nc-sa-4.0.

(주)미디어그룹사람과숲과 (주)마커의 LLM 연구 컨소시엄으로 개발된 모델입니다

🐻‍❄️COKAL_merged_test-v1-13B🐻‍❄️

Model Details

Model Developers Seungyoo Lee(DopeorNope)

Input Models input text only.

Output Models generate text only.

Model Architecture
COKAL_merged_test-v1-13B is an auto-regressive language model based on the LLaMA2 transformer architecture.

Base Model

HumanF-MarkrAI/COKAL-DPO-13b-v2

MarkrAI/DopeorNope-maestro-v2-DPO-13b

Implemented Method

I utilized slerp merge to smoothly blend the gradients of the base models to create it.

The merging approach relies on some luck, but at the same time, if I have an accurate understanding of my model's performance, I can carefully select models that excel in each aspect to develop a well-balanced model.

Thanks to maywell for sharing useful tips related to the merge method.

Model Benchmark

KO-LLM leaderboard

Follow up as Open KO-LLM LeaderBoard.

Model	Average	Ko-ARC	Ko-HellaSwag	Ko-MMLU	Ko-TruthfulQA	Ko-CommonGen V2
COKAL_merged_test-v1-13B🐻‍❄️	52.72	51.45	60.55	44.8	49.05	57.73
COKAL-DPO-13b-v2🐻‍❄️	52.69	54.95	63.02	43.98	51.67	49.82
COKAL-DPO_test-v2-13b🐻‍❄️	52.67	55.63	63.5	43.49	51.5	49.23
hyeogi/Yi-6b-dpo-v0.2	52.63	41.72	52.96	46.69	52.38	69.42
DopeorNope-maestro-v2-DPO-13b🐻‍❄️	49.42	45.14	56.69	41.37	42.26	61.63

Implementation Code

Load model


from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "DopeorNope/COKAL_merged_test-v1-13B"
OpenOrca = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)

Prompt (Alpaca format)


prompt= f"아래는 문제를 설명하는 지시사항과, 구체적인 답변을 방식을 요구하는 입력이 함께 있는 문장입니다. 이 요청에 대해 적절하게 답변해주세요.\n\n### 지시사항:\n{instruction}\n\n### 입력:\n{input}\n\n### 답변:\n"

prompt_no_input = f"아래는 문제를 설명하는 지시사항입니다. 이 요청에 대해 적절하게 답변해주세요.\n\n### 지시사항:\n{instruction}\n\n### 답변:\n"

Acknowledgement

이 모델은 과학기술정보통신부·광주광역시가 공동 지원한 '인공지능 중심 산업융합 집적단지 조성사업'으로 지원을 받아 수행된 연구 결과입니다.
This model was supported by Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT(MSIT, Korea)&Gwangju Metropolitan City.

Downloads last month: 10

Safetensors

Model size

13B params

Tensor type

F32

Model tree for DopeorNope/COKAL_merged_test-v1-13B

Quantizations

1 model

Collection including DopeorNope/COKAL_merged_test-v1-13B

Developed best Ko-LLM Models

Collection

These are the top models I developed on the open Ko-llm leaderboard. • 8 items • Updated Dec 24, 2023 • 2