Text Generation
Transformers
Safetensors
mistral
roleplay
creative-writing
Merge
mergekit
exl2
conversational
text-generation-inference
Instructions to use ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6") model = AutoModelForCausalLM.from_pretrained("ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6
- SGLang
How to use ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6 with Docker Model Runner:
docker model run hf.co/ArtusDev/Delta-Vector_Archaeo-12B-V2_EXL2_2.5bpw_H6
| tags: | |
| - roleplay | |
| - creative-writing | |
| - merge | |
| - mergekit | |
| base_model: | |
| - Delta-Vector/Francois-PE-V2-Huali-12B | |
| - Delta-Vector/Rei-V3-KTO-12B | |
| pipeline_tag: text-generation | |
| library_name: transformers | |
| ``` | |
| __~a~_ | |
| ~~; ~_ | |
| _ ~ ~_ _ | |
| '_\;__._._._._._._] ~_._._._._._.__;/_` | |
| '(/'/'/'/'|'|'|'| ( )|'|'|'|'\'\'\'\)' | |
| (/ / / /, | | | |(/ \) | | | ,\ \ \ \) | |
| (/ / / / / | | | ~(/ \) ~ | | \ \ \ \ \) | |
| (/ / / / / ~ ~ ~ (/ \) ~ ~ \ \ \ \ \) | |
| (/ / / / ~ / (||)| ~ \ \ \ \) | |
| ~ / / ~ M /||\M ~ \ \ ~ | |
| ~ ~ /||\ ~ ~ | |
| //||\\ | |
| //||\\ | |
| //||\\ | |
| '/||\' "Archaeopteryx" | |
| ``` | |
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <style> | |
| @import url('https://fonts.googleapis.com/css2?family=VT323&display=swap'); | |
| body { | |
| background: #0a0017; | |
| margin: 0; | |
| padding: 20px; | |
| font-family: 'VT323', monospace; | |
| color: #ff00aa; | |
| text-shadow: 0 0 8px #ff00aa; | |
| animation: glitch-flicker 0.2s infinite alternate; | |
| } | |
| @keyframes glitch-flicker { | |
| 0% { text-shadow: 0 0 5px #ff00aa, 0 0 15px #ff00aa; } | |
| 100% { text-shadow: 0 0 8px #ff0066, 0 0 18px #ff0066; } | |
| } | |
| .crt-container { | |
| padding: 10px; | |
| max-width: 900px; | |
| margin: auto; | |
| } | |
| .crt-case { | |
| background: linear-gradient(135deg, #130021, #20002c); | |
| border-radius: 10px; | |
| padding: 15px; | |
| box-shadow: | |
| inset 2px 2px 10px rgba(255,0,170,0.5), | |
| 2px 2px 5px rgba(255,0,170,0.3), | |
| 0 0 25px rgba(255,0,170,0.2); | |
| } | |
| .crt-screen { | |
| background: #0c011a; | |
| padding: 20px; | |
| border-radius: 10px; | |
| box-shadow: | |
| inset 0 0 25px rgba(255,0,170,0.3), | |
| 0 0 15px rgba(255,0,170,0.7); | |
| filter: contrast(1.2) brightness(1.2); | |
| text-shadow: 0px 0px 5px #ff00aa; | |
| animation: glow-pulse 3s infinite alternate; | |
| } | |
| @keyframes glow-pulse { | |
| 0% { box-shadow: inset 0 0 20px rgba(255,0,170,0.3), 0 0 15px rgba(255,0,170,0.3); } | |
| 100% { box-shadow: inset 0 0 30px rgba(255,0,170,0.5), 0 0 25px rgba(255,0,170,0.5); } | |
| } | |
| h2 { | |
| color: #ff33cc; | |
| text-align: center; | |
| font-size: 28px; | |
| text-shadow: | |
| 0 0 8px #ff33cc, | |
| 0 0 18px #ff0044; | |
| } | |
| pre { | |
| background: rgba(255,0,170,0.1); | |
| padding: 10px; | |
| border-radius: 10px; | |
| color: #ff66cc; | |
| font-size: 14px; | |
| box-shadow: inset 0 0 10px rgba(255,0,170,0.5); | |
| } | |
| .glitch { | |
| animation: text-glitch 0.5s infinite alternate; | |
| } | |
| @keyframes text-glitch { | |
| 0% { transform: translateX(-2px); text-shadow: 0 0 5px #ff0066, 0 0 10px #ff33cc; } | |
| 100% { transform: translateX(2px); text-shadow: 0 0 8px #ff00aa, 0 0 20px #ff0099; } | |
| } | |
| .neon-link { | |
| color: #ff66cc; | |
| text-decoration: none; | |
| transition: text-shadow 0.3s ease; | |
| } | |
| .neon-link:hover { | |
| text-shadow: 0px 0px 15px #ff66cc, 0 0 25px rgba(255,0,170,0.5); | |
| } | |
| .ascii-art { | |
| text-align: center; | |
| font-size: 12px; | |
| color: #ff33cc; | |
| text-shadow: 0px 0px 5px #ff00ff; | |
| margin-bottom: 20px; | |
| } | |
| .quantso-container { | |
| display: flex; | |
| justify-content: center; | |
| gap: 20px; | |
| margin-top: 20px; | |
| } | |
| .quantso-box { | |
| background: rgba(255,0,170,0.1); | |
| padding: 15px; | |
| border-radius: 10px; | |
| text-align: center; | |
| box-shadow: inset 0 0 10px rgba(255,0,170,0.5); | |
| flex: 1; | |
| max-width: 150px; | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="crt-container"> | |
| <div class="crt-case"> | |
| <div class="crt-screen"> | |
| <p>A series of Merges made for Roleplaying & Creative Writing, This model uses Rei-V3-KTO-12B and Francois-PE-V2-Huali-12B and Slerp to merge the 2 models - as a sequel to the OG Archaeo.</p> | |
| <h3>ChatML formatting</h3> | |
| <pre> | |
| """<|im_start|>system | |
| system prompt<|im_end|> | |
| <|im_start|>user | |
| Hi there!<|im_end|> | |
| <|im_start|>assistant | |
| Nice to meet you!<|im_end|> | |
| <|im_start|>user | |
| Can I ask a question?<|im_end|> | |
| <|im_start|>assistant | |
| """ | |
| </pre> | |
| <h3>MergeKit Configuration</h3> | |
| <pre> | |
| models: | |
| - model: Delta-Vector/Rei-V3-KTO-12B | |
| - model: Delta-Vector/Francois-PE-V2-Huali-12B | |
| merge_method: slerp | |
| base_model: Delta-Vector/Rei-V3-KTO-12B | |
| parameters: | |
| t: | |
| - value: 0.2 | |
| dtype: bfloat16 | |
| tokenizer_source: base | |
| </pre> | |
| <h3>Quants:</h3> | |
| <div class="quantso-container"> | |
| <div class="quantso-box"> | |
| <strong>GGUF</strong><br> | |
| <a class="neon-link" href="#">too lazy bwehh (waiting for mradermacher)/</a> | |
| </div> | |
| <div class="quantso-box"> | |
| <strong>EXL2</strong><br> | |
| <a class="neon-link" href="#">nyooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo</a> | |
| </div> | |
| </div> | |
| <h3>Credits</h3> | |
| <p>Thank you to: Kubernetes-bad, LucyKnada, Intervitens, Samantha Twinkman, Tav, Alicat, Auri, Trappu & The rest of Anthracite</p> | |
| </div> | |
| </div> | |
| </div> | |
| </body> | |
| </html> |