Text Generation
Transformers
Safetensors
Russian
English
alice_ai
custom_code
mixture-of-experts
vllm
Instructions to use yandex/AliceAI-Foundation-80B-A3B-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use yandex/AliceAI-Foundation-80B-A3B-Base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="yandex/AliceAI-Foundation-80B-A3B-Base", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("yandex/AliceAI-Foundation-80B-A3B-Base", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use yandex/AliceAI-Foundation-80B-A3B-Base with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "yandex/AliceAI-Foundation-80B-A3B-Base" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "yandex/AliceAI-Foundation-80B-A3B-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/yandex/AliceAI-Foundation-80B-A3B-Base
- SGLang
How to use yandex/AliceAI-Foundation-80B-A3B-Base with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "yandex/AliceAI-Foundation-80B-A3B-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "yandex/AliceAI-Foundation-80B-A3B-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "yandex/AliceAI-Foundation-80B-A3B-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "yandex/AliceAI-Foundation-80B-A3B-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use yandex/AliceAI-Foundation-80B-A3B-Base with Docker Model Runner:
docker model run hf.co/yandex/AliceAI-Foundation-80B-A3B-Base
Download finetune/chat_template.jinja from yandex/AliceAI-Foundation-80B-A3B-Base: direct link, hf CLI and curl.
- Browser
- Download file 3.63 kB
-
https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base/resolve/main/finetune/chat_template.jinja
- Command line
-
hf download hf://yandex/AliceAI-Foundation-80B-A3B-Base/finetune/chat_template.jinja
-
curl -L -o chat_template.jinja https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base/resolve/main/finetune/chat_template.jinja
3.63 kB
| {%- set roles = {"assistant": "assistant:", "user": "user:", "system": "system:", "meta": "meta:"} -%} | |
| {%- set TOOL_CALL_START = "[TOOL_CALL_START]" -%} | |
| {%- set COT_START = "[COT_START]" -%} | |
| {%- set COT_END = "[COT_END]" -%} | |
| {%- set tools_prefix = "Тебе доступны следующие функции:" -%} | |
| {%- set json_seps = (",", ":") -%} | |
| {%- macro js(x) -%} | |
| {{- x | tojson(separators=json_seps) | replace("&", "\\u0026") | replace("<", "\\u003c") | replace(">", "\\u003e") | replace("'", "\\u0027") | replace("/", "\\/") -}} | |
| {%- endmacro -%} | |
| {%- macro tool_definition(tool) -%} | |
| {%- if tool.function is defined -%} | |
| {%- set tool = tool.function -%} | |
| {%- endif -%} | |
| {{- 'function {"name":"' ~ tool.name ~ '",' -}} | |
| {%- if tool.description is defined and tool.description -%} | |
| {{- '"description":"' ~ tool.description ~ '",' -}} | |
| {%- endif -%} | |
| {{- '"parameters":' ~ js(tool.parameters) -}} | |
| {%- if tool.return_parameters is defined and tool.return_parameters -%} | |
| {{- ',"return_parameters":' ~ js(tool.return_parameters) -}} | |
| {%- endif -%} | |
| {%- if tool.few_shot_examples is defined and tool.few_shot_examples -%} | |
| {{- ',"few_shot_examples":' ~ js(tool.few_shot_examples) -}} | |
| {%- endif -%} | |
| {{- "}" -}} | |
| {%- endmacro -%} | |
| {%- macro render_tools(tools) -%} | |
| {{- tools_prefix -}} | |
| {%- for tool in tools -%} | |
| {{- "\n" ~ tool_definition(tool) -}} | |
| {%- endfor -%} | |
| {%- endmacro -%} | |
| {%- macro render_tool_calls(calls) -%} | |
| {%- for call in calls -%} | |
| {%- set fn = call.function if call.function is defined else call -%} | |
| {{- "\n" ~ TOOL_CALL_START ~ fn.name ~ "\n" -}} | |
| {%- if fn.arguments is mapping -%} | |
| {{- js(fn.arguments) -}} | |
| {%- else -%} | |
| {{- fn.arguments -}} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- endmacro -%} | |
| {%- macro render_assistant(message) -%} | |
| {%- if message.tool_calls is defined and message.tool_calls -%} | |
| {%- set content = render_tool_calls(message.tool_calls) -%} | |
| {%- elif message.content is defined and message.content is not none -%} | |
| {%- set content = message.content -%} | |
| {%- else -%} | |
| {%- set content = "" -%} | |
| {%- endif -%} | |
| {{- roles.assistant -}} | |
| {%- if message.reasoning_content is defined and message.reasoning_content -%} | |
| {{- COT_START ~ message.reasoning_content ~ COT_END -}} | |
| {%- endif -%} | |
| {{- " " ~ content -}} | |
| {%- endmacro -%} | |
| {%- macro render_message(message) -%} | |
| {%- if message.role == "system" -%} | |
| {{- roles.system ~ " " ~ message.content -}} | |
| {%- elif message.role == "user" -%} | |
| {{- roles.user ~ " " ~ message.content -}} | |
| {%- elif message.role == "assistant" -%} | |
| {{- render_assistant(message) -}} | |
| {%- elif message.role == "tool" -%} | |
| {{- "Tool " ~ message.name ~ ": " ~ (message.content | default("", true)) -}} | |
| {%- elif message.role == "meta" -%} | |
| {{- roles.meta ~ message.content -}} | |
| {%- endif -%} | |
| {%- endmacro -%} | |
| {{- bos_token | default("", true) -}} | |
| {%- if tools is defined and tools -%} | |
| {{- render_tools(tools) -}} | |
| {%- endif -%} | |
| {%- for message in messages -%} | |
| {%- set sep = "\n\n " if (tools is defined and tools) or not loop.first else "" -%} | |
| {%- set block = render_message(message) -%} | |
| {%- if loop.last and not (add_generation_prompt | default(false)) -%} | |
| {%- set block = block | trim -%} | |
| {%- endif -%} | |
| {%- if block -%} | |
| {{- sep ~ block -}} | |
| {%- endif -%} | |
| {%- endfor -%} | |
| {%- if add_generation_prompt | default(false) -%} | |
| {{- "\n\n " ~ roles.assistant -}} | |
| {%- endif -%} | |