Instructions to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("finding1/LongCat-Flash-Chat-MLX-5.5bpw") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Transformers
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="finding1/LongCat-Flash-Chat-MLX-5.5bpw", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("finding1/LongCat-Flash-Chat-MLX-5.5bpw", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("finding1/LongCat-Flash-Chat-MLX-5.5bpw", trust_remote_code=True, device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- vLLM
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "finding1/LongCat-Flash-Chat-MLX-5.5bpw" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "finding1/LongCat-Flash-Chat-MLX-5.5bpw", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/finding1/LongCat-Flash-Chat-MLX-5.5bpw
- SGLang
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "finding1/LongCat-Flash-Chat-MLX-5.5bpw" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "finding1/LongCat-Flash-Chat-MLX-5.5bpw", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "finding1/LongCat-Flash-Chat-MLX-5.5bpw" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "finding1/LongCat-Flash-Chat-MLX-5.5bpw", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Pi
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "finding1/LongCat-Flash-Chat-MLX-5.5bpw"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "finding1/LongCat-Flash-Chat-MLX-5.5bpw" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "finding1/LongCat-Flash-Chat-MLX-5.5bpw"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "finding1/LongCat-Flash-Chat-MLX-5.5bpw" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "finding1/LongCat-Flash-Chat-MLX-5.5bpw", "messages": [ {"role": "user", "content": "Hello"} ] }' - Docker Model Runner
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with Docker Model Runner:
docker model run hf.co/finding1/LongCat-Flash-Chat-MLX-5.5bpw
- Hermes Agent
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "finding1/LongCat-Flash-Chat-MLX-5.5bpw"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default finding1/LongCat-Flash-Chat-MLX-5.5bpw
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use finding1/LongCat-Flash-Chat-MLX-5.5bpw with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "finding1/LongCat-Flash-Chat-MLX-5.5bpw"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "finding1/LongCat-Flash-Chat-MLX-5.5bpw" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
| {%- set tool_choice = tool_choice | default('auto') %} | |
| {%- set ns = namespace(rounds = 0, tool_types = [], last_query_index = -1) %} | |
| {%- if tools and tool_choice != 'none' %} | |
| {{- "# Tools | |
| " }} | |
| {{- "You have access to the following tools: | |
| " }} | |
| {%- for tool in tools %} | |
| {%- if tool.type in ['code_interpreter', 'function'] %} | |
| {%- if tool.type not in ns.tool_types %} | |
| {%- set ns.tool_types = ns.tool_types + [tool.type] %} | |
| {{- "## Tool namespace: " ~ tool.type ~ " | |
| " }} | |
| {%- endif %} | |
| {%- if tool.type == 'code_interpreter' %} | |
| {%- set tool = {"type":"code_interpreter","function":{"name":"code_interpreter_preview","description":"The code will be executed in a stateful Jupyter notebook sandbox environment, only supports local computation, data processing, and file operations. | |
| Code sandbox environment (network isolated) Any external network requests or online API calls are prohibited. | |
| If online functionality is needed, please use other permitted tools. | |
| Code will respond with the output of the execution or time out after 60.0 seconds. ","parameters":{"type":"object","properties":{"language":{"type":"string","description":"The programming language of the code to be executed. Available values: python (Default), java, go, js, ts, c, c++."},"code":{"type":"string","description":"Python code to be executed must not include the following: | |
| - Importing network libraries such as requests, httplib, etc. | |
| - Any form of HTTP requests. | |
| - External API calls. | |
| - Network port operations. Example: ```python | |
| import pandas as pd | |
| pd.DataFrame({'A':[1,2]}) | |
| ```"},"timeout":{"type":"number","description":"The maximum execution time of the code, in seconds. Default is 60.0."}}},"required":["code"]}} %} | |
| {%- endif %} | |
| {{- "### Tool name: " + tool.function.name + " | |
| " }} | |
| {{- "Description: " + tool.function.description + " | |
| " }} | |
| {{- "InputSchema: | |
| " + tool.function.parameters | tojson(indent=2) + " | |
| " }} | |
| {%- endif %} | |
| {%- endfor %} | |
| {{- '**Note**: For each function call, return a json object with function name and arguments within <longcat_tool_call></longcat_tool_call> XML tags as follows: | |
| <longcat_tool_call> | |
| {"name": <function-name>, "arguments": <args-dict>} | |
| </longcat_tool_call> | |
| ' }} | |
| {{- 'When multiple functions need to be called simultaneously, each function call should be wrapped in its own <longcat_tool_call> tag and placed consecutively. For example: | |
| <longcat_tool_call> | |
| {"name": <function-name>, "arguments": <args-dict>} | |
| </longcat_tool_call><longcat_tool_call> | |
| {"name": <function-name>, "arguments": <args-dict>} | |
| </longcat_tool_call> | |
| ' }} | |
| {{- "# Messages | |
| " }} | |
| {%- for idx in range(messages|length - 1) %} | |
| {%- set msg = messages[idx] %} | |
| {%- if msg.role == 'assistant' and not msg.tool_calls %} | |
| {%- set ns.last_query_index = idx %} | |
| {%- endif %} | |
| {%- endfor%} | |
| {%- endif %} | |
| {%- for msg in messages %} | |
| {%- if msg.role == "system" %} | |
| {{- "SYSTEM:" + msg.content }} | |
| {%- elif msg.role == "user" %} | |
| {%- if loop.first %} | |
| {{- "[Round " ~ (ns.rounds) ~ "] USER:" }} | |
| {%- else %} | |
| {{- " [Round " ~ (ns.rounds) ~ "] USER:"}} | |
| {%- endif %} | |
| {%- set ns.rounds = ns.rounds + 1 %} | |
| {%- if msg["files"] %} | |
| {{- '<longcat_files> | |
| ' ~ msg.files | tojson(indent=2) ~ ' | |
| </longcat_files>' }} | |
| {%- endif %} | |
| {{- msg.content }} | |
| {%- elif msg.role == "assistant" %} | |
| {{- " ASSISTANT:" }} | |
| {%- if enable_thinking == true and msg.reasoning_content and ns.tool_types != [] and loop.index0 > ns.last_query_index %} | |
| {{- " | |
| <longcat_think> | |
| " ~ msg.reasoning_content ~ " | |
| </longcat_think> | |
| " }} | |
| {%- endif %} | |
| {%- if msg.content%} | |
| {{- msg.content }} | |
| {%- endif %} | |
| {%- if msg.tool_calls %} | |
| {%- for tool_call in msg.tool_calls -%} | |
| {{- "<longcat_tool_call> | |
| " -}} | |
| {%- if tool_call.function.arguments is string -%} | |
| {"name": "{{ tool_call.function.name}}", "arguments": {{tool_call.function.arguments}}} | |
| {%- else -%} | |
| {"name": "{{ tool_call.function.name}}", "arguments": {{tool_call.function.arguments | tojson}}} | |
| {%- endif -%} | |
| {{- " | |
| </longcat_tool_call>" }} | |
| {%- endfor %} | |
| {%- endif %} | |
| {{- "</longcat_s>" -}} | |
| {%- elif msg.role == "tool" %} | |
| {{- " TOOL:" -}} | |
| {%- if msg.name -%} | |
| {"name": {{msg.name | tojson}}, "content": {{msg.content | tojson}}} | |
| {%- else -%} | |
| {"content": {{msg.content | tojson}}} | |
| {%- endif -%} | |
| {%- endif %} | |
| {%- endfor %} | |
| {%- if add_generation_prompt %} | |
| {%- if enable_thinking == true %} | |
| {{- " /think_on" }} | |
| {%- if thinking_budget %} | |
| {%- if thinking_budget < 1024 %} | |
| {%- set thinking_budget = 1024 %} | |
| {%- endif%} | |
| {{- " | |
| thinking_budget: < " ~ thinking_budget ~ "."}} | |
| {%- endif %} | |
| {{- " ASSISTANT:<longcat_think> | |
| "}} | |
| {%- elif enable_thinking == false %} | |
| {{- " /think_off ASSISTANT:<longcat_think> | |
| </longcat_think> | |
| " }} | |
| {%- else %} | |
| {{- " ASSISTANT:" }} | |
| {%- endif %} | |
| {%- endif %} |