Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+library_name: transformers
+license: other
+license_name: lfm1.0
+license_link: LICENSE
+language:
+- en
+- ja
+- ko
+- fr
+- es
+- de
+- ar
+- zh
+pipeline_tag: image-text-to-text
+tags:
+- liquid
+- lfm2
+- lfm2-vl
+- edge
+- lfm2.5-vl
+- lfm2.5
+base_model: LiquidAI/LFM2.5-1.2B-Base
+---
+
+<center>
+<div style="text-align: center;">
+  <img 
+    src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png" 
+    alt="Liquid AI"
+    style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;"
+  />
+</div>
+<div style="display: flex; justify-content: center; gap: 0.5em;">
+  <a href="https://playground.liquid.ai/chat">
+<a href="https://playground.liquid.ai/"><strong>Try LFM</strong></a> • <a href="https://docs.liquid.ai/lfm"><strong>Documentation</strong></a> • <a href="https://leap.liquid.ai/"><strong>LEAP</strong></a></a>
+</div>
+</center>
+
+# LFM2.5‑VL-1.6B
+
+LFM2.5‑VL-1.6B is [Liquid AI](https://www.liquid.ai/)'s refreshed version of the first vision-language model, [LFM2-VL-1.6B](https://huggingface.co/LiquidAI/LFM2-VL-1.6B), built on an updated backbone [LFM2.5-1.2B-Base](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Base) and tuned for stronger real-world performance.
+
+* **Enhanced instruction following** on vision and language tasks.
+* **Improved multilingual vision understanding** in Arabic, Chinese, French, German, Japanese, Korean, and Spanish.
+* **Robust understanding of visual content** with improved results on multi-image inputs, high-resolution images, and OCR.
+
+Find more about LFM2.5 family of models in our [blog post](https://www.liquid.ai/blog/introducing-lfm2-5-the-next-generation-of-on-device-ai).
+
+## 📄 Model details
+
+LFM2.5-VL-1.6B is a general-purpose vision-language model with the following features:
+
+- **LM Backbone**: LFM2.5-1.2B-Base
+- **Vision encoder**: SigLIP2 NaFlex shape‑optimized 400M
+- **Context length**: 32,768 tokens
+- **Vocabulary size**: 65,536
+- **Languages**: English, Arabic, Chinese, French, German, Japanese, Korean, and Spanish
+- **Native resolution processing**: handles images up to 512*512 pixels without upscaling and preserves non-standard aspect ratios without distortion
+- **Tiling strategy**: splits large images into non-overlapping 512×512 patches and includes thumbnail encoding for global context
+- **Inference-time flexibility**: user-tunable maximum image tokens and tile count for speed/quality tradeoff without retraining
+- **Generation parameters**: 
+  - text: `temperature=0.1`, `min_p=0.15`, `repetition_penalty=1.05`
+  - vision: `min_image_tokens=64` `max_image_tokens=256`, `do_image_splitting=True`
+
+We recommend using it for general vision-language workloads, OCR or document comprehension. It’s not well-suited for knowledge-intensive tasks.
+
+### Chat Template
+
+LFM2.5-VL uses a ChatML-like format. See the [Chat Template documentation](https://docs.liquid.ai/lfm/getting-started/vision#chat-template) for details.
+
+```
+<|startoftext|><|im_start|>system
+You are a helpful multimodal assistant by Liquid AI.<|im_end|>
+<|im_start|>user
+<image>Describe this image.<|im_end|>
+<|im_start|>assistant
+This image shows a Caenorhabditis elegans (C. elegans) nematode.<|im_end|>
+```
+
+You can use [`processor.apply_chat_template()`](https://huggingface.co/docs/transformers/en/chat_templating_multimodal) to format your messages automatically.
+
+## 🏃 Inference
+
+You can run LFM2.5-VL-1.6B with Hugging Face [`transformers`](https://github.com/huggingface/transformers):
+
+```bash
+pip install git+https://github.com/huggingface/transformers.git@3c2517727ce28a30f5044e01663ee204deb1cdbe pillow
+```
+
+```python
+from transformers import AutoProcessor, AutoModelForImageTextToText
+from transformers.image_utils import load_image
+
+# Load model and processor
+model_id = "LiquidAI/LFM2.5-VL-1.6B"
+model = AutoModelForImageTextToText.from_pretrained(
+    model_id,
+    device_map="auto",
+    dtype="bfloat16"
+)
+processor = AutoProcessor.from_pretrained(model_id)
+
+# Load image and create conversation
+url = "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
+image = load_image(url)
+conversation = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "image": image},
+            {"type": "text", "text": "What is in this image?"},
+        ],
+    },
+]
+
+# Generate Answer
+inputs = processor.apply_chat_template(
+    conversation,
+    add_generation_prompt=True,
+    return_tensors="pt",
+    return_dict=True,
+    tokenize=True,
+).to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=64)
+processor.batch_decode(outputs, skip_special_tokens=True)[0]
+
+# This image showcases the iconic Statue of Liberty standing majestically on Liberty Island in New York Harbor. The statue is positioned on a small island surrounded by calm blue waters, with the New York City skyline visible in the background.
+```
+
+### Tool Use
+
+LFM2.5 supports function calling for text only input by applying the chat template with the tokenizer. See the [Tool Use documentation](https://docs.liquid.ai/lfm/key-concepts/tool-use) for the full guide.
+
+```python
+tools = [{
+    "name": "get_weather",
+    "description": "Get current weather for a location",
+    "parameters": {
+        "type": "object",
+        "properties": {"location": {"type": "string"}},
+        "required": ["location"]
+    }
+}]
+
+messages = [{"role": "user", "content": "What's the weather in Paris?"}]
+
+# Apply chat template with tools
+inputs = processor.tokenizer.apply_chat_template(
+    messages,
+    tools=tools,
+    add_generation_prompt=True,
+    return_tensors="pt",
+    return_dict=True,
+)
+input_ids = inputs["input_ids"].to(model.device)
+outputs = model.generate(input_ids, max_new_tokens=256)
+response = processor.tokenizer.decode(outputs[0, input_ids.shape[1]:], skip_special_tokens=False)
+
+# <|tool_call_start|>[get_weather(location="Paris")]<|tool_call_end|>I am retrieving the current weather for Paris.<|im_end|>
+```
+
+| Name | Description | Docs | Notebook |
+|------|-------------|------|----------|
+| [Transformers](https://github.com/huggingface/transformers) | Simple inference with direct access to model internals. | <a href="https://docs.liquid.ai/lfm/inference/transformers#vision-models">Link</a>| <a href="https://colab.research.google.com/drive/1WVQpf4XrHgHFkP0FnlZfx2nK8PugvQNZ?usp=sharing"><img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png" width="110" alt="Colab link"></a> |
+| [vLLM](https://github.com/vllm-project/vllm) | High-throughput production deployments with GPU. | coming soon | coming soon |
+| [llama.cpp](https://github.com/ggml-org/llama.cpp) | Cross-platform inference with CPU offloading. | <a href="https://docs.liquid.ai/lfm/inference/llama-cpp#vision-models">Link</a> | <a href="https://colab.research.google.com/drive/10gOUd8TvhWvofdaZvGBP9MBAxY41DG0i?usp=sharing"><img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png" width="110" alt="Colab link"></a> |
+
+## 🔧 Fine-tuning
+
+We recommend fine-tuning LFM2.5-VL-1.6B model on your use cases to maximize performance.
+
+| Notebook  | Description                                                          | Link |
+|-----------|----------------------------------------------------------------------|------|
+| SFT (TRL) | Supervised Fine-Tuning (SFT) notebook with a LoRA adapter using TRL. | <a href="https://colab.research.google.com/drive/10530_jt_Joa5zH2wgYlyXosypq1R7PIz?usp=sharing"><img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png" width="110" alt="Colab link"></a> |
+
+
+## 📊 Performance
+
+| Model              | MMStar | MM-IFEval | BLINK | InfoVQA (Val) | OCRBench (v2) | RealWorldQA | MMMU (Val) | MMMB (avg) | Multilingual MMBench (avg) |
+|--------------------|--------|-----------|-------|---------------|---------------|-------------|------------|------------|----------------------------|
+| **LFM2.5-VL-1.6B** | 50.67  | 52.29     | 48.82 | 62.71         | 41.44         | 64.84       | 40.56      | 76.96      | 65.90                      |
+| LFM2-VL-1.6B       | 49.87  | 46.35     | 44.50 | 58.35         | 35.11         | 65.75       | 39.67      | 72.13      | 60.57                      |
+| InternVL3.5-1B     | 50.27  | 36.17     | 44.19 | 60.99         | 33.53         | 57.12       | 41.89      | 68.93      | 58.32                      |
+| FastVLM-1.5B       | 53.13  | 24.99     | 43.29 | 23.92         | 26.61         | 61.56       | 38.78      | 64.84      | 50.89                      |
+
+All vision benchmark scores are obtained using [VLMEvalKit](https://github.com/open-compass/VLMEvalKit). Multilingual scores are based on the average of benchmarks translated by GPT-4.1-mini from English to Arabic, Chinese, French, German, Japanese, Korean, and Spanish.
+
+## 📬 Contact
+
+If you are interested in custom solutions with edge deployment, please contact [our sales team](https://www.liquid.ai/contact).
+
+## Citation
+
+```
+@article{liquidai2025lfm2,
+ title={LFM2 Technical Report},
+ author={Liquid AI},
+ journal={arXiv preprint arXiv:2511.23404},
+ year={2025}
+}
+```

chat_template.jinja

@@ -0,0 +1,55 @@
+{{- bos_token -}}
+{%- set keep_past_thinking = keep_past_thinking | default(false) -%}
+{%- set ns = namespace(system_prompt="") -%}
+{%- if messages[0]["role"] == "system" -%}
+    {%- set ns.system_prompt = messages[0]["content"] -%}
+    {%- set messages = messages[1:] -%}
+{%- endif -%}
+{%- if tools -%}
+    {%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: [" -%}
+    {%- for tool in tools -%}
+        {%- if tool is not string -%}
+            {%- set tool = tool | tojson -%}
+        {%- endif -%}
+        {%- set ns.system_prompt = ns.system_prompt + tool -%}
+        {%- if not loop.last -%}
+            {%- set ns.system_prompt = ns.system_prompt + ", " -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns.system_prompt = ns.system_prompt + "]" -%}
+{%- endif -%}
+{%- if ns.system_prompt -%}
+    {{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
+{%- endif -%}
+{%- set ns.last_assistant_index = -1 -%}
+{%- for message in messages -%}
+    {%- if message["role"] == "assistant" -%}
+        {%- set ns.last_assistant_index = loop.index0 -%}
+    {%- endif -%}
+{%- endfor -%}
+{%- for message in messages -%}
+    {{- "<|im_start|>" + message["role"] + "\n" -}}
+    {%- set content = message["content"] -%}
+    {%- if content is not string -%}
+        {%- set ns.content = "" -%}
+        {%- for item in content -%}
+            {%- if item["type"] == "image" -%}
+                {%- set ns.content = ns.content + "<image>" -%}
+            {%- elif item["type"] == "text" -%}
+                {%- set ns.content = ns.content + item["text"] -%}
+            {%- else -%}
+                {%- set ns.content = ns.content + item | tojson -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- set content = ns.content -%}
+    {%- endif -%}
+    {%- if message["role"] == "assistant" and not keep_past_thinking and loop.index0 != ns.last_assistant_index -%}
+        {%- if "</think>" in content -%}
+            {%- set content = content.split("</think>")[-1] | trim -%}
+        {%- endif -%}
+    {%- endif -%}
+    {{- content + "<|im_end|>\n" -}}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{- "<|im_start|>assistant\n" -}}
+{%- endif -%}

config.json

@@ -0,0 +1,97 @@
+{
+  "architectures": [
+    "Lfm2VlForConditionalGeneration"
+  ],
+  "do_image_splitting": true,
+  "downsample_factor": 2,
+  "dtype": "bfloat16",
+  "encoder_patch_size": 16,
+  "image_token_id": 396,
+  "max_image_tokens": 256,
+  "max_pixels_tolerance": 2.0,
+  "max_tiles": 10,
+  "min_image_tokens": 64,
+  "min_tiles": 2,
+  "model_type": "lfm2_vl",
+  "projector_bias": true,
+  "projector_hidden_act": "gelu",
+  "projector_hidden_size": 2048,
+  "projector_use_layernorm": false,
+  "text_config": {
+    "_name_or_path": "LiquidAI/LFM2-1.2B",
+    "architectures": [
+      "Lfm2ForCausalLM"
+    ],
+    "block_auto_adjust_ff_dim": true,
+    "block_dim": 2048,
+    "block_ff_dim": 12288,
+    "block_ffn_dim_multiplier": 1.0,
+    "block_mlp_init_scale": 1.0,
+    "block_multiple_of": 256,
+    "block_norm_eps": 1e-05,
+    "block_out_init_scale": 1.0,
+    "block_use_swiglu": true,
+    "block_use_xavier_init": true,
+    "conv_L_cache": 3,
+    "conv_bias": false,
+    "conv_dim": 2048,
+    "conv_dim_out": 2048,
+    "conv_use_xavier_init": true,
+    "dtype": "bfloat16",
+    "eos_token_id": 7,
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "layer_types": [
+      "conv",
+      "conv",
+      "full_attention",
+      "conv",
+      "conv",
+      "full_attention",
+      "conv",
+      "conv",
+      "full_attention",
+      "conv",
+      "full_attention",
+      "conv",
+      "full_attention",
+      "conv",
+      "full_attention",
+      "conv"
+    ],
+    "max_position_embeddings": 128000,
+    "model_type": "lfm2",
+    "norm_eps": 1e-05,
+    "num_attention_heads": 32,
+    "num_heads": 32,
+    "num_hidden_layers": 16,
+    "num_key_value_heads": 8,
+    "rope_parameters": {
+      "rope_theta": 1000000.0,
+      "rope_type": "default"
+    },
+    "use_cache": true,
+    "use_pos_enc": true,
+    "vocab_size": 65536
+  },
+  "tile_size": 512,
+  "transformers_version": "5.0.0.dev0",
+  "use_image_special_tokens": true,
+  "use_thumbnail": true,
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip2_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "num_patches": 256,
+    "patch_size": 16,
+    "vision_use_head": false
+  }
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 7,
+  "pad_token_id": 0,
+  "transformers_version": "5.0.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fc7458e4382fc6e558cfdda45857fbf9ab5b40a8bf199c9cd073003b14ac26d
+size 3193334216

processor_config.json

@@ -0,0 +1,39 @@
+{
+  "image_processor": {
+    "data_format": "channels_first",
+    "do_image_splitting": true,
+    "do_normalize": true,
+    "do_pad": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "downsample_factor": 2,
+    "encoder_patch_size": 16,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Lfm2VlImageProcessorFast",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_image_tokens": 256,
+    "max_num_patches": 1024,
+    "max_pixels_tolerance": 2.0,
+    "max_tiles": 10,
+    "min_image_tokens": 64,
+    "min_tiles": 2,
+    "resample": 2,
+    "rescale_factor": 0.00392156862745098,
+    "return_row_col_info": true,
+    "size": {
+      "height": 512,
+      "width": 512
+    },
+    "tile_size": 512,
+    "use_thumbnail": true
+  },
+  "processor_class": "Lfm2VlProcessor"
+}

tokenizer_config.json

@@ -0,0 +1,27 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|im_end|>",
+  "image_end_token": "<|image_end|>",
+  "image_start_token": "<|image_start|>",
+  "image_thumbnail": "<|img_thumbnail|>",
+  "image_token": "<image>",
+  "is_local": true,
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "image_end_token": "<|image_end|>",
+    "image_start_token": "<|image_start|>",
+    "image_thumbnail": "<|img_thumbnail|>",
+    "image_token": "<image>"
+  },
+  "pad_token": "<|pad|>",
+  "processor_class": "Lfm2VlProcessor",
+  "return_token_type_ids": false,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "use_default_system_prompt": false,
+  "use_fast": true
+}