OpenVINO

/

LFM2-24B-A2B-int4-ov

Like 1

Follow

OpenVINO Toolkit 926

Hi, it looks like this model doesn't working with OVMS. AI tells that this model uses Liquid AI, so it shouldn't have graph.pbtxt. But I didn't make changes and configured lfm-config.json:

{
    "model_config_list": [
        {
            "config": {
                "name": "lfm-24b",
                "base_path": "c:/models/OpenVINO/LFM2-24B-A2B-int4-ov",
                "target_device": "GPU",
                "plugin_config": {
                    "PERFORMANCE_HINT": "LATENCY"
                }
            }
        }
    ]
}

And started start-lfm.bat

C:\models>start-lfm.bat

C:\models>ovms --config_path lfm-config.json --port 9000 --rest_port 8000
[2026-07-12 14:01:38.365][15744][serving][info][server.cpp:117] OpenVINO Model Server 2026.2.1.1122f03bf
[2026-07-12 14:01:38.365][15744][serving][info][server.cpp:118] OpenVINO backend 2026.2.1-21919-ede283a88e3-releases/2026/2
[2026-07-12 14:01:38.365][15744][serving][info][server.cpp:121] OpenVINO GenAI backend 2026.2.1.0-3123-7dea0459b2a
[2026-07-12 14:01:38.366][15744][serving][info][pythoninterpretermodule.cpp:37] PythonInterpreterModule starting
[2026-07-12 14:01:38.366][15744][serving][info][pythoninterpretermodule.cpp:40] Initializing python interpreter
Python version:
3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
Python sys.path output:
['', 'c:\\ovms\\python\\python312', 'c:\\ovms\\python', 'c:\\ovms\\python\\Scripts', 'c:\\ovms\\python\\Lib\\site-packages']
[2026-07-12 14:01:38.387][15744][serving][info][pythoninterpretermodule.cpp:59] PythonInterpreterModule started
[2026-07-12 14:01:38.547][15744][modelmanager][info][modelmanager.cpp:180] Available devices for Open VINO: CPU, GPU, NPU
[2026-07-12 14:01:38.550][15744][serving][info][capimodule.cpp:40] C-APIModule starting
[2026-07-12 14:01:38.550][15744][serving][info][capimodule.cpp:42] C-APIModule started
[2026-07-12 14:01:38.551][15744][serving][info][grpcservermodule.cpp:110] GRPCServerModule starting
[2026-07-12 14:01:38.551][15744][serving][info][grpcservermodule.cpp:137] Binding gRPC server to address: 0.0.0.0:9000
[2026-07-12 14:01:38.552][15744][serving][info][grpcservermodule.cpp:192] GRPCServerModule started
[2026-07-12 14:01:38.552][15744][serving][info][grpcservermodule.cpp:193] Started gRPC server on port 9000
[2026-07-12 14:01:38.552][15744][serving][info][httpservermodule.cpp:35] HTTPServerModule starting
[2026-07-12 14:01:38.552][15744][serving][info][httpservermodule.cpp:39] Will start 22 REST workers
[2026-07-12 14:01:38.554][22196][serving][info][drogon_http_server.cpp:157] Binding REST server to address: 0.0.0.0:8000
[2026-07-12 14:01:38.611][15744][serving][info][drogon_http_server.cpp:187] REST server listening on port 8000 with 22 unary threads and 22 streaming threads
[2026-07-12 14:01:38.611][15744][serving][info][http_server.cpp:249] API key not provided via --api_key_file or API_KEY environment variable. Authentication will be disabled.
[2026-07-12 14:01:38.612][15744][serving][info][httpservermodule.cpp:52] HTTPServerModule started
[2026-07-12 14:01:38.612][15744][serving][info][httpservermodule.cpp:53] Started REST server at 0.0.0.0:8000
[2026-07-12 14:01:38.612][15744][serving][info][servablemanagermodule.cpp:51] ServableManagerModule starting
[2026-07-12 14:01:38.617][15744][serving][info][mediapipegraphdefinition.cpp:101] Graph queue globally disabled via OVMS_GRAPH_QUEUE_OFF=1 for mediapipe: lfm-24b
[2026-07-12 14:01:38.618][15744][serving][info][mediapipegraphdefinition.cpp:479] MediapipeGraphDefinition initializing graph nodes
[2026-07-12 14:01:38.618][15744][modelmanager][info][servable_initializer.cpp:439] Initializing Language Model Continuous Batching servable
[2026-07-12 14:01:39.076][15744][serving][error][servable_initializer.cpp:240] Error during llm node initialization for models_path: c:/models/OpenVINO/LFM2-24B-A2B-int4-ov\./ exception: Check 'unregistered_parameters.str().empty()' failed at src\core\src\model.cpp:266:
Model references undeclared parameters: opset1::Parameter attention_mask () -> (i64[?,?])


[2026-07-12 14:01:39.076][15744][modelmanager][error][servable_initializer.cpp:444] Error during LLM node resources initialization: The LLM Node resource initialization failed
[2026-07-12 14:01:39.076][15744][serving][error][llm_node_initializer.cpp:60] Failed to process LLM node graph lfm-24b
[2026-07-12 14:01:39.076][15744][modelmanager][info][pipelinedefinitionstatus.hpp:60] Mediapipe: lfm-24b state changed to: LOADING_PRECONDITION_FAILED after handling: ValidationFailedEvent:
[2026-07-12 14:01:39.077][15744][serving][info][servablemanagermodule.cpp:55] ServableManagerModule started
[2026-07-12 14:01:39.077][20532][modelmanager][info][modelmanager.cpp:1079] Started cleaner thread
[2026-07-12 14:01:39.077][21192][modelmanager][info][modelmanager.cpp:1060] Started model manager thread

As you see, it wasn't started. So I tried to rename graph.pbtxt to understand that it could work without by new liquid technology. But it also didn't work:

C:\models>start-lfm.bat

C:\models>ovms --config_path lfm-config.json --port 9000 --rest_port 8000
[2026-07-12 14:09:48.153][22916][serving][info][server.cpp:117] OpenVINO Model Server 2026.2.1.1122f03bf
[2026-07-12 14:09:48.153][22916][serving][info][server.cpp:118] OpenVINO backend 2026.2.1-21919-ede283a88e3-releases/2026/2
[2026-07-12 14:09:48.153][22916][serving][info][server.cpp:121] OpenVINO GenAI backend 2026.2.1.0-3123-7dea0459b2a
[2026-07-12 14:09:48.153][22916][serving][info][pythoninterpretermodule.cpp:37] PythonInterpreterModule starting
[2026-07-12 14:09:48.153][22916][serving][info][pythoninterpretermodule.cpp:40] Initializing python interpreter
Python version:
3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
Python sys.path output:
['', 'c:\\ovms\\python\\python312', 'c:\\ovms\\python', 'c:\\ovms\\python\\Scripts', 'c:\\ovms\\python\\Lib\\site-packages']
[2026-07-12 14:09:48.170][22916][serving][info][pythoninterpretermodule.cpp:59] PythonInterpreterModule started
[2026-07-12 14:09:48.322][22916][modelmanager][info][modelmanager.cpp:180] Available devices for Open VINO: CPU, GPU, NPU
[2026-07-12 14:09:48.325][22916][serving][info][capimodule.cpp:40] C-APIModule starting
[2026-07-12 14:09:48.325][22916][serving][info][capimodule.cpp:42] C-APIModule started
[2026-07-12 14:09:48.325][22916][serving][info][grpcservermodule.cpp:110] GRPCServerModule starting
[2026-07-12 14:09:48.325][22916][serving][info][grpcservermodule.cpp:137] Binding gRPC server to address: 0.0.0.0:9000
[2026-07-12 14:09:48.326][22916][serving][info][grpcservermodule.cpp:192] GRPCServerModule started
[2026-07-12 14:09:48.326][22916][serving][info][grpcservermodule.cpp:193] Started gRPC server on port 9000
[2026-07-12 14:09:48.326][22916][serving][info][httpservermodule.cpp:35] HTTPServerModule starting
[2026-07-12 14:09:48.326][22916][serving][info][httpservermodule.cpp:39] Will start 22 REST workers
[2026-07-12 14:09:48.327][10564][serving][info][drogon_http_server.cpp:157] Binding REST server to address: 0.0.0.0:8000
[2026-07-12 14:09:48.388][22916][serving][info][drogon_http_server.cpp:187] REST server listening on port 8000 with 22 unary threads and 22 streaming threads
[2026-07-12 14:09:48.388][22916][serving][info][http_server.cpp:249] API key not provided via --api_key_file or API_KEY environment variable. Authentication will be disabled.
[2026-07-12 14:09:48.389][22916][serving][info][httpservermodule.cpp:52] HTTPServerModule started
[2026-07-12 14:09:48.389][22916][serving][info][httpservermodule.cpp:53] Started REST server at 0.0.0.0:8000
[2026-07-12 14:09:48.389][22916][serving][info][servablemanagermodule.cpp:51] ServableManagerModule starting
[2026-07-12 14:09:48.393][22916][modelmanager][warning][modelmanager.cpp:1328] Expected version directory name to be in number format. Got: .git
[2026-07-12 14:09:48.393][22916][modelmanager][warning][modelmanager.cpp:1335] No version found for model in path: c:/models/OpenVINO/LFM2-24B-A2B-int4-ov
[2026-07-12 14:09:48.393][22916][serving][info][servablemanagermodule.cpp:55] ServableManagerModule started
[2026-07-12 14:09:48.393][23000][modelmanager][info][modelmanager.cpp:1060] Started model manager thread
[2026-07-12 14:09:48.393][22200][modelmanager][info][modelmanager.cpp:1079] Started cleaner thread
[2026-07-12 14:09:49.400][23000][modelmanager][warning][modelmanager.cpp:1328] Expected version directory name to be in number format. Got: .git
[2026-07-12 14:09:49.400][23000][modelmanager][warning][modelmanager.cpp:1335] No version found for model in path: c:/models/OpenVINO/LFM2-24B-A2B-int4-ov
[2026-07-12 14:09:50.409][23000][modelmanager][warning][modelmanager.cpp:1328] Expected version directory name to be in number format. Got: .git
[2026-07-12 14:09:50.409][23000][modelmanager][warning][modelmanager.cpp:1335] No version found for model in path: c:/models/OpenVINO/LFM2-24B-A2B-int4-ov
[2026-07-12 14:09:51.410][23000][modelmanager][warning][modelmanager.cpp:1328] Expected version directory name to be in number format. Got: .git

Please fix this issues when you will add support of ovms.