--- license: other license_name: lfm1.0 license_link: LICENSE base_model: LiquidAI/LFM2-2.6B pipeline_tag: text-generation library_name: transformers language: - en tags: - cybersecurity - cyberbench - threat-intelligence - named-entity-recognition - phishing-detection - http-anomaly-detection - lfm2 - 2.6b datasets: - jpmorganchase/CyberBench - tihanyin/CyberMetric - secbench-hf/SecBench - XuanwuAI/SecEval --- # Cyber-Prime 1.1 (2.6B) Cyber-Prime 1.1 is a cybersecurity-focused instruction-tuned model in the Liquid Foundation Model (LFM2) family. It is intended for defensive security analysis and cybersecurity text tasks: extracting entities from threat reports, classifying phishing emails and anomalous HTTP requests, answering cybersecurity multiple-choice questions, and producing short threat-intelligence headlines. The checkpoint is a full, merged model. Its training lineage starts from Liquid AI's LFM2-2.6B base through earlier Cyber-Prime checkpoints; the final run continued from the prior Cyber-Prime checkpoint. ## Benchmark results The table uses the same models and scores shown in the benchmark graphic. Values are rounded to three decimals. | Benchmark | Cyber-Prime 1 | Cyber-Prime 1.1 | GPT-4 | Mistral-7B-Instruct-v0.1 | Zephyr-7B-beta | Vicuna-7B-v1.5 | Llama-2-7B | |---|---:|---:|---:|---:|---:|---:|---:| | CyNER | 0.382 | 0.499 | 0.554 | 0.323 | 0.300 | 0.258 | 0.263 | | APTNER | 0.413 | 0.490 | 0.500 | 0.262 | 0.305 | 0.275 | 0.280 | | CyNews | 0.354 | 0.430 | 0.275 | 0.217 | 0.241 | 0.277 | 0.003 | | SecMMLU | 0.580 | 0.600 | 0.830 | 0.720 | 0.740 | 0.640 | 0.630 | | CyQuiz | 0.570 | 0.610 | 0.810 | 0.690 | 0.750 | 0.660 | 0.620 | | Email Phishing Detection | 0.728 | 0.890 | 0.939 | 0.889 | 0.867 | 0.864 | 0.942 | | HTTP Attack Log Analysis | 0.483 | 0.628 | 0.841 | 0.472 | 0.662 | 0.537 | 0.428 | | Overall Cybersecurity Average | 0.501 | 0.592 | 0.678 | 0.510 | 0.552 | 0.502 | 0.452 | Overall average is the unweighted mean of the seven benchmark scores shown; it is a descriptive comparison, not an official CyberBench aggregate. ![Cyber-Prime 1.1 compared with Cyber-Prime 1 and published CyberBench baselines](cyberprime-1.1-benchmark.png) ### Evaluation protocol - Test data were held out from the CyberBench training mix. - CyNER, APTNER, SecMMLU, CyQuiz, email, and HTTP used two in-context examples from their training pools. CyNews was evaluated zero-shot, as in the CyberBench setup. - Generation used greedy decoding (temperature 0) with the model's chat template. Reasoning text was excluded before task scoring where applicable. - Email F1 treats `phishing` as the positive class; HTTP F1 treats `anomalous` as the positive class. - The published baseline models in the graphic use the CyberBench paper's retrieval-based five-shot setup (zero-shot for CyNews). Cyber-Prime 1.1 was evaluated with two shots for the non-summarization tasks, so the cross-model bars are useful context, not a strictly matched comparison. ## Training Cyber-Prime 1.1 was developed from more than 75,000 cybersecurity training rows, combining supervised fine-tuning with a range of reinforcement-learning and reward-guided post-training techniques. ## Intended use - Defensive cybersecurity education and research. - Cybersecurity entity extraction from reports and logs. - Triage assistance for phishing-email and anomalous-HTTP classification. - Cybersecurity multiple-choice question answering. - Short threat-intelligence headline generation. Use the model as an assistive component with human review. It is not a substitute for security controls, incident-response procedures, or expert validation, and it is not a reliable autonomous vulnerability assessment or exploitation agent. ## Limitations - Performance is measured on the listed benchmark test sets; it should not be assumed to transfer to every organization, threat actor, protocol, or language. - NER recall remains lower than precision, especially for rare or densely packed entities. - The MCQ test sets are small, and small score differences may be noise. - The published baseline comparison uses a different few-shot count than the Cyber-Prime 1.1 run; prompts and serving stacks can materially change scores. - The model can produce incorrect, outdated, or overconfident cybersecurity claims. Verify indicators, classifications, and recommendations independently before taking action. ## Loading ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Akahsizrr/Cyber-Prime-1.1-2.6B" tokenizer = AutoTokenizer.from_pretrained(repo) model = AutoModelForCausalLM.from_pretrained( repo, torch_dtype=torch.bfloat16, device_map="auto", ) messages = [{ "role": "user", "content": "Classify this HTTP request as normal or anomalous: GET /index.html HTTP/1.1", }] input_ids = tokenizer.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", ).to(model.device) output = model.generate(input_ids, max_new_tokens=256, do_sample=False) print(tokenizer.decode(output[0][input_ids.shape[-1]:], skip_special_tokens=True)) ``` Use task-specific instructions and validate the output format expected by your downstream system. For NER, request a JSON object; for email and HTTP classification, request only the benchmark label; for CyNews-like summarization, request a concise headline. ## License This model is derived from **LiquidAI/LFM2-2.6B** and is distributed under the upstream **LFM Open License v1.0**, included in [`LICENSE`](LICENSE). This is a custom license, not Apache-2.0. In particular, Section 5 does not license commercial use by a legal entity with annual revenue of USD 10 million or more; such use requires separate permission from the licensor. Review the complete license before use or redistribution. ## Data and references - Liu, Shi, and Buford, [CyberBench: A Multi-Task Benchmark for Evaluating Large Language Models in Cybersecurity](https://zefang-liu.github.io/files/liu2024cyberbench_paper.pdf), AICS 2024. - [CyberBench code and evaluation harness](https://github.com/jpmorganchase/CyberBench). - [CyberMetric](https://huggingface.co/datasets/tihanyin/CyberMetric), [SecBench](https://huggingface.co/datasets/secbench-hf/SecBench), and [SecEval](https://huggingface.co/datasets/XuanwuAI/SecEval) supplied additional cybersecurity multiple-choice training examples. - Published baseline values in the benchmark graphic are referenced from the [CyberBench results table](https://benchmarklist.com/benchmarks/cyberbench/) and the CyberBench paper; they were not re-run as part of this release evaluation.