Instructions to use ddalcu/LTX-2.5-MLX-Serve-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use ddalcu/LTX-2.5-MLX-Serve-4bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir LTX-2.5-MLX-Serve-4bit ddalcu/LTX-2.5-MLX-Serve-4bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Upload folder using huggingface_hub
Browse files- .gitattributes +2 -0
- LICENSE.md +125 -0
- README.md +121 -0
- audio_vae.safetensors +3 -0
- config.json +45 -0
- connector.safetensors +3 -0
- embedded_config.json +81 -0
- gemma4-12b-ltx-v1/chat_template.jinja +390 -0
- gemma4-12b-ltx-v1/config.json +174 -0
- gemma4-12b-ltx-v1/generation_config.json +18 -0
- gemma4-12b-ltx-v1/model.safetensors +3 -0
- gemma4-12b-ltx-v1/processor_config.json +62 -0
- gemma4-12b-ltx-v1/tokenizer.json +3 -0
- gemma4-12b-ltx-v1/tokenizer_config.json +142 -0
- ltx-acceptable-use-policy-snapshot-2026-08-12.pdf +3 -0
- quantize_config.json +7 -0
- spatial_upscaler_x2_v1_1.safetensors +3 -0
- spatial_upscaler_x2_v1_1_config.json +13 -0
- split_model.json +21 -0
- temporal_upscaler_x2_v1_0.safetensors +3 -0
- temporal_upscaler_x2_v1_0_config.json +13 -0
- transformer-dev.safetensors +3 -0
- transformer-distilled.safetensors +3 -0
- vae_decoder.safetensors +3 -0
- vae_encoder.safetensors +3 -0
- vocoder.safetensors +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
gemma4-12b-ltx-v1/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
ltx-acceptable-use-policy-snapshot-2026-08-12.pdf filter=lfs diff=lfs merge=lfs -text
|
LICENSE.md
ADDED
|
@@ -0,0 +1,125 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# LTX-2.x Community License Agreement
|
| 2 |
+
|
| 3 |
+
*License date: August 11, 2026*
|
| 4 |
+
|
| 5 |
+
**By downloading, using, accessing or distributing any portion or element of LTX-2.x, you agree that you have read and accepted to be bound by this Agreement.**
|
| 6 |
+
|
| 7 |
+
**1. Definitions.**
|
| 8 |
+
|
| 9 |
+
> **1.1** **“Agreement”** means the terms and conditions for this LTX-2.x Community License Agreement and the exhibits, attachments, and Complementary Materials, as specified in this document.
|
| 10 |
+
|
| 11 |
+
> **1.2** **“Complementary Materials”** means the accompanying documentation, tutorials, examples, configuration files and other materials made available by Licensor together with the LTX-2.x model weights and parameters, in each case as distributed by Licensor.
|
| 12 |
+
|
| 13 |
+
> **1.3** **“Control”** means the direct or indirect ownership of more than fifty percent (50%) of the voting securities or other ownership interests, or the power to direct the management and policies of such Entity through voting rights, contract, or otherwise.
|
| 14 |
+
|
| 15 |
+
> **1.4** **“Data”** means a collection of information and/or content extracted from the dataset used with LTX-2.x, including to train, pretrain, or otherwise evaluate LTX-2.x. The Data is not licensed under this Agreement.
|
| 16 |
+
|
| 17 |
+
> **1.5** **“Derivatives of LTX-2.x”** means all modifications to LTX-2.x, works based on LTX-2.x, or any other model which is created or initialized by transfer of patterns of the weights, parameters, activations or output of LTX-2.x, to the other model, in order to cause the other model to perform similarly to LTX-2.x, including – but not limited to - distillation methods entailing the use of intermediate data representations or methods based on the generation of synthetic data by LTX-2.x for training the other model. For clarity, Derivatives of LTX-2.x include: (i) any fine-tuned or adapted weights, parameters, or checkpoints derived from LTX-2.x; (ii) derivative model architectures that incorporate or are based upon LTX-2.x’s architecture; and (iii) any modified or extended versions of the Complementary Materials.
|
| 18 |
+
|
| 19 |
+
> **1.6** **“Entity”** means any individual, corporation, partnership, limited liability company, or other legal entity. For purposes of this Agreement, an Entity shall be deemed to include, on an aggregative basis, all subsidiaries, affiliates, and other companies under common Control with such Entity. When determining whether an Entity meets any threshold under this Agreement (including revenue thresholds in Section 2.1), all subsidiaries, affiliates, and companies under common Control shall be considered collectively.
|
| 20 |
+
|
| 21 |
+
> **1.7** **“Harm”** includes but is not limited to physical, mental, psychological, financial and reputational damage, pain, or loss.
|
| 22 |
+
|
| 23 |
+
> **1.8** **“Licensor”** or **“LTX”** means the owner that is granting the license under this Agreement. For the purposes of this Agreement, the Licensor is Lightricks Ltd.
|
| 24 |
+
|
| 25 |
+
> **1.9** **“LTX-2.x”** means the large generative models, text/image/video/audio/3D generation models, and multimodal large language models and their software and algorithms, including trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code, accompanying source code, scripts, and all other elements of the foregoing distributed and made publicly available by LTX (including, for example, at [https://github.com/Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)). This license is applicable to all LTX-2.5 versions released since August 11, 2026, and all future releases of LTX-2.x under this license.
|
| 26 |
+
|
| 27 |
+
> **1.10** **“Output”** means the results of operating LTX-2.x as embodied in informational content resulting therefrom.
|
| 28 |
+
|
| 29 |
+
> **1.11** **“you”** (or **“your”**) means an individual or legal Entity licensing LTX-2.x in accordance with this Agreement and/or otherwise downloading, accessing, distributing or using LTX-2.x for whichever purpose and in any field of use, including usage of LTX-2.x in an end-use application - e.g. chatbot, translator, image generator.
|
| 30 |
+
|
| 31 |
+
**2. Grant of License.**
|
| 32 |
+
|
| 33 |
+
> **2.1** Subject to your compliance with the terms and conditions of this Agreement, you are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Licensor’s intellectual property or other rights owned by Licensor embodied in LTX-2.x to use, reproduce, prepare, distribute, publicly display, publicly perform, sublicense, copy, create derivative works of, and make modifications to LTX-2.x, for any purpose, subject to the restrictions set forth in Attachment A; **provided however, that Entities with annual revenues of at least $10,000,000 (the “Commercial Entities”) are required to obtain a paid license for any use (excluding use solely for a Non-Commercial Purpose as set forth in Section 2.2) of LTX-2.x and Derivatives of LTX-2.x (such paid license referred to herein as a “Commercial Use Agreement”), as will be provided by the Licensor. Commercial Entities interested in such a commercial license are required to [contact Licensor](mailto:ltxv-licensing@lightricks.com). Any use of LTX-2.x or Derivatives of LTX-2.x by Commercial Entities not in accordance with this Agreement and/or the Commercial Use Agreement is strictly prohibited and shall be deemed a material breach of this Agreement. In the event of such material breach, and without limiting Licensor’s right to terminate the Agreement or to pursue any other remedies available at law or in equity, you shall pay Licensor the license fees owed for the period such Commercial Entity used LTX-2.x (calculated at Licensor’s standard commercial license fees, in effect during the relevant period or, absent published standard fees, a reasonable market rate for a comparable license), within thirty (30) days of Licensor’s written demand.**
|
| 34 |
+
|
| 35 |
+
> **2.2** Notwithstanding the foregoing or anything to the contrary in this Agreement, a Commercial Entity may download and use LTX-2.x and Derivatives of LTX-2.x without obtaining the Commercial Use Agreement solely for a Non-Commercial Purpose. **“Non-Commercial Purpose”** means any of the following uses, but only so far as such Commercial Entity does not receive any direct or indirect payment arising from the use of LTX-2.x or Derivatives of LTX-2.x: (i) use by an individual acting in a personal capacity for research, experimentation, learning, private study, hobby or recreational projects, or personal entertainment, in each case where such use is not connected, directly or indirectly, to any commercial activity, business operation, or the performance of duties for an employer or any other Entity; and (ii) use by a Commercial Entity for testing, evaluation, or non-commercial research and development in a non-production or development environment. For clarity, use (a) for revenue-generating activity in any manner, whether direct or indirect, (b) in direct interactions with or that has impact on end users, or (c) to train, fine-tune, or distill any model (including any Derivative of LTX-2.x) for commercial use, in each case, is not a Non-Commercial Purpose and requires all Commercial Entities to obtain a paid license under the Commercial Use Agreement prior to such use. For the avoidance of doubt, the permission granted under this Section 2.2 is a limited right of use only and does not convey or transfer any ownership right, title, or interest in or to LTX-2.x or any Derivatives of LTX-2.x, and all Derivatives of LTX-2.x created or used pursuant to this Section 2.2 remain subject to the terms of this Agreement, including Section 1.5.
|
| 36 |
+
|
| 37 |
+
**3. Distribution and Redistribution.** You may host for third parties remote access purposes (e.g. software-as-a-service), reproduce and distribute copies of LTX-2.x or Derivatives of LTX-2.x thereof in any medium, with or without modifications, provided that you meet the following conditions:
|
| 38 |
+
|
| 39 |
+
> **3.1** Use-based restrictions as referenced in Section 4 and all provisions of Attachment A MUST be included as an enforceable provision by you in any type of legal agreement (e.g. a license) governing the use and/or distribution of LTX-2.x or Derivatives of LTX-2.x, and you shall give notice to subsequent users you distribute to, that LTX-2.x or Derivatives of LTX-2.x are subject to Section 4 and Attachment A in their entirety, including all use restrictions and acceptable use policies;
|
| 40 |
+
|
| 41 |
+
> **3.2** You must provide any third-party recipients of LTX-2.x or Derivatives of LTX-2.x a copy of this Agreement, including all attachments and use policies. Any Derivative of LTX-2.x (as defined in Section 1.5, including but not limited to fine-tuned weights, modified training code, models trained on Outputs, or any other derivative) must be distributed exclusively under the terms of this Agreement, subject to Section 3.6, with a complete copy of this Agreement included;
|
| 42 |
+
|
| 43 |
+
> **3.3** You must cause any modified files to carry prominent notices stating that you changed the files;
|
| 44 |
+
|
| 45 |
+
> **3.4** You must retain all copyright, patent, trademark, and attribution notices excluding those notices that do not pertain to any part of LTX-2.x, Derivatives of LTX-2.x.
|
| 46 |
+
|
| 47 |
+
> **3.5** Transfer of Derivatives. No transfer of any Derivative of LTX-2.x (including any fine-tuned weights, LoRA adapters, or similar adaptations) to a third party shall grant such third party any right, title, license, or authorization to access, use, reproduce, distribute, or exploit LTX-2.x, or any Derivative of LTX-2.x beyond the rights granted under this Agreement. If the transferee is a Commercial Entity (as defined in Section 2), it must obtain a paid license from Licensor prior to any use of any Derivative of LTX-2.x, regardless of who created such Derivative. Prior to or at the time of any such transfer, you shall notify the transferee in writing that (i) use of such Derivative of LTX-2.x is subject to the terms of this Agreement, and (ii) if the transferee is a Commercial Entity, it must obtain a separate paid license to LTX-2.x from Licensor. You shall not transfer any Derivative of LTX-2.x to a Commercial Entity unless such Commercial Entity has obtained the required paid license from Licensor prior to any use, and unless the proposed transferee has been so informed. You and the transferee shall each be responsible for ensuring the transferee obtains the required license from Licensor prior to any use of LTX-2.x or Derivative of LTX-2.x. Nothing in this Section 3.5 shall require a Commercial Entity to obtain a paid license for use solely for a Non-Commercial Purpose as permitted under Section 2.2.
|
| 48 |
+
|
| 49 |
+
> **3.6** You may add your own copyright statement to your modifications and may provide additional license terms and conditions - **respecting Section 3.1** - for use, reproduction, or distribution of your modifications, or for any such Derivatives of LTX-2.x as a whole, provided your use, reproduction, and distribution of LTX-2.x otherwise complies with the conditions stated in this Agreement, and you provide a complete copy of this Agreement with any such use, reproduction and distribution of LTX-2.x and any Derivatives thereof; provided that any such additional terms shall be additive only and shall not derogate from, conflict with, waive, or purport to modify any term of this Agreement, and this Agreement shall govern in the event of any conflict.
|
| 50 |
+
|
| 51 |
+
**4. Use-based restrictions.** The restrictions set forth in Attachment A are considered Use-based restrictions. Therefore, you cannot use LTX-2.x and the Derivatives of LTX-2.x in violation of the specified restricted uses. You may use LTX-2.x subject to this Agreement, only for lawful purposes and in accordance with the Agreement. **“Use”** may include creating any content with, fine-tuning, updating, running, training, evaluating and/or re-parametrizing LTX-2.x. You shall require all of your users who use LTX-2.x or a Derivative of LTX-2.x to comply with the terms of this Section 4.
|
| 52 |
+
|
| 53 |
+
**5. The Output You Generate.** Except as set forth herein, Licensor claims no rights in the Output you generate using LTX-2.x. You are accountable for input you insert into LTX-2.x, the Output you generate and its subsequent uses. No use of the Output can contravene any provision as stated in the Agreement.
|
| 54 |
+
|
| 55 |
+
**6. Updates and Runtime Restrictions; AI Regulations.** To the maximum extent permitted by law, Licensor reserves the right to restrict (remotely or otherwise) usage of LTX-2.x in violation of this Agreement, update LTX-2.x through electronic means, or modify the Output of LTX-2.x based on updates. You shall undertake reasonable efforts to use the latest version of LTX-2.x. Any use of the non-current version of LTX-2.x is done solely at your risk. To the extent applicable to you, you shall comply with all laws and regulations governing artificial intelligence that apply to your use, deployment, or distribution of LTX-2.x, Derivatives of LTX-2.x, or Outputs, including Regulation (EU) 2024/1689 (the “EU AI Act”) and the California AI Transparency Act (Cal. Bus. & Prof. Code § 22757 et seq.), each as amended from time to time and any other applicable laws, regulations, or binding guidance relating to artificial intelligence, transparency, content provenance, or synthetic media, together with any documentation made available by Licensor regarding compliance with the same (collectively, “AI Regulations”). You shall maintain (including within any application or service through which LTX-2.x, any Derivative of LTX-2.x, or any Output is made available), and shall not remove, disable, alter, or circumvent, any safety or security measures, disclosures, metadata, watermarking, content provenance, latent disclosure, or other transparency features or functionalities included or embedded within LTX-2.x or any Derivative of LTX-2.x, or applied to any Output, in furtherance of AI Regulations, including any capability of LTX-2.x to include latent disclosures in Outputs, and you shall include equivalent obligations in any agreement governing your distribution of LTX-2.x or any Derivative of LTX-2.x. You are solely responsible for any transparency, disclosure, marking, or labeling obligations applicable to you under AI Regulations as a provider or deployer of LTX-2.x, any Derivative of LTX-2.x, or any system incorporating any of the foregoing, including any obligation to disclose that content is artificially generated or manipulated. If Licensor knows or reasonably believes that you have modified LTX-2.x or any Derivative of LTX-2.x such that it is no longer capable of including any disclosure required by AI Regulations in Outputs, or that you have otherwise removed, disabled, or circumvented any feature or functionality described in this Section, Licensor may in its sole discretion revoke the license granted under this Agreement effective immediately upon notice to you, and upon such revocation you shall immediately cease all use of LTX-2.x and Derivatives of LTX-2.x. Licensor makes no representation or warranty that LTX-2.x, any Derivative of LTX-2.x, or any Output complies with any AI Regulations applicable to your specific use case or deployment, and you are solely responsible for determining the applicability of, and ensuring your compliance with, all AI Regulations. You shall indemnify, defend, and hold harmless Licensor and its affiliates from and against any and all claims, liabilities, losses, damages, costs, and expenses (including reasonable attorneys’ fees) arising out of or relating to your use, deployment, distribution, or modification of LTX-2.x, any Derivative of LTX-2.x, or any Output in violation of, or your other failure to comply with, any AI Regulations.
|
| 56 |
+
|
| 57 |
+
> For purposes of the EU AI Act, Licensor makes LTX-2.x openly available under this community license and intends that LTX-2.x be treated as a free and open-source general purpose AI model within the meaning of Article 53(2) of the EU AI Act. You acknowledge and agree that (a) to the extent the free and open source derogations under Article 53(2) of the EU AI Act apply, Licensor’s obligations under the EU AI Act with respect to LTX-2.x are limited to those applicable to providers of free and open source general purpose AI models (it being acknowledged that such derogations do not extend to the obligations under Article 53(1)(c) and (d)), (b) you acknowledge that LTX-2.x is not intended to be integrated into a high risk AI system, and shall be fully and solely responsible for any obligation resulting from such integration, (c) if you integrate LTX-2.x or any Derivative of LTX-2.x into a high-risk AI system you shall be solely responsible for all provider obligations that would otherwise apply to Licensor under the EU AI Act, and (d) you shall not take any action, or omit to take any action, that would cause Licensor to lose the benefit of the free and open source derogations under the EU AI Act, and you shall indemnify and hold Licensor harmless from any liability, costs, or expenses arising from your breach of this Section.
|
| 58 |
+
|
| 59 |
+
**7. Export Controls and Sanctions Compliance.** You acknowledge that LTX-2.x, Derivatives of LTX-2.x may be subject to export control laws and regulations, including but not limited to the U.S. Export Administration Regulations and sanctions programs administered by the Office of Foreign Assets Control (OFAC). You represent and warrant that you and any users of LTX-2.x are not (i) located in, organized under the laws of, or ordinarily resident in any country or territory subject to comprehensive sanctions; (ii) identified on any U.S. government restricted party list, including the Specially Designated Nationals and Blocked Persons List; or (iii) otherwise prohibited from receiving LTX-2.x under applicable law. You shall not export, re-export, or transfer LTX-2.x, directly or indirectly, in violation of any applicable export control or sanctions laws or regulations. You agree to comply with all applicable trade control laws and shall indemnify and hold Licensor harmless from any claims arising from your failure to comply with such laws.
|
| 60 |
+
|
| 61 |
+
**8. Trademarks; Reservation of Rights.** Nothing in this Agreement permits you to make use of Licensor’s trademarks, trade names, logos or to otherwise suggest endorsement or misrepresent the relationship between the parties; and any rights not expressly granted herein are reserved by the Licensor. Except as expressly set forth in this Agreement, Licensor does not grant, directly or by implication, estoppel, statute or otherwise, any right or license in its, or its affiliates’, intellectual property rights or other proprietary rights. For avoidance of doubt, all intellectual property rights in Derivatives of LTX-2.x shall be subject to the terms of this Agreement, and you acquire no right, title, or interest in or to LTX-2.x itself, which is and remains the exclusive property of Licensor. You shall not assert any ownership or other right in LTX-2.x or any Derivative of LTX-2.x in any manner that restricts, encumbers, or is inconsistent with the rights retained by Licensor or granted to other licensees under this Agreement.
|
| 62 |
+
|
| 63 |
+
**9. Disclaimer of Warranty.** Unless required by applicable law or agreed to in writing, Licensor provides LTX-2.x on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing LTX-2.x and Derivatives of LTX-2.x and assume any risks associated with your exercise of permissions under this Agreement.
|
| 64 |
+
|
| 65 |
+
**10. Limitation of Liability.** To the fullest extent permitted by applicable law, in no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall Licensor be liable to you or any other individual or Entity for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this Agreement or out of the use of, or inability to use LTX-2.x or any Derivative of LTX-2.x (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if Licensor has been advised of the possibility of such damages.
|
| 66 |
+
|
| 67 |
+
**11. Accepting Warranty or Additional Liability.** While redistributing LTX-2.x and Derivatives of LTX-2.x, you may, provided you do not violate the terms of this Agreement, choose to offer and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations. However, in accepting such obligations, you may act only on your own behalf and on your sole responsibility, not on behalf of Licensor, and only if you agree to indemnify, defend, and hold Licensor harmless for any liability incurred by, or claims asserted against Licensor, by reason of your accepting any such warranty or additional liability.
|
| 68 |
+
|
| 69 |
+
**12. Governing Law.** This Agreement and all relations, disputes, claims and other matters arising hereunder (including non-contractual disputes or claims) will be governed exclusively by, and construed exclusively in accordance with, the laws of the State of New York and applicable U.S. federal law. To the extent permitted by law, choice of laws rules and the United Nations Convention on Contracts for the International Sale of Goods will not apply. The prevailing party in any claim or dispute between the parties under this Agreement will be entitled to reimbursement of its reasonable attorneys’ fees and costs.
|
| 70 |
+
|
| 71 |
+
**13. Term and Termination.** This Agreement is effective upon your acceptance and continues until terminated. Licensor may terminate this Agreement immediately upon written notice to you if you breach any provision of this Agreement, including but not limited to violations of the use restrictions in Attachment A or unauthorized commercial use. This Agreement also terminates immediately and automatically, without notice, upon any material breach of this Agreement, including any use in violation of applicable AI Regulations or any unauthorized commercial use of LTX-2.x or Derivatives of LTX-2.x by a Commercial Entity. Upon termination: (a) all rights granted to you under this Agreement will immediately cease; (b) you must immediately cease all use of LTX-2.x and Derivatives of LTX-2.x; (c) you must delete or destroy all copies of LTX-2.x and Derivatives of LTX-2.x in your possession or control; and (d) you must notify any third parties to whom you distributed LTX-2.x or Derivatives of LTX-2.x of the termination. Sections 2, 3, 4, 6-16 and Attachment A shall survive termination of this Agreement. Termination does not relieve you of any obligations incurred prior to termination, including payment obligations under Section 2 and adhering to the restrictions under Section 3. In addition, if You commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against Licensor or any person or entity alleging that LTX-2.x or any Output, or any portion of any of the foregoing, infringe any intellectual property or other right owned or licensable by you, then all licenses granted to you under this Agreement shall terminate as of the date such lawsuit or other proceeding is filed.
|
| 72 |
+
|
| 73 |
+
**14. Disputes and Arbitration; Waiver of Jury Trial; Class Action Waiver.** **IF YOU ARE NOT ACTING AS A CONSUMER UNDER APPLICABLE LAW, YOU HEREBY WAIVE THE RIGHT TO A TRIAL BY JURY, TO PARTICIPATE IN A CLASS OR REPRESENTATIVE ACTION (INCLUDING IN ARBITRATION), OR TO COMBINE INDIVIDUAL PROCEEDINGS IN COURT OR IN ARBITRATION WITHOUT THE CONSENT OF ALL PARTIES.** All disputes arising in connection with this Agreement shall be finally settled by arbitration under the Rules of Arbitration of the International Chamber of Commerce (“ICC Rules”), by one (1) arbitrator appointed in accordance with the ICC Rules. The seat of arbitration shall be New York, NY, USA, and the proceedings shall be conducted in English. The arbitrator shall be empowered to grant any relief that a court could grant. Judgment on the arbitration award may be entered by any court having jurisdiction thereof. Notwithstanding the foregoing, either party may seek injunctive or other equitable relief in respect of any actual or threatened breach of the license restrictions under this Agreement (including Attachment A and the Acceptable Use Policy) or any actual or threatened infringement, misappropriation, or violation of Licensor’s intellectual property rights, in the state or federal courts located in the County of New York, State of New York, and each party irrevocably consents to the jurisdiction of, and venue in, such courts for that limited purpose. The foregoing waivers do not apply to, and are not enforceable against, any licensee acting as a consumer under the mandatory consumer-protection laws of its jurisdiction of residence (including, without limitation, the European Union, the United Kingdom, and the State of California), and nothing in this Agreement limits any rights under such laws that cannot be waived or limited by contract. If any waiver in this Section is held invalid or unenforceable as to a particular licensee or dispute, such waiver shall be severed to that extent only and shall not affect the validity or enforceability of the remainder of this Section.
|
| 74 |
+
|
| 75 |
+
**15.** In the event of any exception to the application of binding arbitration, all disputes, claims, and other matters arising hereunder shall be brought exclusively in the state or federal courts located in the County of New York, State of New York. You waive all defenses of lack of personal jurisdiction and forum non conveniens with respect to venue and jurisdiction in such courts, and consent to their exclusive jurisdiction and venue.
|
| 76 |
+
|
| 77 |
+
**16. Severability.** If any provision of this Agreement is held to be invalid, illegal or unenforceable, the remaining provisions shall be unaffected thereby and remain valid as if such provision had not been set forth herein.
|
| 78 |
+
|
| 79 |
+
**END OF TERMS AND CONDITIONS**
|
| 80 |
+
|
| 81 |
+
# Attachment A
|
| 82 |
+
|
| 83 |
+
*Use Restrictions*
|
| 84 |
+
|
| 85 |
+
When using the Outputs, LTX-2.x and any Derivatives thereof, you agree to comply with the [Acceptable Use Policy](https://static.lightricks.com/legal/ltx-acceptable-use-policy.pdf) which is hereby incorporated into and made part of this Agreement by reference. Licensor may update it from time to time, and the version in effect at the time of your use governs; continued use after an update constitutes acceptance. Licensor shall post each version of the Acceptable Use Policy with its effective date, and no update shall apply retroactively to use occurring before that effective date. In addition, you agree not to use the Outputs, LTX-2.x or its Derivatives in any of the following ways:
|
| 86 |
+
|
| 87 |
+
> **1)** In any way that violates any applicable national, federal, state, local or international law or regulation;
|
| 88 |
+
|
| 89 |
+
> **2)** For the purpose of exploiting, Harming or attempting to exploit or Harm minors in any way;
|
| 90 |
+
|
| 91 |
+
> **3)** Knowingly generate or disseminate verifiably false information and/or content with the intent to deceive, defraud, or otherwise unlawfully Harm others;
|
| 92 |
+
|
| 93 |
+
> **4)** To generate or disseminate personal identifiable information that can be used to Harm an individual;
|
| 94 |
+
|
| 95 |
+
> **5)** To generate or disseminate information and/or content (e.g. images, code, posts, articles), and place the information and/or content in any context (e.g. bot generating tweets) without expressly and intelligibly disclaiming that the information and/or content is machine generated;
|
| 96 |
+
|
| 97 |
+
> **6)** To defame others, or to engage in the unlawful harassment of others;
|
| 98 |
+
|
| 99 |
+
> **7)** To impersonate or attempt to impersonate (e.g. deepfakes) others without their consent;
|
| 100 |
+
|
| 101 |
+
> **8)** For fully automated decision making that adversely impacts an individual’s legal rights or otherwise creates or modifies a binding, enforceable obligation;
|
| 102 |
+
|
| 103 |
+
> **9)** For any use intended to or which has the effect of discriminating against or Harming individuals or groups based on online or offline social behavior or known or predicted personal or personality characteristics;
|
| 104 |
+
|
| 105 |
+
> **10)** To exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological Harm;
|
| 106 |
+
|
| 107 |
+
> **11)** For any use intended to or which has the effect of discriminating against individuals or groups based on legally protected characteristics or categories;
|
| 108 |
+
|
| 109 |
+
> **12)** To provide medical advice and medical results interpretation;
|
| 110 |
+
|
| 111 |
+
> **13)** To generate or disseminate information for the purpose to be used for administration of justice, law enforcement, immigration or asylum processes, such as predicting an individual will commit fraud/crime commitment (e.g. by text profiling, drawing causal relationships between assertions made in documents, indiscriminate and arbitrarily-targeted use);
|
| 112 |
+
|
| 113 |
+
> **14)** To generate and/or disseminate malware (including – but not limited to – ransomware) or any other content to be used for the purpose of harming electronic systems;
|
| 114 |
+
|
| 115 |
+
> **15)** To engage in, promote, incite, or facilitate discrimination or other unlawful or harmful conduct in the provision of employment, employment benefits, credit, housing, or other essential goods and services;
|
| 116 |
+
|
| 117 |
+
> **16)** To engage in, promote, incite, or facilitate the harassment, abuse, threatening, or bullying of individuals or groups of individuals;
|
| 118 |
+
|
| 119 |
+
> **17)** For military, warfare, nuclear industries or applications, weapons development, or any use in connection with activities that may cause death, personal injury, or severe physical or environmental damage;
|
| 120 |
+
|
| 121 |
+
> **18)** For commercial use only: To train, improve, or fine-tune any other machine learning model, artificial intelligence system, or competing model, except for Derivatives of LTX-2.x as expressly permitted under this Agreement;
|
| 122 |
+
|
| 123 |
+
> **19)** To circumvent, disable, or interfere with any technical limitations, safety features, content filters, watermarking, content provenance or latent disclosure functionalities, or use restrictions implemented in LTX-2.x by Licensor;
|
| 124 |
+
|
| 125 |
+
> **20)** To use LTX-2.x or Derivatives of LTX-2.x in any product, service, or application that directly competes with Licensor’s commercial products or services, or is designed to replace or substitute Licensor’s offerings in the market, without obtaining a separate commercial license from Licensor.
|
README.md
ADDED
|
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: ltx-2-community-license-agreement
|
| 4 |
+
license_link: LICENSE.md
|
| 5 |
+
base_model: mlx-community/ltx-2.5-mlx
|
| 6 |
+
base_model_relation: quantized
|
| 7 |
+
library_name: mlx
|
| 8 |
+
tags:
|
| 9 |
+
- mlx
|
| 10 |
+
- mlx-serve
|
| 11 |
+
- ltx-video
|
| 12 |
+
- ltx-2.5
|
| 13 |
+
- text-to-video
|
| 14 |
+
- image-to-video
|
| 15 |
+
- audio-video
|
| 16 |
+
pipeline_tag: text-to-video
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# LTX-2.5, MLX 4-bit for mlx-serve
|
| 20 |
+
|
| 21 |
+
4-bit MLX conversion of [mlx-community/ltx-2.5-mlx](https://huggingface.co/mlx-community/ltx-2.5-mlx)
|
| 22 |
+
(itself a conversion of [Lightricks/LTX-2.5](https://huggingface.co/Lightricks/LTX-2.5)),
|
| 23 |
+
built for **[mlx-serve](https://mlxserve.com)** — the native Zig MLX server for
|
| 24 |
+
Apple Silicon ([mlxserve.com](https://mlxserve.com)).
|
| 25 |
+
|
| 26 |
+
Joint audio+video: one model generates the frames and their soundtrack together.
|
| 27 |
+
Text-to-video, image-to-video (first-frame conditioning) and audio-to-video, plus
|
| 28 |
+
the two-stage pipelines, all served natively — no Python.
|
| 29 |
+
|
| 30 |
+
**36 GB on disk, down from 110 GB.** The upstream conversion ships bf16 only; this
|
| 31 |
+
is the same pack at the 4-bit width mlx-serve's LTX engine reads, and it is
|
| 32 |
+
self-contained — the text encoder is inside it, so nothing else downloads.
|
| 33 |
+
|
| 34 |
+
## ⚠️ License — read before use
|
| 35 |
+
|
| 36 |
+
These weights are **Derivatives of LTX-2.5** under the **LTX-2.x Community
|
| 37 |
+
License Agreement**. A complete copy ships here as [`LICENSE.md`](LICENSE.md), and
|
| 38 |
+
the Acceptable Use Policy it incorporates by reference is snapshotted as
|
| 39 |
+
[`ltx-acceptable-use-policy-snapshot-2026-08-12.pdf`](ltx-acceptable-use-policy-snapshot-2026-08-12.pdf)
|
| 40 |
+
(the version in effect at your time of use governs — check
|
| 41 |
+
[Lightricks' current AUP](https://static.lightricks.com/legal/ltx-acceptable-use-policy.pdf)).
|
| 42 |
+
|
| 43 |
+
**Transfer notice (Agreement §3.5).** Your use of these weights is subject to the
|
| 44 |
+
LTX-2.x Community License Agreement. If you (aggregated across entities under
|
| 45 |
+
common control) have annual revenues of **US $10,000,000 or more**, you are a
|
| 46 |
+
"Commercial Entity" under the Agreement and **must obtain a paid license from
|
| 47 |
+
Lightricks before any use** other than the Agreement's non-commercial-purpose
|
| 48 |
+
carve-outs (testing, evaluation, non-commercial R&D in non-production
|
| 49 |
+
environments). Sub-threshold commercial and production use is royalty-free under
|
| 50 |
+
the Agreement's terms.
|
| 51 |
+
|
| 52 |
+
Further obligations that travel with these weights include (not exhaustive — read
|
| 53 |
+
the license): machine-generated content disclosure (Attachment A §5), no removal
|
| 54 |
+
or circumvention of any transparency/provenance features (§6), EU AI Act /
|
| 55 |
+
CA AI Transparency Act responsibilities for providers/deployers (§6), and the
|
| 56 |
+
Attachment A acceptable-use terms.
|
| 57 |
+
|
| 58 |
+
## Modification notice (Agreement §3.3)
|
| 59 |
+
|
| 60 |
+
Modified from `mlx-community/ltx-2.5-mlx`. The change is **quantization only** — no
|
| 61 |
+
training, no fine-tuning, no architectural change:
|
| 62 |
+
|
| 63 |
+
- Both DiT variants: affine **4-bit, group size 64**, applied to the 34 linear
|
| 64 |
+
projections in each of the 48 transformer blocks (1632 weights per variant) —
|
| 65 |
+
the six attention modules' `to_q`/`to_k`/`to_v`/`to_out`/`to_gate_logits` and
|
| 66 |
+
the two feed-forwards' `proj_in`/`proj_out`. 38.0 GB → 11.3 GB each.
|
| 67 |
+
- Text encoder (`gemma4-12b-ltx-v1/`): affine 4-bit, group size 64, on every 2-D
|
| 68 |
+
projection plus the embedding table (329 weights). 23.8 GB → 6.7 GB.
|
| 69 |
+
- **Kept at their shipped dtype**: every adaLN table, `patchify_proj`/`proj_out`,
|
| 70 |
+
the q/k norms, `keyframes_abs_pos_embedding`, and all of the connector, both
|
| 71 |
+
video VAEs, the audio VAE, the vocoder and both latent upscalers — copied
|
| 72 |
+
byte-for-byte.
|
| 73 |
+
|
| 74 |
+
That split is the same recipe the established 2.3 MLX packs use
|
| 75 |
+
(`only_transformer_blocks`), which is what mlx-serve's loader expects: it treats a
|
| 76 |
+
tensor as quantized iff a sibling `<name>.scales` exists.
|
| 77 |
+
|
| 78 |
+
Conversion script: [`scripts/quantize_ltx25.py`](https://github.com/ddalcu/mlx-serve/blob/main/scripts/quantize_ltx25.py).
|
| 79 |
+
The embedded LTX-2.x license text in upstream file metadata is preserved.
|
| 80 |
+
|
| 81 |
+
## Contents
|
| 82 |
+
|
| 83 |
+
| File | Contents |
|
| 84 |
+
|---|---|
|
| 85 |
+
| `transformer-distilled.safetensors` | distilled joint-AV DiT, 4-bit — fixed 8-step, CFG 1 (the one-stage default) |
|
| 86 |
+
| `transformer-dev.safetensors` | dev (full) joint-AV DiT, 4-bit — two-stage stage 1, takes CFG |
|
| 87 |
+
| `gemma4-12b-ltx-v1/` | Lightricks-tuned Gemma-4-12B text encoder, 4-bit |
|
| 88 |
+
| `connector.safetensors` | text-embedding connectors + aggregate projections (bf16) |
|
| 89 |
+
| `vae_encoder` / `vae_decoder` | conv video VAE |
|
| 90 |
+
| `audio_vae` / `vocoder` | audio VAE + BigVGAN v2 |
|
| 91 |
+
| `spatial_upscaler_x2_v1_1` / `temporal_upscaler_x2_v1_0` | ×2 latent upscalers |
|
| 92 |
+
|
| 93 |
+
Only what the engine reads. The upstream repo's `vae_diffusion_decoder` and
|
| 94 |
+
`duration_head` are not included — mlx-serve does not implement them yet.
|
| 95 |
+
|
| 96 |
+
## Usage
|
| 97 |
+
|
| 98 |
+
```bash
|
| 99 |
+
mlx-serve --model <path>/LTX-2.5-MLX-Serve-4bit --serve
|
| 100 |
+
```
|
| 101 |
+
|
| 102 |
+
```bash
|
| 103 |
+
curl -X POST http://127.0.0.1:11434/v1/video/generations \
|
| 104 |
+
-H 'Content-Type: application/json' \
|
| 105 |
+
-d '{"prompt":"a red fox trots across fresh snow in a pine forest at golden hour",
|
| 106 |
+
"num_frames":97,"height":480,"width":704,"steps":8,"seed":42}'
|
| 107 |
+
```
|
| 108 |
+
|
| 109 |
+
Or pick **LTX-Video 2.5** in the mlx-serve app's Video window.
|
| 110 |
+
|
| 111 |
+
### Measured
|
| 112 |
+
|
| 113 |
+
M4 Max (128 GB): 704×480, 97 frames @ 24 fps, 8 steps — **2m17s** end to end,
|
| 114 |
+
including the text encode and the VAE decode, with the generated soundtrack.
|
| 115 |
+
|
| 116 |
+
## Credits
|
| 117 |
+
|
| 118 |
+
All credit for the model to [Lightricks](https://huggingface.co/Lightricks) — see
|
| 119 |
+
the [LTX-2 reference implementation](https://github.com/Lightricks/LTX-2). The
|
| 120 |
+
MLX layout and the bf16 conversion this quantizes are
|
| 121 |
+
[mlx-community/ltx-2.5-mlx](https://huggingface.co/mlx-community/ltx-2.5-mlx)'s.
|
audio_vae.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d5cde5980a1e27c49188c6b1a17056ec5c06d333e5ecb66ef3e09d6e9e8cc011
|
| 3 |
+
size 106509020
|
config.json
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_version": "2.5.0",
|
| 3 |
+
"is_v2": true,
|
| 4 |
+
"model_type": "AudioVideo",
|
| 5 |
+
"num_attention_heads": 32,
|
| 6 |
+
"attention_head_dim": 128,
|
| 7 |
+
"in_channels": 128,
|
| 8 |
+
"out_channels": 128,
|
| 9 |
+
"num_layers": 48,
|
| 10 |
+
"cross_attention_dim": 4096,
|
| 11 |
+
"caption_channels": null,
|
| 12 |
+
"apply_gated_attention": true,
|
| 13 |
+
"audio_num_attention_heads": 32,
|
| 14 |
+
"audio_attention_head_dim": 64,
|
| 15 |
+
"audio_in_channels": 128,
|
| 16 |
+
"audio_out_channels": 128,
|
| 17 |
+
"audio_cross_attention_dim": 2048,
|
| 18 |
+
"positional_embedding_theta": 10000.0,
|
| 19 |
+
"positional_embedding_max_pos": [
|
| 20 |
+
20,
|
| 21 |
+
2048,
|
| 22 |
+
2048
|
| 23 |
+
],
|
| 24 |
+
"audio_positional_embedding_max_pos": [
|
| 25 |
+
20
|
| 26 |
+
],
|
| 27 |
+
"timestep_scale_multiplier": 1000,
|
| 28 |
+
"av_ca_timestep_scale_multiplier": 1000,
|
| 29 |
+
"norm_eps": 1e-06,
|
| 30 |
+
"connector_positional_embedding_max_pos": [
|
| 31 |
+
4096
|
| 32 |
+
],
|
| 33 |
+
"connector_rope_type": "SPLIT",
|
| 34 |
+
"variants": {
|
| 35 |
+
"distilled": {
|
| 36 |
+
"cross_attention_adaln": true
|
| 37 |
+
},
|
| 38 |
+
"dev": {
|
| 39 |
+
"cross_attention_adaln": true
|
| 40 |
+
}
|
| 41 |
+
},
|
| 42 |
+
"use_keyframes_abs_pos_embedding": true,
|
| 43 |
+
"ff_bias": false,
|
| 44 |
+
"audio_ff_bias": true
|
| 45 |
+
}
|
connector.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:028792ba757f509fef60b364a72976a86990930f29757f616a29149150476243
|
| 3 |
+
size 6344495770
|
embedded_config.json
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"transformer": {
|
| 3 |
+
"_class_name": "AVTransformer3DModel",
|
| 4 |
+
"activation_fn": "gelu-approximate",
|
| 5 |
+
"attention_bias": true,
|
| 6 |
+
"attention_head_dim": 128,
|
| 7 |
+
"attention_type": "default",
|
| 8 |
+
"caption_channels": 3840,
|
| 9 |
+
"cross_attention_dim": 4096,
|
| 10 |
+
"double_self_attention": false,
|
| 11 |
+
"dropout": 0.0,
|
| 12 |
+
"in_channels": 128,
|
| 13 |
+
"norm_elementwise_affine": false,
|
| 14 |
+
"norm_eps": 1e-06,
|
| 15 |
+
"norm_num_groups": 32,
|
| 16 |
+
"num_attention_heads": 32,
|
| 17 |
+
"num_embeds_ada_norm": 1000,
|
| 18 |
+
"num_layers": 48,
|
| 19 |
+
"num_vector_embeds": null,
|
| 20 |
+
"only_cross_attention": false,
|
| 21 |
+
"cross_attention_norm": true,
|
| 22 |
+
"out_channels": 128,
|
| 23 |
+
"upcast_attention": false,
|
| 24 |
+
"use_linear_projection": false,
|
| 25 |
+
"qk_norm": "rms_norm",
|
| 26 |
+
"standardization_norm": "rms_norm",
|
| 27 |
+
"positional_embedding_type": "rope",
|
| 28 |
+
"positional_embedding_theta": 10000.0,
|
| 29 |
+
"positional_embedding_max_pos": [
|
| 30 |
+
20,
|
| 31 |
+
2048,
|
| 32 |
+
2048
|
| 33 |
+
],
|
| 34 |
+
"timestep_scale_multiplier": 1000,
|
| 35 |
+
"av_ca_timestep_scale_multiplier": 1000.0,
|
| 36 |
+
"causal_temporal_positioning": true,
|
| 37 |
+
"audio_num_attention_heads": 32,
|
| 38 |
+
"audio_attention_head_dim": 64,
|
| 39 |
+
"use_audio_video_cross_attention": true,
|
| 40 |
+
"ff_bias": false,
|
| 41 |
+
"share_ff": false,
|
| 42 |
+
"audio_out_channels": 128,
|
| 43 |
+
"audio_cross_attention_dim": 2048,
|
| 44 |
+
"audio_positional_embedding_max_pos": [
|
| 45 |
+
20
|
| 46 |
+
],
|
| 47 |
+
"av_cross_ada_norm": true,
|
| 48 |
+
"use_embeddings_connector": true,
|
| 49 |
+
"connector_attention_head_dim": 128,
|
| 50 |
+
"connector_num_attention_heads": 32,
|
| 51 |
+
"connector_num_layers": 8,
|
| 52 |
+
"connector_positional_embedding_max_pos": [
|
| 53 |
+
4096
|
| 54 |
+
],
|
| 55 |
+
"connector_num_learnable_registers": 128,
|
| 56 |
+
"connector_norm_output": true,
|
| 57 |
+
"use_middle_indices_grid": true,
|
| 58 |
+
"apply_gated_attention": true,
|
| 59 |
+
"connector_apply_gated_attention": true,
|
| 60 |
+
"caption_projection_first_linear": false,
|
| 61 |
+
"caption_projection_second_linear": false,
|
| 62 |
+
"caption_proj_input_norm": false,
|
| 63 |
+
"connector_learnable_registers_std": 1,
|
| 64 |
+
"caption_proj_before_connector": true,
|
| 65 |
+
"audio_connector_attention_head_dim": 64,
|
| 66 |
+
"audio_connector_num_attention_heads": 32,
|
| 67 |
+
"cross_attention_adaln": true,
|
| 68 |
+
"rope_type": "split",
|
| 69 |
+
"frequencies_precision": "float64",
|
| 70 |
+
"text_encoder_norm_type": "PER_TOKEN_RMS",
|
| 71 |
+
"use_keyframes_abs_pos_embedding": true
|
| 72 |
+
},
|
| 73 |
+
"scheduler": {
|
| 74 |
+
"_class_name": "RectifiedFlowScheduler",
|
| 75 |
+
"_diffusers_version": "0.25.1",
|
| 76 |
+
"num_train_timesteps": 1000,
|
| 77 |
+
"shifting": null,
|
| 78 |
+
"base_resolution": null,
|
| 79 |
+
"sampler": "LinearQuadratic"
|
| 80 |
+
}
|
| 81 |
+
}
|
gemma4-12b-ltx-v1/chat_template.jinja
ADDED
|
@@ -0,0 +1,390 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{#
|
| 2 |
+
Template: Google Gemma 4 Canonical Chat Template
|
| 3 |
+
Author: Google Gemma Engineering Team
|
| 4 |
+
Published: 2026-07-09
|
| 5 |
+
Context: Fixed tool-calling loops, turn closures, and thinking content-ordering.
|
| 6 |
+
#}
|
| 7 |
+
{%- macro format_parameters(properties, required, filter_keys=false) -%}
|
| 8 |
+
{%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
|
| 9 |
+
{%- set ns = namespace(found_first=false) -%}
|
| 10 |
+
{%- for key, value in properties | dictsort -%}
|
| 11 |
+
{%- set add_comma = false -%}
|
| 12 |
+
{%- if not filter_keys or key not in standard_keys -%}
|
| 13 |
+
{%- if ns.found_first %},{% endif -%}
|
| 14 |
+
{%- set ns.found_first = true -%}
|
| 15 |
+
{{ key }}:{
|
| 16 |
+
{%- if value['description'] -%}
|
| 17 |
+
description:<|"|>{{ value['description'] }}<|"|>
|
| 18 |
+
{%- set add_comma = true -%}
|
| 19 |
+
{%- endif -%}
|
| 20 |
+
{%- if value['type'] | upper == 'STRING' -%}
|
| 21 |
+
{%- if value['enum'] -%}
|
| 22 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 23 |
+
enum:{{ format_argument(value['enum']) }}
|
| 24 |
+
{%- endif -%}
|
| 25 |
+
{%- elif value['type'] | upper == 'ARRAY' -%}
|
| 26 |
+
{%- if value['items'] is mapping and value['items'] -%}
|
| 27 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 28 |
+
items:{
|
| 29 |
+
{%- set ns_items = namespace(found_first=false) -%}
|
| 30 |
+
{%- for item_key, item_value in value['items'] | dictsort -%}
|
| 31 |
+
{%- if item_value is not none -%}
|
| 32 |
+
{%- if ns_items.found_first %},{% endif -%}
|
| 33 |
+
{%- set ns_items.found_first = true -%}
|
| 34 |
+
{%- if item_key == 'properties' -%}
|
| 35 |
+
properties:{
|
| 36 |
+
{%- if item_value is mapping -%}
|
| 37 |
+
{{- format_parameters(item_value, value['items']['required'] | default([])) -}}
|
| 38 |
+
{%- endif -%}
|
| 39 |
+
}
|
| 40 |
+
{%- elif item_key == 'required' -%}
|
| 41 |
+
required:[
|
| 42 |
+
{%- for req_item in item_value -%}
|
| 43 |
+
<|"|>{{- req_item -}}<|"|>
|
| 44 |
+
{%- if not loop.last %},{% endif -%}
|
| 45 |
+
{%- endfor -%}
|
| 46 |
+
]
|
| 47 |
+
{%- elif item_key == 'type' -%}
|
| 48 |
+
{%- if item_value is string -%}
|
| 49 |
+
type:{{ format_argument(item_value | upper) }}
|
| 50 |
+
{%- else -%}
|
| 51 |
+
type:{{ format_argument(item_value | map('upper') | list) }}
|
| 52 |
+
{%- endif -%}
|
| 53 |
+
{%- else -%}
|
| 54 |
+
{{ item_key }}:{{ format_argument(item_value) }}
|
| 55 |
+
{%- endif -%}
|
| 56 |
+
{%- endif -%}
|
| 57 |
+
{%- endfor -%}
|
| 58 |
+
}
|
| 59 |
+
{%- endif -%}
|
| 60 |
+
{%- endif -%}
|
| 61 |
+
{%- if value['nullable'] %}
|
| 62 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 63 |
+
nullable:true
|
| 64 |
+
{%- endif -%}
|
| 65 |
+
{%- if value['type'] | upper == 'OBJECT' -%}
|
| 66 |
+
{%- if value['properties'] is defined and value['properties'] is mapping -%}
|
| 67 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 68 |
+
properties:{
|
| 69 |
+
{{- format_parameters(value['properties'], value['required'] | default([])) -}}
|
| 70 |
+
}
|
| 71 |
+
{%- elif value is mapping -%}
|
| 72 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 73 |
+
properties:{
|
| 74 |
+
{{- format_parameters(value, value['required'] | default([]), filter_keys=true) -}}
|
| 75 |
+
}
|
| 76 |
+
{%- endif -%}
|
| 77 |
+
{%- if value['required'] -%}
|
| 78 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 79 |
+
required:[
|
| 80 |
+
{%- for item in value['required'] | default([]) -%}
|
| 81 |
+
<|"|>{{- item -}}<|"|>
|
| 82 |
+
{%- if not loop.last %},{% endif -%}
|
| 83 |
+
{%- endfor -%}
|
| 84 |
+
]
|
| 85 |
+
{%- endif -%}
|
| 86 |
+
{%- endif -%}
|
| 87 |
+
{%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
|
| 88 |
+
type:<|"|>{{ value['type'] | upper }}<|"|>}
|
| 89 |
+
{%- endif -%}
|
| 90 |
+
{%- endfor -%}
|
| 91 |
+
{%- endmacro -%}
|
| 92 |
+
{%- macro format_function_declaration(tool_data) -%}
|
| 93 |
+
declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
|
| 94 |
+
{%- set params = tool_data['function']['parameters'] -%}
|
| 95 |
+
{%- if params -%}
|
| 96 |
+
,parameters:{
|
| 97 |
+
{%- if params['properties'] -%}
|
| 98 |
+
properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
|
| 99 |
+
{%- endif -%}
|
| 100 |
+
{%- if params['required'] -%}
|
| 101 |
+
required:[
|
| 102 |
+
{%- for item in params['required'] -%}
|
| 103 |
+
<|"|>{{- item -}}<|"|>
|
| 104 |
+
{{- ',' if not loop.last -}}
|
| 105 |
+
{%- endfor -%}
|
| 106 |
+
],
|
| 107 |
+
{%- endif -%}
|
| 108 |
+
{%- if params['type'] -%}
|
| 109 |
+
type:<|"|>{{- params['type'] | upper -}}<|"|>}
|
| 110 |
+
{%- endif -%}
|
| 111 |
+
{%- endif -%}
|
| 112 |
+
{%- if 'response' in tool_data['function'] -%}
|
| 113 |
+
{%- set response_declaration = tool_data['function']['response'] -%}
|
| 114 |
+
,response:{
|
| 115 |
+
{%- if response_declaration['description'] -%}
|
| 116 |
+
description:<|"|>{{- response_declaration['description'] -}}<|"|>,
|
| 117 |
+
{%- endif -%}
|
| 118 |
+
{%- if response_declaration['type'] | upper == 'OBJECT' -%}
|
| 119 |
+
type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
|
| 120 |
+
{%- endif -%}
|
| 121 |
+
{%- endif -%}
|
| 122 |
+
}
|
| 123 |
+
{%- endmacro -%}
|
| 124 |
+
{%- macro format_argument(argument, escape_keys=True) -%}
|
| 125 |
+
{%- if argument is none -%}
|
| 126 |
+
{{- 'null' -}}
|
| 127 |
+
{%- elif argument is string -%}
|
| 128 |
+
{{- '<|"|>' + argument + '<|"|>' -}}
|
| 129 |
+
{%- elif argument is boolean -%}
|
| 130 |
+
{{- 'true' if argument else 'false' -}}
|
| 131 |
+
{%- elif argument is mapping -%}
|
| 132 |
+
{{- '{' -}}
|
| 133 |
+
{%- set ns = namespace(found_first=false) -%}
|
| 134 |
+
{%- for key, value in argument | dictsort -%}
|
| 135 |
+
{%- if ns.found_first %},{% endif -%}
|
| 136 |
+
{%- set ns.found_first = true -%}
|
| 137 |
+
{%- if escape_keys -%}
|
| 138 |
+
{{- '<|"|>' + key + '<|"|>' -}}
|
| 139 |
+
{%- else -%}
|
| 140 |
+
{{- key -}}
|
| 141 |
+
{%- endif -%}
|
| 142 |
+
:{{- format_argument(value, escape_keys=escape_keys) -}}
|
| 143 |
+
{%- endfor -%}
|
| 144 |
+
{{- '}' -}}
|
| 145 |
+
{%- elif argument is sequence -%}
|
| 146 |
+
{{- '[' -}}
|
| 147 |
+
{%- for item in argument -%}
|
| 148 |
+
{{- format_argument(item, escape_keys=escape_keys) -}}
|
| 149 |
+
{%- if not loop.last %},{% endif -%}
|
| 150 |
+
{%- endfor -%}
|
| 151 |
+
{{- ']' -}}
|
| 152 |
+
{%- else -%}
|
| 153 |
+
{{- argument -}}
|
| 154 |
+
{%- endif -%}
|
| 155 |
+
{%- endmacro -%}
|
| 156 |
+
{%- macro strip_thinking(text) -%}
|
| 157 |
+
{%- set ns = namespace(result='') -%}
|
| 158 |
+
{%- for part in text.split('<channel|>') -%}
|
| 159 |
+
{%- if '<|channel>' in part -%}
|
| 160 |
+
{%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
|
| 161 |
+
{%- else -%}
|
| 162 |
+
{%- set ns.result = ns.result + part -%}
|
| 163 |
+
{%- endif -%}
|
| 164 |
+
{%- endfor -%}
|
| 165 |
+
{{- ns.result | trim -}}
|
| 166 |
+
{%- endmacro -%}
|
| 167 |
+
|
| 168 |
+
{%- macro format_tool_response_block(tool_name, response) -%}
|
| 169 |
+
{{- '<|tool_response>' -}}
|
| 170 |
+
{%- if response is mapping -%}
|
| 171 |
+
{{- 'response:' + tool_name + '{' -}}
|
| 172 |
+
{%- for key, value in response | dictsort -%}
|
| 173 |
+
{{- key -}}:{{- format_argument(value, escape_keys=False) -}}
|
| 174 |
+
{%- if not loop.last %},{% endif -%}
|
| 175 |
+
{%- endfor -%}
|
| 176 |
+
{{- '}' -}}
|
| 177 |
+
{%- else -%}
|
| 178 |
+
{{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}
|
| 179 |
+
{%- endif -%}
|
| 180 |
+
{{- '<tool_response|>' -}}
|
| 181 |
+
{%- endmacro -%}
|
| 182 |
+
|
| 183 |
+
{#- ===== SETUP ===== -#}
|
| 184 |
+
{%- set ns = namespace(prev_message_type=None, prev_non_tool_role=None) -%}
|
| 185 |
+
{%- set loop_messages = messages -%}
|
| 186 |
+
{%- set enable_thinking = enable_thinking | default(false) -%}
|
| 187 |
+
{%- set preserve_thinking = preserve_thinking | default(false) -%}
|
| 188 |
+
{{- bos_token -}}
|
| 189 |
+
{#- Handle System/Tool Definitions Block -#}
|
| 190 |
+
{%- if enable_thinking or tools or (messages and messages[0]['role'] in ['system', 'developer']) -%}
|
| 191 |
+
{{- '<|turn>system\n' -}}
|
| 192 |
+
{#- Inject Thinking token at the very top of the FIRST system turn -#}
|
| 193 |
+
{%- if enable_thinking -%}
|
| 194 |
+
{{- '<|think|>\n' -}}
|
| 195 |
+
{%- set ns.prev_message_type = 'think' -%}
|
| 196 |
+
{%- endif -%}
|
| 197 |
+
{%- if messages and messages[0]['role'] in ['system', 'developer'] -%}
|
| 198 |
+
{%- if messages[0]['content'] is string -%}
|
| 199 |
+
{{- messages[0]['content'] | trim -}}
|
| 200 |
+
{%- elif messages[0]['content'] is sequence -%}
|
| 201 |
+
{%- for item in messages[0]['content'] -%}
|
| 202 |
+
{{- item['text'] | trim + ' '-}}
|
| 203 |
+
{%- endfor -%}
|
| 204 |
+
{%- endif -%}
|
| 205 |
+
{%- set loop_messages = messages[1:] -%}
|
| 206 |
+
{%- endif -%}
|
| 207 |
+
{%- if tools -%}
|
| 208 |
+
{%- for tool in tools %}
|
| 209 |
+
{{- '<|tool>' -}}
|
| 210 |
+
{{- format_function_declaration(tool) | trim -}}
|
| 211 |
+
{{- '<tool|>' -}}
|
| 212 |
+
{%- endfor %}
|
| 213 |
+
{%- set ns.prev_message_type = 'tool' -%}
|
| 214 |
+
{%- endif -%}
|
| 215 |
+
{{- '<turn|>\n' -}}
|
| 216 |
+
{%- endif %}
|
| 217 |
+
|
| 218 |
+
{#- Pre-scan: find last user message index for reasoning guard -#}
|
| 219 |
+
{%- set ns_turn = namespace(last_user_idx=-1) -%}
|
| 220 |
+
{%- for i in range(loop_messages | length) -%}
|
| 221 |
+
{%- if loop_messages[i]['role'] == 'user' -%}
|
| 222 |
+
{%- set ns_turn.last_user_idx = i -%}
|
| 223 |
+
{%- endif -%}
|
| 224 |
+
{%- endfor -%}
|
| 225 |
+
|
| 226 |
+
{#- Loop through messages -#}
|
| 227 |
+
{%- for message in loop_messages -%}
|
| 228 |
+
{%- if message['role'] != 'tool' -%}
|
| 229 |
+
{%- set ns.prev_message_type = None -%}
|
| 230 |
+
{%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
|
| 231 |
+
{#- Detect continuation using tracked state — O(1) instead of O(n) backward scan -#}
|
| 232 |
+
{%- set continue_same_model_turn = (role == 'model' and ns.prev_non_tool_role == 'assistant') -%}
|
| 233 |
+
{%- if not continue_same_model_turn -%}
|
| 234 |
+
{{- '<|turn>' + role + '\n' }}
|
| 235 |
+
|
| 236 |
+
{%- endif -%}
|
| 237 |
+
|
| 238 |
+
{#- Render reasoning/reasoning_content as thinking channel -#}
|
| 239 |
+
{%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
|
| 240 |
+
{%- set thinking_gate = (loop.index0 > ns_turn.last_user_idx) or (preserve_thinking and message.get('tool_calls')) -%}
|
| 241 |
+
{%- if thinking_text and thinking_gate -%}
|
| 242 |
+
{{- '<|channel>thought\n' + thinking_text + '\n<channel|>' -}}
|
| 243 |
+
{%- endif -%}
|
| 244 |
+
|
| 245 |
+
{%- if message.get('tool_calls') -%}
|
| 246 |
+
{%- for tool_call in message.get('tool_calls') -%}
|
| 247 |
+
{%- set function = tool_call['function'] -%}
|
| 248 |
+
{{- '<|tool_call>call:' + function['name'] + '{' -}}
|
| 249 |
+
{%- if function['arguments'] is mapping -%}
|
| 250 |
+
{%- set ns_args = namespace(found_first=false) -%}
|
| 251 |
+
{%- for key, value in function['arguments'] | dictsort -%}
|
| 252 |
+
{%- if ns_args.found_first %},{% endif -%}
|
| 253 |
+
{%- set ns_args.found_first = true -%}
|
| 254 |
+
{{- key -}}:{{- format_argument(value, escape_keys=False) -}}
|
| 255 |
+
{%- endfor -%}
|
| 256 |
+
{%- elif function['arguments'] is none -%}
|
| 257 |
+
{%- else -%}
|
| 258 |
+
{{- raise_exception(
|
| 259 |
+
"chat_template: tool_calls[].function.arguments must be a "
|
| 260 |
+
"JSON object (mapping), not a string. Deserialize arguments "
|
| 261 |
+
"before passing to the template."
|
| 262 |
+
) -}}
|
| 263 |
+
{%- endif -%}
|
| 264 |
+
{{- '}<tool_call|>' -}}
|
| 265 |
+
{%- endfor -%}
|
| 266 |
+
{%- set ns.prev_message_type = 'tool_call' -%}
|
| 267 |
+
{%- endif -%}
|
| 268 |
+
|
| 269 |
+
{%- set ns_tr_out = namespace(flag=false) -%}
|
| 270 |
+
{%- if message.get('tool_responses') -%}
|
| 271 |
+
{#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}
|
| 272 |
+
{%- for tool_response in message.get('tool_responses') -%}
|
| 273 |
+
{{- format_tool_response_block(tool_response['name'] | default('unknown', true), tool_response['response']) -}}
|
| 274 |
+
{%- set ns_tr_out.flag = true -%}
|
| 275 |
+
{%- set ns.prev_message_type = 'tool_response' -%}
|
| 276 |
+
{%- endfor -%}
|
| 277 |
+
{%- elif message.get('tool_calls') -%}
|
| 278 |
+
{#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}
|
| 279 |
+
{%- set ns_tool_scan = namespace(stopped=false) -%}
|
| 280 |
+
{%- for k in range(loop.index0 + 1, loop_messages | length) -%}
|
| 281 |
+
{%- if ns_tool_scan.stopped -%}
|
| 282 |
+
{%- elif loop_messages[k]['role'] != 'tool' -%}
|
| 283 |
+
{%- set ns_tool_scan.stopped = true -%}
|
| 284 |
+
{%- else -%}
|
| 285 |
+
{%- set follow = loop_messages[k] -%}
|
| 286 |
+
{#- Resolve tool_call_id to function name -#}
|
| 287 |
+
{%- set ns_tname = namespace(name=follow.get('name') or 'unknown') -%}
|
| 288 |
+
{%- for tc in message.get('tool_calls') -%}
|
| 289 |
+
{%- if tc.get('id') == follow.get('tool_call_id') -%}
|
| 290 |
+
{%- set ns_tname.name = tc['function']['name'] -%}
|
| 291 |
+
{%- endif -%}
|
| 292 |
+
{%- endfor -%}
|
| 293 |
+
{#- Handle content as string or content-parts array -#}
|
| 294 |
+
{%- set tool_body = follow.get('content') -%}
|
| 295 |
+
{%- if tool_body is string -%}
|
| 296 |
+
{{- format_tool_response_block(ns_tname.name, tool_body) -}}
|
| 297 |
+
{%- elif tool_body is sequence and tool_body is not string -%}
|
| 298 |
+
{%- set ns_txt = namespace(s='') -%}
|
| 299 |
+
{%- for part in tool_body -%}
|
| 300 |
+
{%- if part.get('type') == 'text' -%}
|
| 301 |
+
{%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}
|
| 302 |
+
{%- endif -%}
|
| 303 |
+
{%- endfor -%}
|
| 304 |
+
{{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}
|
| 305 |
+
{%- for part in tool_body -%}
|
| 306 |
+
{%- if part.get('type') in ['image', 'image_url'] -%}
|
| 307 |
+
{{- '<|image|>' -}}
|
| 308 |
+
{%- elif part.get('type') in ['audio', 'input_audio'] -%}
|
| 309 |
+
{{- '<|audio|>' -}}
|
| 310 |
+
{%- elif part.get('type') == 'video' -%}
|
| 311 |
+
{{- '<|video|>' -}}
|
| 312 |
+
{%- endif -%}
|
| 313 |
+
{%- endfor -%}
|
| 314 |
+
{%- else -%}
|
| 315 |
+
{{- format_tool_response_block(ns_tname.name, tool_body) -}}
|
| 316 |
+
{%- endif -%}
|
| 317 |
+
{%- set ns_tr_out.flag = true -%}
|
| 318 |
+
{%- set ns.prev_message_type = 'tool_response' -%}
|
| 319 |
+
{%- endif -%}
|
| 320 |
+
{%- endfor -%}
|
| 321 |
+
{%- endif -%}
|
| 322 |
+
|
| 323 |
+
{%- set captured_content -%}
|
| 324 |
+
{%- if message.get('content') is string -%}
|
| 325 |
+
{%- if role == 'model' -%}
|
| 326 |
+
{{- strip_thinking(message['content']) -}}
|
| 327 |
+
{%- else -%}
|
| 328 |
+
{{- message['content'] | trim -}}
|
| 329 |
+
{%- endif -%}
|
| 330 |
+
{%- elif message.get('content') is sequence -%}
|
| 331 |
+
{%- for item in message['content'] -%}
|
| 332 |
+
{%- if item.get('type') == 'text' -%}
|
| 333 |
+
{%- if role == 'model' -%}
|
| 334 |
+
{{- strip_thinking(item['text']) -}}
|
| 335 |
+
{%- else -%}
|
| 336 |
+
{{- item['text'] | trim -}}
|
| 337 |
+
{%- endif -%}
|
| 338 |
+
{%- elif item.get('type') in ['image', 'image_url'] -%}
|
| 339 |
+
{{- '<|image|>' -}}
|
| 340 |
+
{%- elif item.get('type') in ['audio', 'input_audio'] -%}
|
| 341 |
+
{{- '<|audio|>' -}}
|
| 342 |
+
{%- elif item.get('type') == 'video' -%}
|
| 343 |
+
{{- '<|video|>' -}}
|
| 344 |
+
{%- endif -%}
|
| 345 |
+
{%- endfor -%}
|
| 346 |
+
{%- endif -%}
|
| 347 |
+
{%- endset -%}
|
| 348 |
+
|
| 349 |
+
{{- captured_content -}}
|
| 350 |
+
{%- set has_content = captured_content | trim | length > 0 -%}
|
| 351 |
+
|
| 352 |
+
{#- Forward-scan: find next non-tool message role for continuation detection -#}
|
| 353 |
+
{%- set next_nt = namespace(role=None, found=false) -%}
|
| 354 |
+
{%- for j in range(loop.index0 + 1, loop_messages | length) -%}
|
| 355 |
+
{%- if not next_nt.found -%}
|
| 356 |
+
{%- if loop_messages[j]['role'] != 'tool' -%}
|
| 357 |
+
{%- set next_nt.role = loop_messages[j]['role'] -%}
|
| 358 |
+
{%- set next_nt.found = true -%}
|
| 359 |
+
{%- endif -%}
|
| 360 |
+
{%- endif -%}
|
| 361 |
+
{%- endfor -%}
|
| 362 |
+
|
| 363 |
+
{%- set continues_into_next = (
|
| 364 |
+
role == 'model'
|
| 365 |
+
and next_nt.role == 'assistant'
|
| 366 |
+
and (not message.get('tool_calls') or ns_tr_out.flag)
|
| 367 |
+
) -%}
|
| 368 |
+
|
| 369 |
+
{%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
|
| 370 |
+
{{- '<|tool_response>' -}}
|
| 371 |
+
{%- elif continues_into_next -%}
|
| 372 |
+
{%- elif not (ns_tr_out.flag and not has_content and not next_nt.found) -%}
|
| 373 |
+
{{- '<turn|>\n' -}}
|
| 374 |
+
{%- endif -%}
|
| 375 |
+
|
| 376 |
+
{#- Track previous non-tool role for next iteration (avoids O(n) backward scan) -#}
|
| 377 |
+
{%- set ns.prev_non_tool_role = message['role'] -%}
|
| 378 |
+
{%- endif -%}
|
| 379 |
+
{%- endfor -%}
|
| 380 |
+
|
| 381 |
+
{%- if add_generation_prompt -%}
|
| 382 |
+
{%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
|
| 383 |
+
{{- '<|turn>model\n' -}}
|
| 384 |
+
{%- if not enable_thinking -%}
|
| 385 |
+
{{- '<|channel>thought\n<channel|>' -}}
|
| 386 |
+
{%- endif -%}
|
| 387 |
+
{%- elif ns.prev_message_type == 'tool_response' and enable_thinking -%}
|
| 388 |
+
{{- '<|channel>thought\n' -}}
|
| 389 |
+
{%- endif -%}
|
| 390 |
+
{%- endif -%}
|
gemma4-12b-ltx-v1/config.json
ADDED
|
@@ -0,0 +1,174 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Gemma4UnifiedForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"audio_config": {
|
| 6 |
+
"_name_or_path": "",
|
| 7 |
+
"architectures": null,
|
| 8 |
+
"audio_embed_dim": 640,
|
| 9 |
+
"chunk_size_feed_forward": 0,
|
| 10 |
+
"dtype": "bfloat16",
|
| 11 |
+
"id2label": {
|
| 12 |
+
"0": "LABEL_0",
|
| 13 |
+
"1": "LABEL_1"
|
| 14 |
+
},
|
| 15 |
+
"initializer_range": 0.02,
|
| 16 |
+
"is_encoder_decoder": false,
|
| 17 |
+
"label2id": {
|
| 18 |
+
"LABEL_0": 0,
|
| 19 |
+
"LABEL_1": 1
|
| 20 |
+
},
|
| 21 |
+
"model_type": "gemma4_unified_audio",
|
| 22 |
+
"output_attentions": false,
|
| 23 |
+
"output_hidden_states": false,
|
| 24 |
+
"problem_type": null,
|
| 25 |
+
"return_dict": true,
|
| 26 |
+
"rms_norm_eps": 1e-06
|
| 27 |
+
},
|
| 28 |
+
"audio_token_id": 258881,
|
| 29 |
+
"boa_token_id": 256000,
|
| 30 |
+
"boi_token_id": 255999,
|
| 31 |
+
"dtype": "bfloat16",
|
| 32 |
+
"eoa_token_index": 258883,
|
| 33 |
+
"eoi_token_id": 258882,
|
| 34 |
+
"eos_token_id": [
|
| 35 |
+
1,
|
| 36 |
+
106
|
| 37 |
+
],
|
| 38 |
+
"image_token_id": 258880,
|
| 39 |
+
"initializer_range": 0.02,
|
| 40 |
+
"model_type": "gemma4_unified",
|
| 41 |
+
"text_config": {
|
| 42 |
+
"attention_bias": false,
|
| 43 |
+
"attention_dropout": 0.0,
|
| 44 |
+
"attention_k_eq_v": true,
|
| 45 |
+
"bos_token_id": 2,
|
| 46 |
+
"dtype": "bfloat16",
|
| 47 |
+
"enable_moe_block": false,
|
| 48 |
+
"eos_token_id": 1,
|
| 49 |
+
"final_logit_softcapping": 30.0,
|
| 50 |
+
"global_head_dim": 512,
|
| 51 |
+
"head_dim": 256,
|
| 52 |
+
"hidden_activation": "gelu_pytorch_tanh",
|
| 53 |
+
"hidden_size": 3840,
|
| 54 |
+
"hidden_size_per_layer_input": 0,
|
| 55 |
+
"initializer_range": 0.02,
|
| 56 |
+
"intermediate_size": 15360,
|
| 57 |
+
"layer_types": [
|
| 58 |
+
"sliding_attention",
|
| 59 |
+
"sliding_attention",
|
| 60 |
+
"sliding_attention",
|
| 61 |
+
"sliding_attention",
|
| 62 |
+
"sliding_attention",
|
| 63 |
+
"full_attention",
|
| 64 |
+
"sliding_attention",
|
| 65 |
+
"sliding_attention",
|
| 66 |
+
"sliding_attention",
|
| 67 |
+
"sliding_attention",
|
| 68 |
+
"sliding_attention",
|
| 69 |
+
"full_attention",
|
| 70 |
+
"sliding_attention",
|
| 71 |
+
"sliding_attention",
|
| 72 |
+
"sliding_attention",
|
| 73 |
+
"sliding_attention",
|
| 74 |
+
"sliding_attention",
|
| 75 |
+
"full_attention",
|
| 76 |
+
"sliding_attention",
|
| 77 |
+
"sliding_attention",
|
| 78 |
+
"sliding_attention",
|
| 79 |
+
"sliding_attention",
|
| 80 |
+
"sliding_attention",
|
| 81 |
+
"full_attention",
|
| 82 |
+
"sliding_attention",
|
| 83 |
+
"sliding_attention",
|
| 84 |
+
"sliding_attention",
|
| 85 |
+
"sliding_attention",
|
| 86 |
+
"sliding_attention",
|
| 87 |
+
"full_attention",
|
| 88 |
+
"sliding_attention",
|
| 89 |
+
"sliding_attention",
|
| 90 |
+
"sliding_attention",
|
| 91 |
+
"sliding_attention",
|
| 92 |
+
"sliding_attention",
|
| 93 |
+
"full_attention",
|
| 94 |
+
"sliding_attention",
|
| 95 |
+
"sliding_attention",
|
| 96 |
+
"sliding_attention",
|
| 97 |
+
"sliding_attention",
|
| 98 |
+
"sliding_attention",
|
| 99 |
+
"full_attention",
|
| 100 |
+
"sliding_attention",
|
| 101 |
+
"sliding_attention",
|
| 102 |
+
"sliding_attention",
|
| 103 |
+
"sliding_attention",
|
| 104 |
+
"sliding_attention",
|
| 105 |
+
"full_attention"
|
| 106 |
+
],
|
| 107 |
+
"max_position_embeddings": 262144,
|
| 108 |
+
"model_type": "gemma4_unified_text",
|
| 109 |
+
"moe_intermediate_size": null,
|
| 110 |
+
"num_attention_heads": 16,
|
| 111 |
+
"num_experts": null,
|
| 112 |
+
"num_global_key_value_heads": 1,
|
| 113 |
+
"num_hidden_layers": 48,
|
| 114 |
+
"num_key_value_heads": 8,
|
| 115 |
+
"num_kv_shared_layers": 0,
|
| 116 |
+
"pad_token_id": 0,
|
| 117 |
+
"rms_norm_eps": 1e-06,
|
| 118 |
+
"rope_parameters": {
|
| 119 |
+
"full_attention": {
|
| 120 |
+
"partial_rotary_factor": 0.25,
|
| 121 |
+
"rope_theta": 1000000.0,
|
| 122 |
+
"rope_type": "proportional"
|
| 123 |
+
},
|
| 124 |
+
"sliding_attention": {
|
| 125 |
+
"rope_theta": 10000.0,
|
| 126 |
+
"rope_type": "default"
|
| 127 |
+
}
|
| 128 |
+
},
|
| 129 |
+
"sliding_window": 1024,
|
| 130 |
+
"tie_word_embeddings": true,
|
| 131 |
+
"top_k_experts": null,
|
| 132 |
+
"use_bidirectional_attention": "vision",
|
| 133 |
+
"use_cache": true,
|
| 134 |
+
"use_double_wide_mlp": false,
|
| 135 |
+
"vocab_size": 262144,
|
| 136 |
+
"vocab_size_per_layer_input": 262144
|
| 137 |
+
},
|
| 138 |
+
"tie_word_embeddings": true,
|
| 139 |
+
"transformers_version": "5.10.1",
|
| 140 |
+
"video_token_id": 258884,
|
| 141 |
+
"vision_config": {
|
| 142 |
+
"_name_or_path": "",
|
| 143 |
+
"architectures": null,
|
| 144 |
+
"chunk_size_feed_forward": 0,
|
| 145 |
+
"dtype": "bfloat16",
|
| 146 |
+
"id2label": {
|
| 147 |
+
"0": "LABEL_0",
|
| 148 |
+
"1": "LABEL_1"
|
| 149 |
+
},
|
| 150 |
+
"initializer_range": 0.02,
|
| 151 |
+
"is_encoder_decoder": false,
|
| 152 |
+
"label2id": {
|
| 153 |
+
"LABEL_0": 0,
|
| 154 |
+
"LABEL_1": 1
|
| 155 |
+
},
|
| 156 |
+
"mm_embed_dim": 3840,
|
| 157 |
+
"mm_posemb_size": 1120,
|
| 158 |
+
"model_type": "gemma4_unified_vision",
|
| 159 |
+
"num_soft_tokens": 280,
|
| 160 |
+
"output_attentions": false,
|
| 161 |
+
"output_hidden_states": false,
|
| 162 |
+
"output_proj_dims": 3840,
|
| 163 |
+
"patch_size": 16,
|
| 164 |
+
"pooling_kernel_size": 3,
|
| 165 |
+
"problem_type": null,
|
| 166 |
+
"return_dict": true,
|
| 167 |
+
"rms_norm_eps": 1e-06
|
| 168 |
+
},
|
| 169 |
+
"gemma_version": "gemma4-12b-ltx-v1",
|
| 170 |
+
"quantization": {
|
| 171 |
+
"group_size": 64,
|
| 172 |
+
"bits": 4
|
| 173 |
+
}
|
| 174 |
+
}
|
gemma4-12b-ltx-v1/generation_config.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token_id": 2,
|
| 3 |
+
"do_sample": true,
|
| 4 |
+
"eos_token_id": [
|
| 5 |
+
1,
|
| 6 |
+
106,
|
| 7 |
+
50
|
| 8 |
+
],
|
| 9 |
+
"pad_token_id": 0,
|
| 10 |
+
"suppress_tokens": [
|
| 11 |
+
258883,
|
| 12 |
+
258882
|
| 13 |
+
],
|
| 14 |
+
"temperature": 1.0,
|
| 15 |
+
"top_k": 64,
|
| 16 |
+
"top_p": 0.95,
|
| 17 |
+
"transformers_version": "5.10.1"
|
| 18 |
+
}
|
gemma4-12b-ltx-v1/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:be2d3f3d7f13b61a6ab391981b2c8c98f99135edfbde6adcae32d81f155744dd
|
| 3 |
+
size 6699162168
|
gemma4-12b-ltx-v1/processor_config.json
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"audio_ms_per_token": 40,
|
| 3 |
+
"audio_seq_length": 750,
|
| 4 |
+
"feature_extractor": {
|
| 5 |
+
"audio_samples_per_token": 640,
|
| 6 |
+
"feature_extractor_type": "Gemma4UnifiedAudioFeatureExtractor",
|
| 7 |
+
"feature_size": 640,
|
| 8 |
+
"padding_side": "right",
|
| 9 |
+
"padding_value": 0.0,
|
| 10 |
+
"return_attention_mask": true,
|
| 11 |
+
"sampling_rate": 16000
|
| 12 |
+
},
|
| 13 |
+
"image_processor": {
|
| 14 |
+
"do_convert_rgb": true,
|
| 15 |
+
"do_normalize": false,
|
| 16 |
+
"do_rescale": true,
|
| 17 |
+
"do_resize": true,
|
| 18 |
+
"image_mean": [
|
| 19 |
+
0.0,
|
| 20 |
+
0.0,
|
| 21 |
+
0.0
|
| 22 |
+
],
|
| 23 |
+
"image_processor_type": "Gemma4UnifiedImageProcessor",
|
| 24 |
+
"image_std": [
|
| 25 |
+
1.0,
|
| 26 |
+
1.0,
|
| 27 |
+
1.0
|
| 28 |
+
],
|
| 29 |
+
"max_soft_tokens": 280,
|
| 30 |
+
"patch_size": 16,
|
| 31 |
+
"pooling_kernel_size": 3,
|
| 32 |
+
"resample": 3,
|
| 33 |
+
"rescale_factor": 0.00392156862745098
|
| 34 |
+
},
|
| 35 |
+
"image_seq_length": 280,
|
| 36 |
+
"processor_class": "Gemma4UnifiedProcessor",
|
| 37 |
+
"video_processor": {
|
| 38 |
+
"do_convert_rgb": true,
|
| 39 |
+
"do_normalize": true,
|
| 40 |
+
"do_rescale": true,
|
| 41 |
+
"do_resize": true,
|
| 42 |
+
"do_sample_frames": true,
|
| 43 |
+
"image_mean": [
|
| 44 |
+
0.0,
|
| 45 |
+
0.0,
|
| 46 |
+
0.0
|
| 47 |
+
],
|
| 48 |
+
"image_std": [
|
| 49 |
+
1.0,
|
| 50 |
+
1.0,
|
| 51 |
+
1.0
|
| 52 |
+
],
|
| 53 |
+
"max_soft_tokens": 70,
|
| 54 |
+
"num_frames": 32,
|
| 55 |
+
"patch_size": 16,
|
| 56 |
+
"pooling_kernel_size": 3,
|
| 57 |
+
"resample": 3,
|
| 58 |
+
"rescale_factor": 0.00392156862745098,
|
| 59 |
+
"return_metadata": false,
|
| 60 |
+
"video_processor_type": "Gemma4UnifiedVideoProcessor"
|
| 61 |
+
}
|
| 62 |
+
}
|
gemma4-12b-ltx-v1/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
|
| 3 |
+
size 32169626
|
gemma4-12b-ltx-v1/tokenizer_config.json
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"audio_token": "<|audio|>",
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"boa_token": "<|audio>",
|
| 5 |
+
"boi_token": "<|image>",
|
| 6 |
+
"bos_token": "<bos>",
|
| 7 |
+
"eoa_token": "<audio|>",
|
| 8 |
+
"eoc_token": "<channel|>",
|
| 9 |
+
"eoi_token": "<image|>",
|
| 10 |
+
"eos_token": "<eos>",
|
| 11 |
+
"eot_token": "<turn|>",
|
| 12 |
+
"escape_token": "<|\"|>",
|
| 13 |
+
"etc_token": "<tool_call|>",
|
| 14 |
+
"etd_token": "<tool|>",
|
| 15 |
+
"etr_token": "<tool_response|>",
|
| 16 |
+
"extra_special_tokens": [
|
| 17 |
+
"<|video|>"
|
| 18 |
+
],
|
| 19 |
+
"image_token": "<|image|>",
|
| 20 |
+
"is_local": false,
|
| 21 |
+
"local_files_only": false,
|
| 22 |
+
"mask_token": "<mask>",
|
| 23 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 24 |
+
"model_specific_special_tokens": {
|
| 25 |
+
"audio_token": "<|audio|>",
|
| 26 |
+
"boa_token": "<|audio>",
|
| 27 |
+
"boi_token": "<|image>",
|
| 28 |
+
"eoa_token": "<audio|>",
|
| 29 |
+
"eoc_token": "<channel|>",
|
| 30 |
+
"eoi_token": "<image|>",
|
| 31 |
+
"eot_token": "<turn|>",
|
| 32 |
+
"escape_token": "<|\"|>",
|
| 33 |
+
"etc_token": "<tool_call|>",
|
| 34 |
+
"etd_token": "<tool|>",
|
| 35 |
+
"etr_token": "<tool_response|>",
|
| 36 |
+
"image_token": "<|image|>",
|
| 37 |
+
"soc_token": "<|channel>",
|
| 38 |
+
"sot_token": "<|turn>",
|
| 39 |
+
"stc_token": "<|tool_call>",
|
| 40 |
+
"std_token": "<|tool>",
|
| 41 |
+
"str_token": "<|tool_response>",
|
| 42 |
+
"think_token": "<|think|>"
|
| 43 |
+
},
|
| 44 |
+
"pad_token": "<pad>",
|
| 45 |
+
"padding_side": "left",
|
| 46 |
+
"processor_class": "Gemma4UnifiedProcessor",
|
| 47 |
+
"response_schema": {
|
| 48 |
+
"properties": {
|
| 49 |
+
"content": {
|
| 50 |
+
"type": "string"
|
| 51 |
+
},
|
| 52 |
+
"role": {
|
| 53 |
+
"const": "assistant"
|
| 54 |
+
},
|
| 55 |
+
"thinking": {
|
| 56 |
+
"type": "string"
|
| 57 |
+
},
|
| 58 |
+
"tool_calls": {
|
| 59 |
+
"items": {
|
| 60 |
+
"properties": {
|
| 61 |
+
"function": {
|
| 62 |
+
"properties": {
|
| 63 |
+
"arguments": {
|
| 64 |
+
"additionalProperties": {},
|
| 65 |
+
"type": "object",
|
| 66 |
+
"x-parser": "gemma4-tool-call"
|
| 67 |
+
},
|
| 68 |
+
"name": {
|
| 69 |
+
"type": "string"
|
| 70 |
+
}
|
| 71 |
+
},
|
| 72 |
+
"type": "object",
|
| 73 |
+
"x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
|
| 74 |
+
},
|
| 75 |
+
"type": {
|
| 76 |
+
"const": "function"
|
| 77 |
+
}
|
| 78 |
+
},
|
| 79 |
+
"type": "object"
|
| 80 |
+
},
|
| 81 |
+
"type": "array",
|
| 82 |
+
"x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
|
| 83 |
+
}
|
| 84 |
+
},
|
| 85 |
+
"type": "object",
|
| 86 |
+
"x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
|
| 87 |
+
},
|
| 88 |
+
"response_template": {
|
| 89 |
+
"defaults": {
|
| 90 |
+
"role": "assistant"
|
| 91 |
+
},
|
| 92 |
+
"fields": {
|
| 93 |
+
"content": {
|
| 94 |
+
"close": [
|
| 95 |
+
"<turn|>",
|
| 96 |
+
"<|tool_response>",
|
| 97 |
+
"<eos>"
|
| 98 |
+
],
|
| 99 |
+
"content": "text"
|
| 100 |
+
},
|
| 101 |
+
"thinking": {
|
| 102 |
+
"close": "<channel|>",
|
| 103 |
+
"content": "text",
|
| 104 |
+
"open": "<|channel>thought\n"
|
| 105 |
+
},
|
| 106 |
+
"tool_calls": {
|
| 107 |
+
"close": "<tool_call|>",
|
| 108 |
+
"content": "json",
|
| 109 |
+
"content_args": {
|
| 110 |
+
"string_delims": [
|
| 111 |
+
[
|
| 112 |
+
"<|\"|>",
|
| 113 |
+
"<|\"|>"
|
| 114 |
+
]
|
| 115 |
+
],
|
| 116 |
+
"unquoted_keys": true
|
| 117 |
+
},
|
| 118 |
+
"open_pattern": "<\\|tool_call>call:(?P<name>\\w+)",
|
| 119 |
+
"repeats": true,
|
| 120 |
+
"transform": {
|
| 121 |
+
"function": {
|
| 122 |
+
"arguments": "{content}",
|
| 123 |
+
"name": "{name}"
|
| 124 |
+
},
|
| 125 |
+
"type": "function"
|
| 126 |
+
}
|
| 127 |
+
}
|
| 128 |
+
},
|
| 129 |
+
"start_anchor": [
|
| 130 |
+
"<|turn>model\n",
|
| 131 |
+
"<tool_response|>"
|
| 132 |
+
]
|
| 133 |
+
},
|
| 134 |
+
"soc_token": "<|channel>",
|
| 135 |
+
"sot_token": "<|turn>",
|
| 136 |
+
"stc_token": "<|tool_call>",
|
| 137 |
+
"std_token": "<|tool>",
|
| 138 |
+
"str_token": "<|tool_response>",
|
| 139 |
+
"think_token": "<|think|>",
|
| 140 |
+
"tokenizer_class": "GemmaTokenizer",
|
| 141 |
+
"unk_token": "<unk>"
|
| 142 |
+
}
|
ltx-acceptable-use-policy-snapshot-2026-08-12.pdf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:054b9e6e9d4b8632dd13abe64faded682c344cf9e514813b8d262d0ea94329f1
|
| 3 |
+
size 110423
|
quantize_config.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"quantization": {
|
| 3 |
+
"bits": 4,
|
| 4 |
+
"group_size": 64,
|
| 5 |
+
"only_transformer_blocks": true
|
| 6 |
+
}
|
| 7 |
+
}
|
spatial_upscaler_x2_v1_1.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:755f69dcb8a3f9589045925eb4a2237e9969f9c026a016393bc395e65ebd5788
|
| 3 |
+
size 995745061
|
spatial_upscaler_x2_v1_1_config.json
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": {
|
| 3 |
+
"_class_name": "LatentUpsampler",
|
| 4 |
+
"in_channels": 128,
|
| 5 |
+
"mid_channels": 1024,
|
| 6 |
+
"num_blocks_per_stage": 4,
|
| 7 |
+
"dims": 3,
|
| 8 |
+
"spatial_upsample": true,
|
| 9 |
+
"temporal_upsample": false,
|
| 10 |
+
"spatial_scale": 2.0,
|
| 11 |
+
"rational_resampler": false
|
| 12 |
+
}
|
| 13 |
+
}
|
split_model.json
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "split",
|
| 3 |
+
"model_version": "2.5.0",
|
| 4 |
+
"components": [
|
| 5 |
+
"connector",
|
| 6 |
+
"vae_decoder",
|
| 7 |
+
"vae_encoder",
|
| 8 |
+
"audio_vae",
|
| 9 |
+
"vocoder",
|
| 10 |
+
"spatial_upscaler_x2_v1_1",
|
| 11 |
+
"temporal_upscaler_x2_v1_0"
|
| 12 |
+
],
|
| 13 |
+
"transformer_variants": [
|
| 14 |
+
"distilled",
|
| 15 |
+
"dev"
|
| 16 |
+
],
|
| 17 |
+
"text_encoder": "gemma4-12b-ltx-v1",
|
| 18 |
+
"source": "mlx-community/ltx-2.5-mlx",
|
| 19 |
+
"quantized": true,
|
| 20 |
+
"quantization_bits": 4
|
| 21 |
+
}
|
temporal_upscaler_x2_v1_0.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:982bc28b455ed22de1e39a9fcf5213299de4e15f948ae3bd2cee65550e019628
|
| 3 |
+
size 261945581
|
temporal_upscaler_x2_v1_0_config.json
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": {
|
| 3 |
+
"_class_name": "LatentUpsampler",
|
| 4 |
+
"in_channels": 128,
|
| 5 |
+
"mid_channels": 512,
|
| 6 |
+
"num_blocks_per_stage": 4,
|
| 7 |
+
"dims": 3,
|
| 8 |
+
"spatial_upsample": false,
|
| 9 |
+
"temporal_upsample": true,
|
| 10 |
+
"spatial_scale": 1.0,
|
| 11 |
+
"rational_resampler": true
|
| 12 |
+
}
|
| 13 |
+
}
|
transformer-dev.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bc27304a10eec28c78b2d56854aed0c24e2594c8fd99a706334951aa6b83bd0a
|
| 3 |
+
size 11320068903
|
transformer-distilled.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:de348de0c038e7d430fbae6ebccf2badcfd861088a64ab20f3851cdbde2af635
|
| 3 |
+
size 11320068903
|
vae_decoder.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:edcc3294126fc3ec9f11e83088cc39026b8efdce0fd17987f2c90661da2e0ddb
|
| 3 |
+
size 814349515
|
vae_encoder.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:101970a6cdc83345fa591027d0e5106ae53a1b291220f76640b58c2177e407a5
|
| 3 |
+
size 637885335
|
vocoder.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7316de9d2700a91497a833cb572ded629b0d2709b964917525acc22c7439a3f8
|
| 3 |
+
size 258314115
|