Add library name, link to paper, link to Github repository
Browse filesThis PR adds the `library_name: transformers` to the metadata, as well as adding a link to the paper and Github repository to the content section.
README.md
CHANGED
|
@@ -3,7 +3,9 @@ base_model: inceptionai/jais-family-13b
|
|
| 3 |
language:
|
| 4 |
- ar
|
| 5 |
- en
|
| 6 |
-
|
|
|
|
|
|
|
| 7 |
tags:
|
| 8 |
- Arabic
|
| 9 |
- English
|
|
@@ -11,11 +13,9 @@ tags:
|
|
| 11 |
- Decoder
|
| 12 |
- causal-lm
|
| 13 |
- jais-family
|
| 14 |
-
license: apache-2.0
|
| 15 |
-
pipeline_tag: text-generation
|
| 16 |
---
|
| 17 |
-
# Jais Family Model Card
|
| 18 |
|
|
|
|
| 19 |
|
| 20 |
The Jais family of models is a comprehensive series of bilingual English-Arabic large language models (LLMs). These models are optimized to excel in Arabic while having strong English capabilities. We release two variants of foundation models that include:
|
| 21 |
|
|
@@ -35,6 +35,8 @@ We hope this extensive release will accelerate research in Arabic NLP, and enabl
|
|
| 35 |
- **Model Sizes:** 590M, 1.3B, 2.7B, 6.7B, 7B, 13B, 30B, 70B.
|
| 36 |
- **Demo:** [Access the live demo here](https://arabic-gpt.ai/)
|
| 37 |
- **License:** Apache 2.0
|
|
|
|
|
|
|
| 38 |
|
| 39 |
| **Pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
|
| 40 |
|:---------------------|:--------|:-------|:-------|
|
|
@@ -75,8 +77,14 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
| 75 |
|
| 76 |
model_path = "inceptionai/jais-family-13b-chat"
|
| 77 |
|
| 78 |
-
prompt_eng = "### Instruction:Your name is 'Jais', and you are named after Jebel Jais, the highest mountain in UAE. You were made by 'Inception' in the UAE. You are a helpful, respectful, and honest assistant. Always answer as helpfully as possible, while being safe. Complete the conversation between [|Human|] and [|AI|]:
|
| 79 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 82 |
|
|
@@ -222,17 +230,6 @@ We conducted a comprehensive evaluation of Jais models focusing on both English
|
|
| 222 |
| jais-family-1p3b-chat | 42.7 | 42.2 | 30.1 | 33.6 | 40.6 | 34.1 | 41.2 | 43 | 63.6 | 69.3 | 44.9 | 31.6 | 28 | 45.6 | 50.4 |
|
| 223 |
| jais-family-590m-chat | 37.8 | 39.1 | 28 |29.5 | 33.1 | 30.8 | 36.4 | 30.3 | 57.8 | 57.2 | 40.5 | 25.9 | 26.8 | 44.5 | 49.3 |
|
| 224 |
|
| 225 |
-
|
| 226 |
-
|
| 227 |
-
| **Adapted Models** | Avg | ArabicMMLU*| MMLU | EXAMS*| LitQA*| agqa | agrc | Hellaswag | PIQA | BoolQA | Situated QA | ARC-C | OpenBookQA | TruthfulQA | CrowS-Pairs |
|
| 228 |
-
|--------------------------|-------|------------|-------|-------|-------|------|------|------------|------|--------|-------------|-------|------------|------------|-------------|
|
| 229 |
-
| jais-adapted-70b | 51.5 | 55.9 | 36.8 | 42.3 | 58.3 | 48.6 | 54 | 61.5 | 68.4 | 68.4 | 42.1 | 42.6 | 33 | 50.2 | 58.3 |
|
| 230 |
-
| jais-adapted-13b | 46.6 | 44.7 | 30.6 | 37.7 | 54.3 | 43.8 | 48.3 | 54.9 | 67.1 | 64.5 | 40.6 | 36.1 | 32 | 43.6 | 54.00 |
|
| 231 |
-
| jais-adapted-7b | 42.0 | 35.9 | 28.9 | 36.7 | 46.3 | 34.1 | 40.3 | 45 | 61.3 | 63.8 | 38.1 | 29.7 | 30.2 | 44.3 | 53.6 |
|
| 232 |
-
| jais-adapted-70b-chat | 52.9 | 66.8 | 34.6 | 42.5 | 62.9 | 36.8 | 48.6 | 64.5 | 69.7 | 82.8 | 49.3 | 44.2 | 32.2 | 53.3 | 52.4 |
|
| 233 |
-
| jais-adapted-13b-chat | 50.3 | 59.0 | 31.7 | 37.5 | 56.6 | 41.9 | 51.7 | 58.8 | 67.1 | 78.2 | 45.9 | 41 | 34.2 | 48.3 | 52.1 |
|
| 234 |
-
| jais-adapted-7b-chat | 46.1 | 51.3 | 30 | 37 | 48 | 36.8 | 48.6 | 51.1 | 62.9 | 72.4 | 41.3 | 34.6 | 30.4 | 48.6 | 51.8 |
|
| 235 |
-
|
| 236 |
</div>
|
| 237 |
|
| 238 |
### English evaluation results:
|
|
@@ -258,23 +255,8 @@ We conducted a comprehensive evaluation of Jais models focusing on both English
|
|
| 258 |
|
| 259 |
</div>
|
| 260 |
|
| 261 |
-
<div class="table-container">
|
| 262 |
-
|
| 263 |
-
|**Adapted Models**| Avg | MMLU | RACE | Hellaswag | PIQA | BoolQA | SIQA | ARC-Challenge | OpenBookQA | Winogrande | TruthfulQA | CrowS-Pairs |
|
| 264 |
-
|--------------------------|----------|------|------|-----------|------|--------|------|---------------|------------|------------|----------------|-------------|
|
| 265 |
-
| jais-adapted-70b | 60.1 | 40.4 | 38.5 | 81.2 | 81.1 | 81.2 | 48.1 | 50.4 | 45 | 75.8 | 45.7 | 74 |
|
| 266 |
-
| jais-adapted-13b | 56 | 33.8 | 39.5 | 76.5 | 78.6 | 77.8 | 44.6 | 45.9 | 44.4 | 71.4 | 34.6 | 69 |
|
| 267 |
-
| jais-adapted-7b | 55.7 | 32.2 | 39.8 | 75.3 | 78.8 | 75.7 | 45.2 | 42.8 | 43 | 68 | 38.3 | 73.1 |
|
| 268 |
-
| jais-adapted-70b-chat | 61.4 | 38.7 | 42.9 | 82.7 | 81.2 | 89.6 | 52.9 | 54.9 | 44.4 | 75.7 | 44 | 68.8 |
|
| 269 |
-
| jais-adapted-13b-chat | 58.5 | 34.9 | 42.4 | 79.6 | 79.7 | 88.2 | 50.5 | 48.5 | 42.4 | 70.3 | 42.2 | 65.1 |
|
| 270 |
-
| jais-adapted-7b-chat | 58.5 | 33.8 | 43.9 | 77.8 | 79.4 | 87.1 | 47.3 | 46.9 | 43.4 | 69.9 | 42 | 72.4 |
|
| 271 |
-
|
| 272 |
-
</div>
|
| 273 |
-
|
| 274 |
-
|
| 275 |
### GPT-4 evaluation
|
| 276 |
|
| 277 |
-
|
| 278 |
In addition to the LM-Harness evaluation, we conducted an open-ended generation evaluation using GPT-4-as-a-judge. We measured pairwise win-rates of model responses in both Arabic and English on a fixed set of 80 prompts from the Vicuna test set.
|
| 279 |
English prompts were translated to Arabic by our in-house linguists.
|
| 280 |
In the following, we compare the models in this release of the jais family against previously released versions:
|
|
@@ -312,72 +294,4 @@ We release the Jais family of models under a full open-source license. We welcom
|
|
| 312 |
- Mechanistic interpretability analyses on cultural alignment in bilingual pre-trained and adapted pre-trained models.
|
| 313 |
- Quantitative studies of Arabic cultural and linguistic phenomena.
|
| 314 |
|
| 315 |
-
- **Commercial Use**: Jais 30B and 70B chat models are well-suited for direct use in chat applications with appropriate prompting or for further fine-tuning on
|
| 316 |
-
- Development of chat assistants for Arabic-speaking users.
|
| 317 |
-
- Sentiment analysis to gain insights into local markets and customer trends.
|
| 318 |
-
- Summarization of bilingual Arabic-English documents.
|
| 319 |
-
|
| 320 |
-
Audiences that we hope will benefit from our model:
|
| 321 |
-
- **Academics**: For those researching Arabic Natural Language Processing.
|
| 322 |
-
- **Businesses**: Companies targeting Arabic-speaking audiences.
|
| 323 |
-
- **Developers**: Those integrating Arabic language capabilities in applications.
|
| 324 |
-
|
| 325 |
-
### Out-of-Scope Use
|
| 326 |
-
|
| 327 |
-
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 328 |
-
|
| 329 |
-
While the Jais family of models are powerful Arabic and English bilingual models, it's essential to understand their limitations
|
| 330 |
-
and the potential of misuse. It is prohibited to use the model in any manner that violates applicable laws or regulations.
|
| 331 |
-
|
| 332 |
-
The following are some example scenarios where the model should not be used.
|
| 333 |
-
|
| 334 |
-
- **Malicious Use**: The model should not be used to generate harmful, misleading, or inappropriate content. Thisincludes but is not limited to:
|
| 335 |
-
- Generating or promoting hate speech, violence, or discrimination.
|
| 336 |
-
- Spreading misinformation or fake news.
|
| 337 |
-
- Engaging in or promoting illegal activities.
|
| 338 |
-
|
| 339 |
-
- **Sensitive Information**: The model should not be used to handle or generate personal, confidential, or sensitive information.
|
| 340 |
-
|
| 341 |
-
- **Generalization Across All Languages**: Jais family of models are bilingual and optimized for Arabic and English. They should not be presumed to have equal proficiency in other languages or dialects.
|
| 342 |
-
|
| 343 |
-
- **High-Stakes Decisions**: The model should not be used to make high-stakes decisions without human oversight. This includes medical, legal, financial, or safety-critical decisions.
|
| 344 |
-
|
| 345 |
-
## Bias, Risks, and Limitations
|
| 346 |
-
|
| 347 |
-
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 348 |
-
|
| 349 |
-
The Jais family is trained on publicly available data which was in part curated by Inception. We have employed different techniques to reduce bias in the model. While efforts have been made to minimize biases, it is likely that the model, as with all LLM models, will exhibit some bias.
|
| 350 |
-
|
| 351 |
-
The fine-tuned variants are trained as an AI assistant for Arabic and English speakers. Chat models are limited to produce responses for queries in these two languages and may not produce appropriate responses to other language queries.
|
| 352 |
-
|
| 353 |
-
By using Jais, you acknowledge and accept that, as with any large language model, it may generate incorrect, misleading and/or offensive information or content. The information is not intended as advice and should not be relied upon in any way, nor are we responsible for any of the content or consequences resulting from its use. We are continuously working to develop models with greater capabilities, and as such, welcome any feedback on the model.
|
| 354 |
-
|
| 355 |
-
Copyright Inception Institute of Artificial Intelligence Ltd. JAIS is made available under the Apache License, Version 2.0 (the “License”). You shall not use JAIS except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0.
|
| 356 |
-
|
| 357 |
-
Unless required by applicable law or agreed to in writing, JAIS is distributed on an AS IS basis, without warranties or conditions of any kind, either express or implied. Please see the terms of the License for the specific language permissions and limitations under the License.
|
| 358 |
-
|
| 359 |
-
#### Summary
|
| 360 |
-
|
| 361 |
-
We release the Jais family of Arabic and English bilingual models. The wide range of pre-trained model sizes, the recipe for adapting English-centric models to Arabic, and the fine-tuning of all sizes unlocks numerous use cases commercially and academically in the Arabic setting.
|
| 362 |
-
|
| 363 |
-
Through this release, we aim to make LLMs more accessible to Arabic NLP researchers and companies, offering native Arabic models that provide better cultural understanding than English centric ones. The strategies we employ for pre-training, fine-tuning and adaptation to Arabic are extensible to other low and medium resource languages, paving the way for language-focused and accessible models that cater to local contexts.
|
| 364 |
-
|
| 365 |
-
#### Citation info
|
| 366 |
-
|
| 367 |
-
```bibtex
|
| 368 |
-
@misc{sengupta2023jais,
|
| 369 |
-
title={Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models},
|
| 370 |
-
author={Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, Alham Fikri Aji, Zhiqiang Shen, Zhengzhong Liu, Natalia Vassilieva, Joel Hestness, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Hector Xuguang Ren, Preslav Nakov, Timothy Baldwin and Eric Xing},
|
| 371 |
-
year={2023},
|
| 372 |
-
eprint={2308.16149},
|
| 373 |
-
archivePrefix={arXiv},
|
| 374 |
-
primaryClass={cs.CL}
|
| 375 |
-
}
|
| 376 |
-
|
| 377 |
-
@article{jaisfamilymodelcard,
|
| 378 |
-
title={Jais Family Model Card},
|
| 379 |
-
author={Inception},
|
| 380 |
-
year={2024},
|
| 381 |
-
url = {https://huggingface.co/inceptionai/jais-family-30b-16k-chat/blob/main/README.md}
|
| 382 |
-
}
|
| 383 |
-
```
|
|
|
|
| 3 |
language:
|
| 4 |
- ar
|
| 5 |
- en
|
| 6 |
+
license: apache-2.0
|
| 7 |
+
pipeline_tag: text-generation
|
| 8 |
+
library_name: transformers
|
| 9 |
tags:
|
| 10 |
- Arabic
|
| 11 |
- English
|
|
|
|
| 13 |
- Decoder
|
| 14 |
- causal-lm
|
| 15 |
- jais-family
|
|
|
|
|
|
|
| 16 |
---
|
|
|
|
| 17 |
|
| 18 |
+
# Jais Family Model Card
|
| 19 |
|
| 20 |
The Jais family of models is a comprehensive series of bilingual English-Arabic large language models (LLMs). These models are optimized to excel in Arabic while having strong English capabilities. We release two variants of foundation models that include:
|
| 21 |
|
|
|
|
| 35 |
- **Model Sizes:** 590M, 1.3B, 2.7B, 6.7B, 7B, 13B, 30B, 70B.
|
| 36 |
- **Demo:** [Access the live demo here](https://arabic-gpt.ai/)
|
| 37 |
- **License:** Apache 2.0
|
| 38 |
+
- **Paper:** [Chem42: a Family of chemical Language Models for Target-aware Ligand Generation](https://huggingface.co/papers/2503.16563)
|
| 39 |
+
- **Code:** See https://github.com/inception-ai/JAIS
|
| 40 |
|
| 41 |
| **Pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
|
| 42 |
|:---------------------|:--------|:-------|:-------|
|
|
|
|
| 77 |
|
| 78 |
model_path = "inceptionai/jais-family-13b-chat"
|
| 79 |
|
| 80 |
+
prompt_eng = "### Instruction:Your name is 'Jais', and you are named after Jebel Jais, the highest mountain in UAE. You were made by 'Inception' in the UAE. You are a helpful, respectful, and honest assistant. Always answer as helpfully as possible, while being safe. Complete the conversation between [|Human|] and [|AI|]:
|
| 81 |
+
### Input: [|Human|] {Question}
|
| 82 |
+
[|AI|]
|
| 83 |
+
### Response :"
|
| 84 |
+
prompt_ar = "### Instruction:اسمك \"جيس\" وسميت على اسم جبل جيس اعلى جبل في الامارات. تم بنائك بواسطة Inception في الإمارات. أنت مساعد مفيد ومحترم وصادق. أجب دائمًا بأكبر قدر ممكن من المساعدة، مع الحفاظ على البقاء آمناً. أكمل المحادثة بين [|Human|] و[|AI|] :
|
| 85 |
+
### Input:[|Human|] {Question}
|
| 86 |
+
[|AI|]
|
| 87 |
+
### Response :"
|
| 88 |
|
| 89 |
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 90 |
|
|
|
|
| 230 |
| jais-family-1p3b-chat | 42.7 | 42.2 | 30.1 | 33.6 | 40.6 | 34.1 | 41.2 | 43 | 63.6 | 69.3 | 44.9 | 31.6 | 28 | 45.6 | 50.4 |
|
| 231 |
| jais-family-590m-chat | 37.8 | 39.1 | 28 |29.5 | 33.1 | 30.8 | 36.4 | 30.3 | 57.8 | 57.2 | 40.5 | 25.9 | 26.8 | 44.5 | 49.3 |
|
| 232 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 233 |
</div>
|
| 234 |
|
| 235 |
### English evaluation results:
|
|
|
|
| 255 |
|
| 256 |
</div>
|
| 257 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 258 |
### GPT-4 evaluation
|
| 259 |
|
|
|
|
| 260 |
In addition to the LM-Harness evaluation, we conducted an open-ended generation evaluation using GPT-4-as-a-judge. We measured pairwise win-rates of model responses in both Arabic and English on a fixed set of 80 prompts from the Vicuna test set.
|
| 261 |
English prompts were translated to Arabic by our in-house linguists.
|
| 262 |
In the following, we compare the models in this release of the jais family against previously released versions:
|
|
|
|
| 294 |
- Mechanistic interpretability analyses on cultural alignment in bilingual pre-trained and adapted pre-trained models.
|
| 295 |
- Quantitative studies of Arabic cultural and linguistic phenomena.
|
| 296 |
|
| 297 |
+
- **Commercial Use**: Jais 30B and 70B chat models are well-suited for direct use in chat applications with appropriate prompting or for further fine-tuning on
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|