Hugging Face's logo

dicta-il
/
dictabert-joint

Feature Extraction
Transformers
PyTorch
Safetensors
Hebrew
bert
custom_code
text-embeddings-inference
Model card Files
xet
 Community
6

DictaBERT: A State-of-the-Art BERT Suite for Modern Hebrew

State-of-the-art language model for Hebrew, released here.

This is the fine-tuned model for the joint parsing of the following tasks:

  • Prefix Segmentation
  • Morphological Disabmgiuation
  • Lexicographical Analysis (Lemmatization)
  • Syntactical Parsing (Dependency-Tree)
  • Named-Entity Recognition

A live demo of the model with instant visualization of the syntax tree can be found here.

For a faster model, you can use the equivalent bert-tiny model for this task here.

For the bert-base models for other tasks, see here.

The model currently supports 3 types of output:

  1. JSON: The model returns a JSON object for each sentence in the input, where for each sentence we have the sentence text, the NER entities, and the list of tokens. For each token we include the output from each of the tasks.

    model.predict(..., output_style='json')

  2. UD: The model returns the full UD output for each sentence, according to the style of the Hebrew UD Treebank. 

    model.predict(..., output_style='ud')

  3. UD, in the style of IAHLT: This model returns the full UD output, with slight modifications to match the style of IAHLT. This differences are mostly granularity of some dependency relations, how the suffix of a word is broken up, and implicit definite articles. The actual tagging behavior doesn't change.

    model.predict(..., output_style='iahlt_ud')

If you only need the output for one of the tasks, you can tell the model to not initialize some of the heads, for example:

model = AutoModel.from_pretrained('dicta-il/dictabert-joint', trust_remote_code=True, do_lex=False)

The list of options are: do_lex, do_syntax, do_ner, do_prefix, do_morph.

Sample usage:

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('dicta-il/dictabert-joint')
model = AutoModel.from_pretrained('dicta-il/dictabert-joint', trust_remote_code=True)

model.eval()

sentence = 'ื‘ืฉื ืช 1948 ื”ืฉืœื™ื ืืคืจื™ื ืงื™ืฉื•ืŸ ืืช ืœื™ืžื•ื“ื™ื• ื‘ืคื™ืกื•ืœ ืžืชื›ืช ื•ื‘ืชื•ืœื“ื•ืช ื”ืืžื ื•ืช ื•ื”ื—ืœ ืœืคืจืกื ืžืืžืจื™ื ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื'
print(model.predict([sentence], tokenizer, output_style='json')) # see below for other return formats

Output:

[
  {
    "text": "ื‘ืฉื ืช 1948 ื”ืฉืœื™ื ืืคืจื™ื ืงื™ืฉื•ืŸ ืืช ืœื™ืžื•ื“ื™ื• ื‘ืคื™ืกื•ืœ ืžืชื›ืช ื•ื‘ืชื•ืœื“ื•ืช ื”ืืžื ื•ืช ื•ื”ื—ืœ ืœืคืจืกื ืžืืžืจื™ื ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื",
    "tokens": [
      {
        "token": "ื‘ืฉื ืช",
        "syntax": {
          "word": "ื‘ืฉื ืช",
          "dep_head_idx": 2,
          "dep_func": "obl",
          "dep_head": "ื”ืฉืœื™ื"
        },
        "seg": [
          "ื‘",
          "ืฉื ืช"
        ],
        "lex": "ืฉื ื”",
        "morph": {
          "token": "ื‘ืฉื ืช",
          "pos": "NOUN",
          "feats": {
            "Gender": "Fem",
            "Number": "Sing"
          },
          "prefixes": [
            "ADP"
          ],
          "suffix": false
        }
      },
      {
        "token": "1948",
        "syntax": {
          "word": "1948",
          "dep_head_idx": 0,
          "dep_func": "compound",
          "dep_head": "ื‘ืฉื ืช"
        },
        "seg": [
          "1948"
        ],
        "lex": "1948",
        "morph": {
          "token": "1948",
          "pos": "NUM",
          "feats": {},
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ื”ืฉืœื™ื",
        "syntax": {
          "word": "ื”ืฉืœื™ื",
          "dep_head_idx": -1,
          "dep_func": "root",
          "dep_head": "ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื"
        },
        "seg": [
          "ื”ืฉืœื™ื"
        ],
        "lex": "ื”ืฉืœื™ื",
        "morph": {
          "token": "ื”ืฉืœื™ื",
          "pos": "VERB",
          "feats": {
            "Gender": "Masc",
            "Number": "Sing",
            "Person": "3",
            "Tense": "Past"
          },
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ืืคืจื™ื",
        "syntax": {
          "word": "ืืคืจื™ื",
          "dep_head_idx": 2,
          "dep_func": "nsubj",
          "dep_head": "ื”ืฉืœื™ื"
        },
        "seg": [
          "ืืคืจื™ื"
        ],
        "lex": "ืืคืจื™ื",
        "morph": {
          "token": "ืืคืจื™ื",
          "pos": "PROPN",
          "feats": {},
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ืงื™ืฉื•ืŸ",
        "syntax": {
          "word": "ืงื™ืฉื•ืŸ",
          "dep_head_idx": 3,
          "dep_func": "flat",
          "dep_head": "ืืคืจื™ื"
        },
        "seg": [
          "ืงื™ืฉื•ืŸ"
        ],
        "lex": "ืงื™ืฉื•ืŸ",
        "morph": {
          "token": "ืงื™ืฉื•ืŸ",
          "pos": "PROPN",
          "feats": {},
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ืืช",
        "syntax": {
          "word": "ืืช",
          "dep_head_idx": 6,
          "dep_func": "case",
          "dep_head": "ืœื™ืžื•ื“ื™ื•"
        },
        "seg": [
          "ืืช"
        ],
        "lex": "ืืช",
        "morph": {
          "token": "ืืช",
          "pos": "ADP",
          "feats": {},
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ืœื™ืžื•ื“ื™ื•",
        "syntax": {
          "word": "ืœื™ืžื•ื“ื™ื•",
          "dep_head_idx": 2,
          "dep_func": "obj",
          "dep_head": "ื”ืฉืœื™ื"
        },
        "seg": [
          "ืœื™ืžื•ื“ื™ื•"
        ],
        "lex": "ืœื™ืžื•ื“",
        "morph": {
          "token": "ืœื™ืžื•ื“ื™ื•",
          "pos": "NOUN",
          "feats": {
            "Gender": "Masc",
            "Number": "Plur"
          },
          "prefixes": [],
          "suffix": "PRON",
          "suffix_feats": {
            "Gender": "Masc",
            "Number": "Sing",
            "Person": "3"
          }
        }
      },
      {
        "token": "ื‘ืคื™ืกื•ืœ",
        "syntax": {
          "word": "ื‘ืคื™ืกื•ืœ",
          "dep_head_idx": 6,
          "dep_func": "nmod",
          "dep_head": "ืœื™ืžื•ื“ื™ื•"
        },
        "seg": [
          "ื‘",
          "ืคื™ืกื•ืœ"
        ],
        "lex": "ืคื™ืกื•ืœ",
        "morph": {
          "token": "ื‘ืคื™ืกื•ืœ",
          "pos": "NOUN",
          "feats": {
            "Gender": "Masc",
            "Number": "Sing"
          },
          "prefixes": [
            "ADP"
          ],
          "suffix": false
        }
      },
      {
        "token": "ืžืชื›ืช",
        "syntax": {
          "word": "ืžืชื›ืช",
          "dep_head_idx": 7,
          "dep_func": "compound",
          "dep_head": "ื‘ืคื™ืกื•ืœ"
        },
        "seg": [
          "ืžืชื›ืช"
        ],
        "lex": "ืžืชื›ืช",
        "morph": {
          "token": "ืžืชื›ืช",
          "pos": "NOUN",
          "feats": {
            "Gender": "Fem",
            "Number": "Sing"
          },
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ื•ื‘ืชื•ืœื“ื•ืช",
        "syntax": {
          "word": "ื•ื‘ืชื•ืœื“ื•ืช",
          "dep_head_idx": 7,
          "dep_func": "conj",
          "dep_head": "ื‘ืคื™ืกื•ืœ"
        },
        "seg": [
          "ื•ื‘",
          "ืชื•ืœื“ื•ืช"
        ],
        "lex": "ืชื•ืœื“ื”",
        "morph": {
          "token": "ื•ื‘ืชื•ืœื“ื•ืช",
          "pos": "NOUN",
          "feats": {
            "Gender": "Fem",
            "Number": "Plur"
          },
          "prefixes": [
            "CCONJ",
            "ADP"
          ],
          "suffix": false
        }
      },
      {
        "token": "ื”ืืžื ื•ืช",
        "syntax": {
          "word": "ื”ืืžื ื•ืช",
          "dep_head_idx": 9,
          "dep_func": "compound",
          "dep_head": "ื•ื‘ืชื•ืœื“ื•ืช"
        },
        "seg": [
          "ื”",
          "ืืžื ื•ืช"
        ],
        "lex": "ืื•ืžื ื•ืช",
        "morph": {
          "token": "ื”ืืžื ื•ืช",
          "pos": "NOUN",
          "feats": {
            "Gender": "Fem",
            "Number": "Sing"
          },
          "prefixes": [
            "DET"
          ],
          "suffix": false
        }
      },
      {
        "token": "ื•ื”ื—ืœ",
        "syntax": {
          "word": "ื•ื”ื—ืœ",
          "dep_head_idx": 2,
          "dep_func": "conj",
          "dep_head": "ื”ืฉืœื™ื"
        },
        "seg": [
          "ื•",
          "ื”ื—ืœ"
        ],
        "lex": "ื”ื—ืœ",
        "morph": {
          "token": "ื•ื”ื—ืœ",
          "pos": "VERB",
          "feats": {
            "Gender": "Masc",
            "Number": "Sing",
            "Person": "3",
            "Tense": "Past"
          },
          "prefixes": [
            "CCONJ"
          ],
          "suffix": false
        }
      },
      {
        "token": "ืœืคืจืกื",
        "syntax": {
          "word": "ืœืคืจืกื",
          "dep_head_idx": 11,
          "dep_func": "xcomp",
          "dep_head": "ื•ื”ื—ืœ"
        },
        "seg": [
          "ืœืคืจืกื"
        ],
        "lex": "ืคืจืกื",
        "morph": {
          "token": "ืœืคืจืกื",
          "pos": "VERB",
          "feats": {},
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ืžืืžืจื™ื",
        "syntax": {
          "word": "ืžืืžืจื™ื",
          "dep_head_idx": 12,
          "dep_func": "obj",
          "dep_head": "ืœืคืจืกื"
        },
        "seg": [
          "ืžืืžืจื™ื"
        ],
        "lex": "ืžืืžืจ",
        "morph": {
          "token": "ืžืืžืจื™ื",
          "pos": "NOUN",
          "feats": {
            "Gender": "Masc",
            "Number": "Plur"
          },
          "prefixes": [],
          "suffix": false
        }
      },
      {
        "token": "ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื",
        "syntax": {
          "word": "ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื",
          "dep_head_idx": 13,
          "dep_func": "amod",
          "dep_head": "ืžืืžืจื™ื"
        },
        "seg": [
          "ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื"
        ],
        "lex": "ื”ื•ืžื•ืจื™ืกื˜ื™",
        "morph": {
          "token": "ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื",
          "pos": "ADJ",
          "feats": {
            "Gender": "Masc",
            "Number": "Plur"
          },
          "prefixes": [],
          "suffix": false
        }
      }
    ],
    "root_idx": 2,
    "ner_entities": [
      {
        "phrase": "1948",
        "label": "TIMEX"
      },
      {
        "phrase": "ืืคืจื™ื ืงื™ืฉื•ืŸ",
        "label": "PER"
      }
    ]
  }
]

You can also choose to get your response in UD format:

sentence = 'ื‘ืฉื ืช 1948 ื”ืฉืœื™ื ืืคืจื™ื ืงื™ืฉื•ืŸ ืืช ืœื™ืžื•ื“ื™ื• ื‘ืคื™ืกื•ืœ ืžืชื›ืช ื•ื‘ืชื•ืœื“ื•ืช ื”ืืžื ื•ืช ื•ื”ื—ืœ ืœืคืจืกื ืžืืžืจื™ื ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื'
print(model.predict([sentence], tokenizer, output_style='ud'))

Results:

[
  [
    "# sent_id = 1",
    "# text = ื‘ืฉื ืช 1948 ื”ืฉืœื™ื ืืคืจื™ื ืงื™ืฉื•ืŸ ืืช ืœื™ืžื•ื“ื™ื• ื‘ืคื™ืกื•ืœ ืžืชื›ืช ื•ื‘ืชื•ืœื“ื•ืช ื”ืืžื ื•ืช ื•ื”ื—ืœ ืœืคืจืกื ืžืืžืจื™ื ื”ื•ืžื•ืจื™ืกื˜ื™ื™ื",
    "1-2\tื‘ืฉื ืช\t_\t_\t_\t_\t_\t_\t_\t_",
    "1\tื‘\tื‘\tADP\tADP\t_\t2\tcase\t_\t_",
    "2\tืฉื ืช\tืฉื ื”\tNOUN\tNOUN\tGender=Fem|Number=Sing\t4\tobl\t_\t_",
    "3\t1948\t1948\tNUM\tNUM\t\t2\tcompound:smixut\t_\t_",
    "4\tื”ืฉืœื™ื\tื”ืฉืœื™ื\tVERB\tVERB\tGender=Masc|Number=Sing|Person=3|Tense=Past\t0\troot\t_\t_",
    "5\tืืคืจื™ื\tืืคืจื™ื\tPROPN\tPROPN\t\t4\tnsubj\t_\t_",
    "6\tืงื™ืฉื•ืŸ\tืงื™ืฉื•ืŸ\tPROPN\tPROPN\t\t5\tflat\t_\t_",
    "7\tืืช\tืืช\tADP\tADP\t\t8\tcase:acc\t_\t_",
    "8-10\tืœื™ืžื•ื“ื™ื•\t_\t_\t_\t_\t_\t_\t_\t_",
    "8\tืœื™ืžื•ื“_\tืœื™ืžื•ื“\tNOUN\tNOUN\tGender=Masc|Number=Plur\t4\tobj\t_\t_",
    "9\t_ืฉืœ_\tืฉืœ\tADP\tADP\t_\t10\tcase\t_\t_",
    "10\t_ื”ื•ื\tื”ื•ื\tPRON\tPRON\tGender=Masc|Number=Sing|Person=3\t8\tnmod:poss\t_\t_",
    "11-12\tื‘ืคื™ืกื•ืœ\t_\t_\t_\t_\t_\t_\t_\t_",
    "11\tื‘\tื‘\tADP\tADP\t_\t12\tcase\t_\t_",
    "12\tืคื™ืกื•ืœ\tืคื™ืกื•ืœ\tNOUN\tNOUN\tGender=Masc|Number=Sing\t8\tnmod\t_\t_",
    "13\tืžืชื›ืช\tืžืชื›ืช\tNOUN\tNOUN\tGender=Fem|Number=Sing\t12\tcompound:smixut\t_\t_",
    "14-16\tื•ื‘ืชื•ืœื“ื•ืช\t_\t_\t_\t_\t_\t_\t_\t_",
    "14\tื•\tื•\tCCONJ\tCCONJ\t_\t16\tcc\t_\t_",
    "15\tื‘\tื‘\tADP\tADP\t_\t16\tcase\t_\t_",
    "16\tืชื•ืœื“ื•ืช\tืชื•ืœื“ื”\tNOUN\tNOUN\tGender=Fem|Number=Plur\t12\tconj\t_\t_",
    "17-18\tื”ืืžื ื•ืช\t_\t_\t_\t_\t_\t_\t_\t_",
    "17\tื”\tื”\tDET\tDET\t_\t18\tdet\t_\t_",
    "18\tืืžื ื•ืช\tืื•ืžื ื•ืช\tNOUN\tNOUN\tGender=Fem|Number=Sing\t16\tcompound:smixut\t_\t_",
    "19-20\tื•ื”ื—ืœ\t_\t_\t_\t_\t_\t_\t_\t_",
    "19\tื•\tื•\tCCONJ\tCCONJ\t_\t20\tcc\t_\t_",
    "20\tื”ื—ืœ\tื”ื—ืœ\tVERB\tVERB\tGender=Masc|Number=Sing|Person=3|Tense=Past\t4\tconj\t_\t_",
    "21\tืœืคืจืกื\tืคืจืกื\tVERB\tVERB\t\t20\txcomp\t_\t_",
    "22\tืžืืžืจื™ื\tืžืืžืจ\tNOUN\tNOUN\tGender=Masc|Number=Plur\t21\tobj\t_\t_",
    "23\tื”ื•ืžื•ืจื™ืกื˜ื™ื™ื\tื”ื•ืžื•ืจื™ืกื˜ื™\tADJ\tADJ\tGender=Masc|Number=Plur\t22\tamod\t_\t_"
  ]
]

Citation

If you use DictaBERT-joint in your research, please cite MRL Parsing without Tears: The Case of Hebrew

BibTeX:

@misc{shmidman2024mrl,
      title={MRL Parsing Without Tears: The Case of Hebrew}, 
      author={Shaltiel Shmidman and Avi Shmidman and Moshe Koppel and Reut Tsarfaty},
      year={2024},
      eprint={2403.06970},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

License

Shield: CC BY 4.0

This work is licensed under a Creative Commons Attribution 4.0 International License.

CC BY 4.0

Downloads last month
14,000
Safetensors
Model size
0.2B params
Tensor type
F32
ยท
Inference Providers NEW
Feature Extraction
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Spaces using dicta-il/dictabert-joint 3

Collection including dicta-il/dictabert-joint

Paper for dicta-il/dictabert-joint