| --- |
| license: mit |
| widget: |
| - text: MQIFVKTLTGKTITLEVEPS<mask>TIENVKAKIQDKEGIPPDQQRLIFAGKQLEDGRTLSDYNIQKESTLHLVLRLRGG |
| tags: |
| - biology |
| - protein-language-model |
| - proteins |
| - embeddings |
| - drug-discovery |
| - aurigene |
| pipeline_tag: fill-mask |
| library_name: transformers |
| --- |
| |
| <!-- aurigene-header --> |
| > ### Mirrored by [Aurigene AI](https://huggingface.co/Aurigene-AI) |
| > **Discovery stage:** Target identification |
| > |
| > Meta's ESM-2 protein language model. Per-residue embeddings that transfer to binding-site prediction, variant-effect scoring and structure-aware target featurisation. |
| > |
| > Upstream: [`facebook/esm2_t33_650M_UR50D`](https://huggingface.co/facebook/esm2_t33_650M_UR50D) - all credit to the original authors; the model card and licence below are theirs. |
| > |
| > Explore the rest of the catalogue: [Molecule Explorer](https://huggingface.co/spaces/Aurigene-AI/molecule-explorer) - [Protein Target Explorer](https://huggingface.co/spaces/Aurigene-AI/protein-target-explorer) - [Drug Discovery Model Hub](https://huggingface.co/spaces/Aurigene-AI/drug-discovery-model-hub) |
|
|
| --- |
|
|
| ## ESM-2 |
|
|
| ESM-2 is a state-of-the-art protein model trained on a masked language modelling objective. It is suitable for fine-tuning on a wide range of tasks that take protein sequences as input. For detailed information on the model architecture and training data, please refer to the [accompanying paper](https://www.biorxiv.org/content/10.1101/2022.07.20.500902v2). You may also be interested in some demo notebooks ([PyTorch](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/protein_language_modeling.ipynb), [TensorFlow](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/protein_language_modeling-tf.ipynb)) which demonstrate how to fine-tune ESM-2 models on your tasks of interest. |
|
|
| Several ESM-2 checkpoints are available in the Hub with varying sizes. Larger sizes generally have somewhat better accuracy, but require much more memory and time to train: |
|
|
| | Checkpoint name | Num layers | Num parameters | |
| |------------------------------|----|----------| |
| | [esm2_t48_15B_UR50D](https://huggingface.co/facebook/esm2_t48_15B_UR50D) | 48 | 15B | |
| | [esm2_t36_3B_UR50D](https://huggingface.co/facebook/esm2_t36_3B_UR50D) | 36 | 3B | |
| | [esm2_t33_650M_UR50D](https://huggingface.co/facebook/esm2_t33_650M_UR50D) | 33 | 650M | |
| | [esm2_t30_150M_UR50D](https://huggingface.co/facebook/esm2_t30_150M_UR50D) | 30 | 150M | |
| | [esm2_t12_35M_UR50D](https://huggingface.co/facebook/esm2_t12_35M_UR50D) | 12 | 35M | |
| | [esm2_t6_8M_UR50D](https://huggingface.co/facebook/esm2_t6_8M_UR50D) | 6 | 8M | |
