Aayan Mishra commited on
Commit
17ba61c
Β·
verified Β·
1 Parent(s): 83f549c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +125 -23
README.md CHANGED
@@ -1,53 +1,155 @@
1
  ---
 
2
  license: mit
3
  tags:
4
  - protein-folding
5
  - biology
6
- - esm2
7
  - alphafold
8
- library_name: fractal
 
 
9
  ---
10
 
11
- # FRACTAL Protein Folding Model (ESM2-3B)
 
 
 
 
 
 
 
 
 
 
12
 
13
- AlphaFold-style protein structure prediction using ESM-2 (3B parameters) backbone with constraint prediction heads.
 
 
14
 
15
- ## Model Description
16
 
17
- - **Backbone**: ESM2-3B (3 billion parameters, frozen)
18
- - **Task**: Predict distance, contact, and torsion constraints
19
- - **Training**: 1000 PDB structures from RCSB
20
- - **Hardware**: 2xT4 GPUs on Kaggle
 
 
 
 
 
21
 
22
  ## Usage
23
 
 
 
24
  ```python
25
- from fractal.inference.pipeline import predict_constraints_from_fasta
26
  from fractal.geometry.folding import fold_from_constraints
27
 
28
- # Predict constraints
29
- constraints = predict_constraints_from_fasta(
30
- fasta_path="protein.fasta",
31
- checkpoint_dir="models/trained/best",
32
- device="cuda"
33
  )
34
 
 
 
 
 
35
  # Fold to 3D structure
36
  structure = fold_from_constraints(
37
- sequence=constraints.sequence,
38
- distance_logits=constraints.distance_logits,
39
- torsion_angles=constraints.torsion_angles,
40
- contact_logits=constraints.contact_logits,
41
- steps=500
42
  )
43
 
 
44
  structure.to_pdb("output.pdb")
45
  ```
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  ## Training Details
48
 
49
- See config at `configs/train.yaml` for hyperparameters.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
 
51
- ## Examples
52
 
53
- See example predictions in the repository.
 
 
1
  ---
2
+ library_name: fractal
3
  license: mit
4
  tags:
5
  - protein-folding
6
  - biology
 
7
  - alphafold
8
+ - structure-prediction
9
+ - esm
10
+ pipeline_tag: feature-extraction
11
  ---
12
 
13
+ # FRACTAL 3B Protein Structure Predictor
14
+
15
+ **FRACTAL** (Framework for Representation-guided Atomic ConsTruction & ALignment) is a protein structure prediction system that combines ESM-2 language model embeddings with geometric constraint prediction and deterministic folding.
16
+
17
+ ## Model Overview
18
+
19
+ This model uses the **3 billion parameter ESM-2** backbone (`esm2_t36_3B_UR50D`) with lightweight prediction heads trained to output:
20
+ - **Distance constraints** between residue pairs
21
+ - **Contact maps** for spatial proximity
22
+ - **Torsion angles** (Ο†, ψ, Ο‰) for backbone geometry
23
+ - **Confidence scores** per residue
24
 
25
+ Unlike end-to-end coordinate prediction models, FRACTAL uses a two-stage pipeline:
26
+ 1. **Constraint Prediction** (this model) - predicts geometric constraints from sequence
27
+ 2. **Deterministic Folding** - converts constraints to 3D structure using gradient descent
28
 
29
+ ## Installation
30
 
31
+ ```bash
32
+ # Install from GitHub
33
+ pip install git+https://github.com/Aayan-Mishra/FRACTAL.git
34
+
35
+ # Or clone and install locally
36
+ git clone https://github.com/Aayan-Mishra/FRACTAL.git
37
+ cd FRACTAL
38
+ pip install -e .
39
+ ```
40
 
41
  ## Usage
42
 
43
+ ### Using the Python API
44
+
45
  ```python
46
+ from fractal.models import ConstraintPredictor
47
  from fractal.geometry.folding import fold_from_constraints
48
 
49
+ # Load the trained model
50
+ model = ConstraintPredictor.from_pretrained(
51
+ "Spestly/FRACTAL-1-3B",
52
+ device="cuda" # or "cpu"
 
53
  )
54
 
55
+ # Predict constraints from sequence
56
+ sequence = "MNIFEMLRIDEGLRLKIYKDTEGYYTIGIGHLLTKSPSLNAAKSELDKAIGRNTNGVITKDEAEKLFNQDVDAAVRGILRNAKLKPVYDSLDAVRRAALINMVFQMGETGVAGFTNSLRMLQQKRWDEAAVNLAKSRWYNQTPNRAKRVITTFRTGTWDAYKNL"
57
+ predictions = model.predict_from_sequence(sequence, device="cuda")
58
+
59
  # Fold to 3D structure
60
  structure = fold_from_constraints(
61
+ predictions,
62
+ num_steps=1000,
63
+ lr=0.01,
64
+ device="cuda"
 
65
  )
66
 
67
+ # Save as PDB
68
  structure.to_pdb("output.pdb")
69
  ```
70
 
71
+ ### Using the CLI
72
+
73
+ ```bash
74
+ # Predict and fold in one command
75
+ proteinfold fold input.fasta --checkpoint Spestly/FRACTAL-1-3B --viz
76
+
77
+ # This generates:
78
+ # - input.pdb (3D structure)
79
+ # - input.html (interactive 3D viewer)
80
+ # - input.png (static render)
81
+ ```
82
+
83
+ ### Just Predict Constraints (No Folding)
84
+
85
+ ```python
86
+ from fractal.models import ConstraintPredictor
87
+
88
+ model = ConstraintPredictor.from_pretrained("Spestly/FRACTAL-1-3B/fractal-3b")
89
+ predictions = model.predict_from_sequence("MNIFEMLR...")
90
+
91
+ # Access predictions
92
+ dist_logits = predictions.distance_logits # [L, L, 64] distance bins
93
+ contact_logits = predictions.contact_logits # [L, L] contact map
94
+ torsions = predictions.torsion_angles # [L, 3] Ο†, ψ, Ο‰
95
+ confidence = predictions.confidence # [L] per-residue pLDDT
96
+ ```
97
+
98
  ## Training Details
99
 
100
+ - **Backbone**: ESM-2 3B (`esm2_t36_3B_UR50D`) - frozen during training
101
+ - **Training Data**: ~1000 high-resolution PDB structures (resolution < 2.0Γ…)
102
+ - **Batch Size**: 1 (with 16 gradient accumulation steps)
103
+ - **Optimizer**: AdamW with learning rate 5e-5
104
+ - **Hardware**: Kaggle 2xT4 GPUs (16GB each)
105
+ - **Training Time**: ~3-4 hours
106
+
107
+ ### Loss Function
108
+
109
+ Multi-task loss combining:
110
+ - Distance binned cross-entropy (64 bins, 0-20Γ…)
111
+ - Contact binary cross-entropy (8Γ… threshold)
112
+ - Torsion angle MSE with circular wrapping
113
+ - Confidence MSE (pLDDT-style)
114
+
115
+ ## Model Architecture
116
+
117
+ ```
118
+ Input Sequence β†’ ESM-2 3B Encoder β†’ Per-residue embeddings (2560-dim)
119
+ ↓
120
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
121
+ ↓ ↓
122
+ PairwiseConstraintHead TorsionAngleHead
123
+ (outer product + conv) (linear layers)
124
+ ↓ ↓
125
+ Distance + Contact Maps Ο†, ψ, Ο‰ angles
126
+ ```
127
+
128
+ ## Limitations
129
+
130
+ - **Maximum sequence length**: 1024 residues
131
+ - **Training data**: Limited to ~1000 structures; may not generalize to all protein families
132
+ - **Folding speed**: Deterministic folding takes 30-60s for medium proteins (150-300 residues)
133
+ - **Accuracy**: Not competitive with AlphaFold2/3 on CASP benchmarks (research/educational project)
134
+
135
+ ## Citation
136
+
137
+ If you use this model, please cite ESM-2:
138
+
139
+ ```bibtex
140
+ @article{lin2022language,
141
+ title={Language models of protein sequences at the scale of evolution enable accurate structure prediction},
142
+ author={Lin, Zeming and Akin, Halil and Rao, Roshan and Hie, Brian and Zhu, Zhongkai and Lu, Wenting and Smetanin, Nikita and Verkuil, Robert and Kabeli, Ori and Shmueli, Yair and others},
143
+ journal={Science},
144
+ year={2022}
145
+ }
146
+ ```
147
+
148
+ ## License
149
+
150
+ MIT - See repository for details.
151
 
152
+ ## Links
153
 
154
+ - **Repository**: https://github.com/Aayan-Mishra/FRACTAL
155
+ - **Issues**: https://github.com/Aayan-Mishra/FRACTAL/issues