Added Readme.md

README.md

@@ -0,0 +1,198 @@
+---
+language:
+- en
+datasets:
+- RosettaCommons/PISCES-CulledPDB
+license: mit
+library_name: pytorch
+base_model: facebook/esm2_t6_8M_UR50D
+tags:
+- biology
+- bioinformatics
+- protein-secondary-structure
+- esm2
+- pytorch
+- bilstm
+pipeline_tag: token-classification
+model-index:
+- name: SERAPH
+  results:
+  - task:
+      type: token-classification
+      name: Secondary Structure Prediction (Q3)
+    metrics:
+    - name: Q3 Test Accuracy
+      type: accuracy
+      value: 75.31
+---
+
+# SERAPH (Secondary Structure Recognition & Prediction Hub)
+
+**SERAPH** is a deep learning model designed for 3-state (Q3) protein secondary structure prediction. It processes raw single amino acid sequences and predicts residue-level secondary structure states: **Alpha Helix (`H`)**, **Beta Sheet (`E`)**, or **Coil/Loop (`C`)**.
+
+The model leverages a fine-tuned `facebook/esm2_t6_8M_UR50D` backbone combined with a 1D Convolutional feature extractor and a 2-layer Bidirectional LSTM to capture local motifs and long-range sequence context simultaneously.
+
+## Model Details
+
+### Model Description
+
+- **Developed by:** Rogue Builds
+- **Model Type:** Protein Language Model + Conv1D + BiLSTM
+- **Language(s):** Protein Sequences (Amino Acid single-letter codes)
+- **License:** MIT
+- **Finetuned from model:** `facebook/esm2_t6_8M_UR50D`
+
+### Model Sources
+
+- **Repository:** `PypCoder/SERAPH`
+
+---
+
+## Intended Uses & Limitations
+
+### Direct Use
+* Residue-level 3-state (Q3) protein secondary structure prediction.
+* Single-sequence inference when Multiple Sequence Alignment (MSA) generation is computationally prohibitive or unavailable.
+* Integration into downstream bioinformatics analysis pipelines and structural annotation tools.
+
+### Out-of-Scope & Misuse
+* **3D Coordinate Generation**: SERAPH predicts 1D structural states (`H`, `E`, `C`), not 3D atomic coordinates.
+* **Q8 DSSP Prediction**: The model is trained strictly for 3-state classification and does not differentiate between 8-state DSSP assignments (e.g., distinguishing $3_{10}$-helices from $\alpha$-helices).
+
+### Known Limitations
+* **Sequence Length Limit**: Input sequences are capped at **512 tokens** due to the positional encoding window of the underlying ESM-2 backbone.
+* **Single-Sequence Bias**: Lacks explicit MSA input features; evolutionary context is derived solely from pre-trained ESM-2 representations.
+
+---
+
+## How to Get Started
+
+### Prerequisites
+
+```bash
+pip install torch transformers huggingface_hub
+```
+
+### Python Inference Example
+
+```python
+import torch
+import torch.nn as nn
+from transformers import EsmModel, EsmTokenizer
+
+# 1. Define SERAPH Architecture
+class SERAPH(nn.Module):
+    def __init__(self, esm_model, conv_channels=256, kernel_size=7, lstm_hidden=256, num_classes=3, dropout=0.3):
+        super().__init__()
+        self.esm = esm_model
+        esm_embed_dim = self.esm.config.hidden_size
+        self.conv = nn.Conv1d(esm_embed_dim, conv_channels, kernel_size=kernel_size, padding=kernel_size // 2)
+        self.bn = nn.BatchNorm1d(conv_channels)
+        self.dropout = nn.Dropout(dropout)
+        self.bilstm = nn.LSTM(conv_channels, lstm_hidden, num_layers=2, batch_first=True, bidirectional=True)
+        self.fc = nn.Linear(lstm_hidden * 2, num_classes)
+
+    def forward(self, input_ids, attention_mask=None):
+        x = self.esm(input_ids=input_ids, attention_mask=attention_mask).last_hidden_state
+        x = x.transpose(1, 2)
+        x = torch.relu(self.bn(self.conv(x)))
+        x = self.dropout(x)
+        x = x.transpose(1, 2)
+        x, _ = self.bilstm(x)
+        x = self.dropout(x)
+        return self.fc(x)
+
+# 2. Load Tokenizer & Base Backbone
+ESM_MODEL_ID = "facebook/esm2_t6_8M_UR50D"
+tokenizer = EsmTokenizer.from_pretrained(ESM_MODEL_ID)
+esm_backbone = EsmModel.from_pretrained(ESM_MODEL_ID)
+
+model = SERAPH(esm_model=esm_backbone)
+
+# Load weight checkpoint
+# checkpoint = torch.load("SERAPH.pth", map_location="cpu")
+# model.load_state_dict(checkpoint["model_state_dict"])
+model.eval()
+
+# 3. Perform Prediction
+IDX_TO_LABEL = {0: 'H', 1: 'E', 2: 'C'}
+sequence = "MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHFDLSHGSAQVKGHGKKVADALTNAVAHVDDMPNALSALSDLHAHKLRVDPVNFKLLSHCLLVTLAAHLPAEFTPAVHASLDKFLASVSTVLTSKYR"
+
+tokens = tokenizer(sequence, return_tensors="pt", truncation=True, max_length=512)
+
+with torch.no_grad():
+    output = model(input_ids=tokens["input_ids"], attention_mask=tokens["attention_mask"])
+    preds = output.argmax(dim=-1)[0]
+
+# Omit special tokens [CLS] and [EOS]
+prediction = "".join([IDX_TO_LABEL[p.item()] for p in preds[1:-1]])
+print(f"Sequence:   {sequence}")
+print(f"Prediction: {prediction}")
+```
+
+---
+
+## Training Details
+
+### Training Data
+
+* **Dataset**: CullPDB (~6,000 non-redundant protein chains).
+
+### Training Procedure
+
+* **Optimizer**: Adam (`lr=5e-5`, `weight_decay=1e-4`)
+* **Loss Function**: `CrossEntropyLoss` with class weight adjustments `[H: 1.3, E: 1.3, C: 1.0]`
+* **Gradient Clipping**: `max_norm = 1.0`
+* **Scheduler**: `ReduceLROnPlateau` (`patience=3`, `factor=0.5`)
+* **Batch Size**: 32 (with dynamic sequence padding)
+* **Epochs**: 15
+* **Backbone Unfreezing**: Top 2 transformer layers of `facebook/esm2_t6_8M_UR50D` unfrozen during training.
+
+### Parameter Distribution
+
+| Layer Component | Trainable Parameters |
+|---|---|
+| ESM-2 Backbone (Unfrozen layers) | ~2,600,000 |
+| Conv1D (`320 → 256`, `k=7`) | 573,440 |
+| BatchNorm1d (`256`) | 512 |
+| BiLSTM (2 Layers, hidden=256) | ~1,311,232 |
+| Linear Head (`512 → 3`) | 1,539 |
+| **Total Trainable Parameters** | **3,205,379** |
+
+---
+
+## Evaluation Results
+
+### Evaluation Benchmark
+
+Evaluated on the standard **CB513** benchmark dataset.
+
+### Metrics
+
+| Evaluation Metric | Score |
+|---|---|
+| **Q3 Test Accuracy (CB513)** | **75.31%** |
+| **Q3 Training Accuracy** | **79.34%** |
+
+#### Class Breakdown
+
+| Structure Class | Precision | Recall |
+|---|---|---|
+| **Helix (`H`)** | 0.82 | 0.80 |
+| **Sheet (`E`)** | 0.63 | 0.81 |
+| **Coil (`C`)** | 0.79 | 0.68 |
+
+---
+
+## Citation & Contact
+
+If you use SERAPH in your work, please cite the underlying ESM-2 paper and reference this repository:
+
+```bibtex
+@software{seraph2026,
+  author = {Muhammad Asad Ullah},
+  title = {SERAPH: Secondary Structure Recognition & Prediction Hub},
+  year = {2026},
+  url = {https://huggingface.co/PypCoder/SERAPH}
+}
+```