tomleung1996/section-title-sequence-classifier

Paper Section Title Sequence Classifier

This is a model based on allenai/scibert_scivocab_uncased, fine-tuned to classify sequences of section titles from scientific papers.

The model takes a single string of section titles, separated by [SEP], and predicts a class label for each section in the sequence. It is crucial that the section sequence provided to the model preserves the ORIGINAL ORDER as it appears in the source document.

Model Details

• Model Architecture: This model uses a BertModel base with a custom classification head. It first extracts the embeddings corresponding to all [SEP] separators in the input sequence. Then, it passes these embeddings through a TimeDistributed layer and a linear classifier to generate a class prediction for each section title (represented by its [SEP] token).

• Base Model: allenai/scibert_scivocab_uncased

• Language: English

• Labels: The model can identify the following five classes:

  • Introduction

  • Methods

  • Results

  • DiscussionConclusion

  • Closing (e.g., Acknowledgements, References)

How to Use

Note: This model uses a custom architecture. You must set trust_remote_code=True when loading it to allow Hugging Face to execute the model code defined in the repository.

from transformers import AutoTokenizer, AutoModel

# 1. Specify model name
hub_model_name = "tomleung1996/section-title-sequence-classifier" 

# 2. Load the tokenizer and model from the Hub
# `trust_remote_code=True` is required because it needs to execute the custom model code from the repo
tokenizer = AutoTokenizer.from_pretrained(hub_model_name)
model = AutoModel.from_pretrained(hub_model_name, trust_remote_code=True)

# 3. Prepare the input data
# List all section titles from a paper, ensuring they are in their original order.
sections = [
    "Introduction",
    "Conceptualization of the directed collaboration network",
    "Analysis of the collaboration order based on the DCN",
    "An example",
    "Conclusions and discussion",
    "Acknowledgement",
    "References"
]

# 4. Join the list into a single string using the [SEP] token
input_text = " [SEP] ".join(sections)

# 5. Tokenize the input
inputs = tokenizer(input_text, return_tensors='pt')

# 6. Run model inference
# Note: The tokenizer must be passed to the model's forward pass
# because the model internally needs it to locate the [SEP] tokens.
outputs = model(**inputs, tokenizer=tokenizer)

# 7. Get the predictions
predictions = outputs.logits.argmax(dim=-1).squeeze()

# Get the label mapping from the model's config
pred_labels = [model.config.id2label[i.item()] for i in predictions]

# 8. Print the results
print("Input Sections:")
print(sections)
print("\nPredicted Labels:")
print(pred_labels)
# Expected output:
# ['Introduction', 'Methods', 'Methods', 'Results', 'DiscussionConclusion', 'Closing', 'Closing']