poltextlab/illframes-climate-binary

DEPRECATED — superseded by poltextlab/illframes-climate-binary-v2. The metrics below were computed on a 969-row test set that does not exist in poltextlab/illframes-climate and cannot be reproduced. The v2 release reports held-out metrics that can, and ships its evaluation and prediction files.

Model Description

This is an xlm-roberta-large model finetuned on English training data labelled with the Illframes Climate Codebook categories:

• 710: Threatening economic growth
• 720: Threatening national sovereignty
• 721: Climate conspiracy
• 722: Scientific scepticism and denial
• 723: Climate movement bashing
• 724: Other polluters as the real problem
• 730: Threatening energy security
• 740: Threatening way of life
• 799: None of them

This is a binary model trained to detect illiberal framing.

The training data is recoded as:

• 1: 710-720-721-722-723-724-730-740
• 0: 799

---

How to Use the Model

from transformers import AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained("xlm-roberta-large")
pipe = pipeline(
    model="poltextlab/illframes-climate-binary",
    task="text-classification",
    tokenizer=tokenizer,
    use_fast=False,
    token="<your_hf_read_only_token>"
)

text = "The European Green Deal is exactly why people, our citizens, pay increasingly expensive energy and food today..."
pipe(text)

Gated Access

This model requires gated access. You must pass the token parameter when loading the model. In earlier versions of the Transformers package, you may need to use the use_auth_token parameter instead.

Model Performance

The model was evaluated on a test set of 969 English examples.

Accuracy: 0.74

Precision: 0.80

Recall: 0.74

Weighted Average F1-score: 0.76

Classification report

Class	Precision	Recall	F1-Score	Support
0	0.90	0.74	0.81	731
1	0.48	0.76	0.59	238