wesleyaag/data2vec-squad-test

data2vec squad

This is a testing fine tuned data2vec model in the squad dataset, any improvements and suggestions are welcome!

Intended use

Question Answering

Training results

Epoch	Training Loss	Validation Loss
1	1.015800	0.997690
2	0.804400	0.950322

Hyperparameters

• evaluation_strategy="epoch"
• learning_rate=2e-5
• per_device_train_batch_size=15
• per_device_eval_batch_size=15
• num_train_epochs=2
• weight_decay=0.01

Frameworks and libraries used:

• transformers
• datasets
• evaluate