Revision_Meta_XLM

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4710
• Accuracy: 0.9220
• F1: 0.7251

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.5102	150	0.3977	0.8526	0.4695
0.5230	1.0204	300	0.2417	0.9188	0.7149
0.5230	1.5306	450	0.2309	0.9220	0.7111
0.2723	2.0408	600	0.2386	0.9180	0.7130
0.2723	2.5510	750	0.2537	0.9259	0.7229
0.2107	3.0612	900	0.2225	0.9354	0.7547
0.2107	3.5714	1050	0.2462	0.9330	0.7376
0.1682	4.0816	1200	0.3432	0.9117	0.7101
0.1682	4.5918	1350	0.2572	0.9267	0.7195
0.1499	5.1020	1500	0.3172	0.9243	0.7344
0.1499	5.6122	1650	0.3080	0.9212	0.7206
0.1219	6.1224	1800	0.3767	0.9133	0.6921
0.1219	6.6327	1950	0.4346	0.9039	0.7024
0.1055	7.1429	2100	0.3059	0.9338	0.7309
0.1055	7.6531	2250	0.3716	0.9236	0.7237
0.0786	8.1633	2400	0.4106	0.9188	0.7174
0.0786	8.6735	2550	0.4007	0.9228	0.7250
0.0673	9.1837	2700	0.4195	0.9236	0.7309
0.0673	9.6939	2850	0.4486	0.9149	0.7102
0.0589	10.2041	3000	0.5117	0.9078	0.6939
0.0589	10.7143	3150	0.4755	0.9149	0.7154
0.0494	11.2245	3300	0.4287	0.9204	0.7198
0.0494	11.7347	3450	0.3905	0.9314	0.7322
0.0426	12.2449	3600	0.4332	0.9243	0.7333
0.0426	12.7551	3750	0.4228	0.9307	0.7385
0.0339	13.2653	3900	0.4589	0.9275	0.7380
0.0339	13.7755	4050	0.5148	0.9141	0.7128
0.0272	14.2857	4200	0.4624	0.9236	0.7273
0.0272	14.7959	4350	0.4710	0.9220	0.7251

Framework versions

• Transformers 5.3.0
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.2