gpt2-multilingual-20-arabic-repair

This model is a fine-tuned version of CausalNLP/gpt2-hf_multilingual-20 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.2296

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 12
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 40
• total_train_batch_size: 480
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 100
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
7.5101	0.0171	50	7.1747
3.3706	0.0341	100	3.3831
3.2621	0.0512	150	3.3133
3.2682	0.0683	200	3.2962
3.2681	0.0854	250	3.2866
3.2542	0.1024	300	3.2799
3.261	0.1195	350	3.2743
3.2559	0.1366	400	3.2699
3.2416	0.1536	450	3.2665
3.2434	0.1707	500	3.2630
3.2255	0.1878	550	3.2604
3.2446	0.2049	600	3.2577
3.2527	0.2219	650	3.2556
3.2245	0.2390	700	3.2533
3.2268	0.2561	750	3.2517
3.2061	0.2732	800	3.2498
3.2078	0.2902	850	3.2483
3.2365	0.3073	900	3.2468
3.1968	0.3244	950	3.2452
3.21	0.3414	1000	3.2439
3.2025	0.3585	1050	3.2427
3.2225	0.3756	1100	3.2416
3.2232	0.3927	1150	3.2403
3.206	0.4097	1200	3.2394
3.2359	0.4268	1250	3.2386
3.2222	0.4439	1300	3.2378
3.1938	0.4609	1350	3.2367
3.2084	0.4780	1400	3.2359
3.2182	0.4951	1450	3.2356
3.2037	0.5122	1500	3.2347
3.1926	0.5292	1550	3.2343
3.2177	0.5463	1600	3.2336
3.2103	0.5634	1650	3.2331
3.2317	0.5805	1700	3.2325
3.1834	0.5975	1750	3.2321
3.2016	0.6146	1800	3.2318
3.193	0.6317	1850	3.2314
3.1893	0.6487	1900	3.2313
3.2283	0.6658	1950	3.2309
3.1886	0.6829	2000	3.2307
3.2216	0.7000	2050	3.2305
3.2233	0.7170	2100	3.2304
3.2162	0.7341	2150	3.2302
3.1999	0.7512	2200	3.2300
3.2097	0.7682	2250	3.2299
3.2089	0.7853	2300	3.2299
3.2223	0.8024	2350	3.2298
3.2088	0.8195	2400	3.2298
3.1991	0.8365	2450	3.2297
3.2237	0.8536	2500	3.2297
3.2222	0.8707	2550	3.2297
3.1806	0.8878	2600	3.2296
3.208	0.9048	2650	3.2297
3.1773	0.9219	2700	3.2297
3.2105	0.9390	2750	3.2296
3.1888	0.9560	2800	3.2296
3.2047	0.9731	2850	3.2296
3.2039	0.9902	2900	3.2296

Framework versions

• Transformers 4.57.3
• Pytorch 2.9.0
• Datasets 4.4.1
• Tokenizers 0.22.1