FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Paper
•
2506.20920
•
Published
•
76
Dataset
-
-
HuggingFaceFW/finewiki
Viewer • Updated • 61.6M • 8.77k • 273 -
nhagar/fineweb_urls
Viewer • Updated • 24.5B • 238 • 1 -
PleIAs/common_corpus
Viewer • Updated • 470M • 43.2k • 330 -
google-research-datasets/mbpp
Viewer • Updated • 1.4k • 1.66M • 205 -
allenai/c4
Viewer • Updated • 10.4B • 574k • 506 -
HuggingFaceFW/fineweb
Viewer • Updated • 52.5B • 180k • 2.6k