Pagination broken for some image datasets (too big row groups)
some datasets have row groups too big instead of being limited to 100 rows

eg https://huggingface.co/datasets/multimodalart/lora-fusing-preferences has all its 5xx rows in one row group in the parquet export

This causes pagination to return an error to avoid OOM
Pagination broken for some image datasets (too big row groups): some datasets have row groups too big instead of being limited to 100 rows

eg https://huggingface.co/datasets/multimodalart/lora-fusing-preferences has all its 5xx rows in one row group in the parquet export

This causes pagination to return an error to avoid OOM