Parquet metadata OOM for tiiuae/falcon-refinedweb
Logs show nothing but the kube YAML of the pod shows OOMKilled.

The job should be improved to be more memory efficient.

This issue prevents us from having pagination for this dataset.
Parquet metadata OOM for tiiuae/falcon-refinedweb: Logs show nothing but the kube YAML of the pod shows OOMKilled.

The job should be improved to be more memory efficient.

This issue prevents us from having pagination for this dataset.