Don't run config-parquet-metadata in light workers
Because it causes an OOM for all the big parquet datasets like the-stack, refinedweb etc.

causing their pagination to hang because it uses the parquet index without metadata which is too slow for big datasets
Don't run config-parquet-metadata in light workers: Because it causes an OOM for all the big parquet datasets like the-stack, refinedweb etc.

causing their pagination to hang because it uses the parquet index without metadata which is too slow for big datasets