Fix estimate info for zip datasets
I simply had to track metadata reads only once to fix the estimator.

(otherwise every file opened in a zip archive triggers an additional read of the metadata with the central directory of the zip file that prevents the estimator from converging)

ex: locally and on only 100MB of parquet conversion (prod is 5GB), it estimates 47871 examples on https://huggingface.co/datasets/datasets-maintainers/test_many_zip_with_images_MSCOCO_zip (true value is exactly 50k)

Unrelared, but I noticed that parquet conversion of zip files is super slooooooowww, we'll have to improve that at one point because it can surely take more than 40min to run. When tested in prod, an error happens before that though (JobManagerCrashedError) which suggests a termination by kubernetes.
Fix estimate info for zip datasets: I simply had to track metadata reads only once to fix the estimator.

(otherwise every file opened in a zip archive triggers an additional read of the metadata with the central directory of the zip file that prevents the estimator from converging)

ex: locally and on only 100MB of parquet conversion (prod is 5GB), it estimates 47871 examples on https://huggingface.co/datasets/datasets-maintainers/test_many_zip_with_images_MSCOCO_zip (true value is exactly 50k)

Unrelared, but I noticed that parquet conversion of zip files is super slooooooowww, we'll have to improve that at one point because it can surely take more than 40min to run. When tested in prod, an error happens before that though (JobManagerCrashedError) which suggests a termination by kubernetes.