Remove parquet index without metadata
The parquet index without metadata is too slow and causes the majority of the OOMs on /rows atm.

I think it's best to remove it completely.

For now it would cause some datasets like [tiiuae/falcon-refinedweb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) and [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca) to have a pagination that returns an error.

PR https://github.com/huggingface/datasets-server/pull/1497 should fix it for [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca), and I opened and issue https://github.com/huggingface/datasets-server/issues/1502 for [tiiuae/falcon-refinedweb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)

Ultimately we will need a way for the Hub to know if pagination is available (i.e. if parquet-metadata are available) instead of relying on the /size endpoint (or whatever it's using right now)
Remove parquet index without metadata: The parquet index without metadata is too slow and causes the majority of the OOMs on /rows atm.

I think it's best to remove it completely.

For now it would cause some datasets like [tiiuae/falcon-refinedweb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) and [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca) to have a pagination that returns an error.

PR https://github.com/huggingface/datasets-server/pull/1497 should fix it for [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca), and I opened and issue https://github.com/huggingface/datasets-server/issues/1502 for [tiiuae/falcon-refinedweb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)

Ultimately we will need a way for the Hub to know if pagination is available (i.e. if parquet-metadata are available) instead of relying on the /size endpoint (or whatever it's using right now)