Skip the job depending on the type of error
Currently, we never skip a job if the previous run has returned an error. But it should be conditional: only retry for an allowlist of errors (e.g., ConnectionError), not the other way: for the same commit, we will get the same result and just lose time and resources.

Here: https://github.com/huggingface/datasets-server/blob/893a70cb7201090b8c64cd127fbe029c723f2aa3/workers/datasets_based/src/datasets_based/worker.py#L221
Skip the job depending on the type of error: Currently, we never skip a job if the previous run has returned an error. But it should be conditional: only retry for an allowlist of errors (e.g., ConnectionError), not the other way: for the same commit, we will get the same result and just lose time and resources.

Here: https://github.com/huggingface/datasets-server/blob/893a70cb7201090b8c64cd127fbe029c723f2aa3/workers/datasets_based/src/datasets_based/worker.py#L221