The number of started jobs is a lot bigger than the number of workers
We currently have nearly 2000 started jobs, and "only" 200 workers

<img width="389" alt="Capture d’écran 2023-06-07 à 18 27 10" src="https://github.com/huggingface/datasets-server/assets/1676121/df48b775-b423-4d99-af59-7117355685c8">

My guess: the jobs raise an uncaught exception and are never finished properly in the database. They will be deleted afterwards anyway (zombie killer) but possibly we must fix something here.
The number of started jobs is a lot bigger than the number of workers: We currently have nearly 2000 started jobs, and "only" 200 workers

<img width="389" alt="Capture d’écran 2023-06-07 à 18 27 10" src="https://github.com/huggingface/datasets-server/assets/1676121/df48b775-b423-4d99-af59-7117355685c8">

My guess: the jobs raise an uncaught exception and are never finished properly in the database. They will be deleted afterwards anyway (zombie killer) but possibly we must fix something here.