POC separate queue
While trying to optimize the queue, I realized most of the data in Job collection is unnecessary to get the next waiting job.
See from a total of 8562 records:
Status | Count | %
-- | -- | --
waiting | 2835 | 33.11%
success | 4614 | 53.89%
error | 840 | 9.81%
started | 24 | 0.28%
cancelled | 249 | 2.91%

This means that the get_next_waiting function would only need about 34% of the data in the collection.
We have seen some months ago (and whenever we have a high load in the queue) that MongoDB complains about `Scanned Objects / Returned has gone above 1000.`
I propose to split the queue data into two different collections:
One for the active jobs (started and waiting) and another for the finished ones (success, error, and canceled). 
This approach would help to optimize the query duration and also would target the index definition in a better way.
  
There is a lot to do yet regarding this PR:
- Evaluate if it is needed MongoDB migrations for the new collection
- Fix tests
- Fix style
- Modify indexes based on requirements

Before proceeding, let me know wdyt @huggingface/datasets-server 
POC separate queue: While trying to optimize the queue, I realized most of the data in Job collection is unnecessary to get the next waiting job.
See from a total of 8562 records:
Status | Count | %
-- | -- | --
waiting | 2835 | 33.11%
success | 4614 | 53.89%
error | 840 | 9.81%
started | 24 | 0.28%
cancelled | 249 | 2.91%

This means that the get_next_waiting function would only need about 34% of the data in the collection.
We have seen some months ago (and whenever we have a high load in the queue) that MongoDB complains about `Scanned Objects / Returned has gone above 1000.`
I propose to split the queue data into two different collections:
One for the active jobs (started and waiting) and another for the finished ones (success, error, and canceled). 
This approach would help to optimize the query duration and also would target the index definition in a better way.
  
There is a lot to do yet regarding this PR:
- Evaluate if it is needed MongoDB migrations for the new collection
- Fix tests
- Fix style
- Modify indexes based on requirements

Before proceeding, let me know wdyt @huggingface/datasets-server 