Adapt the sleep time of the workers
When a worker just finished a job, it should ask for another job right away.

But if it has already polled the dataset multiple times and got no job, it should increase the sleep time between two polls, in order to avoid hammering the database.

By the way, it might just be:
- finish a job: directly ask for a new one
- else sleep for a constant (and large) duration, eg. 10s
Adapt the sleep time of the workers: When a worker just finished a job, it should ask for another job right away.

But if it has already polled the dataset multiple times and got no job, it should increase the sleep time between two polls, in order to avoid hammering the database.

By the way, it might just be:
- finish a job: directly ask for a new one
- else sleep for a constant (and large) duration, eg. 10s