feat: 🎸 write cache + backfill only if job finished as expected
ie: if it has been cancelled, we ignore it. See previous work at https://github.com/huggingface/datasets-server/pull/1188. Note that after #1222, the number of warnings "...has a non-empty finished_at field..." has fallen to 26 logs among 20,000, while it was like 20% of the logs before!

Also:
- upgrade `requests`(vulnerability, fixes the CI)
- wait after the backfill to finish the job (the backfill should finish it anyway)
feat: 🎸 write cache + backfill only if job finished as expected: ie: if it has been cancelled, we ignore it. See previous work at https://github.com/huggingface/datasets-server/pull/1188. Note that after #1222, the number of warnings "...has a non-empty finished_at field..." has fallen to 26 logs among 20,000, while it was like 20% of the logs before!

Also:
- upgrade `requests`(vulnerability, fixes the CI)
- wait after the backfill to finish the job (the backfill should finish it anyway)