Smart dataset update: don't recompute everything all the time
Don't recompute a viewer if the commit is an update in the main readme content or in yaml tags that don't influence the viewer.

Only available for datasets in `datasets-maintainers` org for now.

It works this way

1. take the old_sha and new_sha of the webhook
2. check that the cache revision is old_sha
3. check the updated files and fields in the YAML readme file
4. if there is no need to recomptue anything, update the revisions of the cache entries and storage directories

PS1: it doesn't update the revision of the entries in the queue because:
- running jobs might be writing to the old storage directory: they should fail and be re-run on the new revision
- waiting jobs are not relevant since we need to re-run their parent running jobs anyway

PS2: I added a very simple out-of-order commit handling by retrying in case the old_sha doesn't match

Fix this issue https://github.com/huggingface/dataset-viewer/issues/2757 once available for all datasets
Smart dataset update: don't recompute everything all the time: Don't recompute a viewer if the commit is an update in the main readme content or in yaml tags that don't influence the viewer.

Only available for datasets in `datasets-maintainers` org for now.

It works this way

1. take the old_sha and new_sha of the webhook
2. check that the cache revision is old_sha
3. check the updated files and fields in the YAML readme file
4. if there is no need to recomptue anything, update the revisions of the cache entries and storage directories

PS1: it doesn't update the revision of the entries in the queue because:
- running jobs might be writing to the old storage directory: they should fail and be re-run on the new revision
- waiting jobs are not relevant since we need to re-run their parent running jobs anyway

PS2: I added a very simple out-of-order commit handling by retrying in case the old_sha doesn't match

Fix this issue https://github.com/huggingface/dataset-viewer/issues/2757 once available for all datasets