Avoid providing "mixed" cache while refreshing a dataset
When a dataset is being refreshed, or even after having been refreshed, its cache can be incoherent.

## Case 1

Reported by @AndreaFrancis: during a dataset refresh, the search was enabled in the viewer, but the "first rows" were not shown. The `split-duckdb-index` step had finished, while the `split-first-rows-from...` steps were still being computed. The data shown by the viewer were not coherent, and it would be hard for a user to understand what was occurring.

## Case 2

Seen with https://huggingface.co/datasets/jbrendsel/ECTSum. All the steps had been computed, and the `config-size` step had an error (`PreviousStepFormatError`). The erroneous cache entry remained after fixing the issue and refreshing the dataset. The reason is a previous step (`dataset-config-names`) now gives an error, and the rest of the cache entries are never cleaned, so they stay in the database even if useless. See https://github.com/huggingface/datasets-server/issues/1582#issuecomment-1727151243 and https://github.com/huggingface/datasets-server/issues/1285.

## Proposal

Give each "DAG" execution an identifier, and use the same identifier for all the endpoints. To ensure coherence between the responses, change the identifier we use in the API only when all the steps have been refreshed. If we send the identifier in the response, the client can also use the identifier value to check the coherence. If there is no previous identifier in the database, use the identifier of the current incomplete DAG (the dataset viewer already handles an incomplete state).

Once the new DAG execution has finished, we can delete the cache entries with other identifiers.
Avoid providing "mixed" cache while refreshing a dataset: When a dataset is being refreshed, or even after having been refreshed, its cache can be incoherent.

## Case 1

Reported by @AndreaFrancis: during a dataset refresh, the search was enabled in the viewer, but the "first rows" were not shown. The `split-duckdb-index` step had finished, while the `split-first-rows-from...` steps were still being computed. The data shown by the viewer were not coherent, and it would be hard for a user to understand what was occurring.

## Case 2

Seen with https://huggingface.co/datasets/jbrendsel/ECTSum. All the steps had been computed, and the `config-size` step had an error (`PreviousStepFormatError`). The erroneous cache entry remained after fixing the issue and refreshing the dataset. The reason is a previous step (`dataset-config-names`) now gives an error, and the rest of the cache entries are never cleaned, so they stay in the database even if useless. See https://github.com/huggingface/datasets-server/issues/1582#issuecomment-1727151243 and https://github.com/huggingface/datasets-server/issues/1285.

## Proposal

Give each "DAG" execution an identifier, and use the same identifier for all the endpoints. To ensure coherence between the responses, change the identifier we use in the API only when all the steps have been refreshed. If we send the identifier in the response, the client can also use the identifier value to check the coherence. If there is no previous identifier in the database, use the identifier of the current incomplete DAG (the dataset viewer already handles an incomplete state).

Once the new DAG execution has finished, we can delete the cache entries with other identifiers.