First rows skip compute if a response already exists
Add validation on first-rows job runners to skip compute if a similar response already exists (Parallel response split-first-rows-from-streaming vs split-first-rows-from-parquet).
I moved the validation at job runner level since we have exactly same logic for split-names-from-streaming vs split-names-from-dataset-info.
Context: https://github.com/huggingface/datasets-server/pull/988#issuecomment-1491968842
First rows skip compute if a response already exists: Add validation on first-rows job runners to skip compute if a similar response already exists (Parallel response split-first-rows-from-streaming vs split-first-rows-from-parquet).
I moved the validation at job runner level since we have exactly same logic for split-names-from-streaming vs split-names-from-dataset-info.
Context: https://github.com/huggingface/datasets-server/pull/988#issuecomment-1491968842