remove unused dependencies in workers
See https://github.com/huggingface/datasets-server/issues/2636#issuecomment-2021556425

It will fix #2636, and also partly #2476

It should also reduce the CI duration and the size of the service/worker docker image (no more tensorflow and pytorch!)
remove unused dependencies in workers: See https://github.com/huggingface/datasets-server/issues/2636#issuecomment-2021556425

It will fix #2636, and also partly #2476

It should also reduce the CI duration and the size of the service/worker docker image (no more tensorflow and pytorch!)