Delete empty folders from downloaded duckdb indexes in /search
See comment https://github.com/huggingface/datasets-server/pull/1536#discussion_r1283612238
Currently, the job deletes expired files (more than 3 days in prod) but if the folders remain empty those will continue existing, we should remove them.
Delete empty folders from downloaded duckdb indexes in /search: See comment https://github.com/huggingface/datasets-server/pull/1536#discussion_r1283612238
Currently, the job deletes expired files (more than 3 days in prod) but if the folders remain empty those will continue existing, we should remove them.