Use a kubernetes infrastructure
As done for https://github.com/huggingface/tensorboard-launcher/ and https://github.com/huggingface/moon-landing gitaly before.

- [x] API: see https://github.com/huggingface/datasets-server/pull/227
- [x] workers: see #236
- [x] nginx reverse proxy: see #234
- [x] deploy to the prod cluster: see #249
- [x] respond to `datasets-server.huggingface.tech`: see #222
- [x] add a webhook from moonlanding to the new datasets-server URL: asked on [Slack](https://huggingface.slack.com/archives/C023JAKTR2P/p1652790074627839)
- [x] warm the cache on all the public datasets - see the state of the cache here: https://grafana.huggingface.tech/d/SaHl2KX7z/datasets-server?orgId=1
- [x] update moonlanding: see https://github.com/huggingface/moon-landing/pull/2957
- [x] update the moonlanding proxy: see https://github.com/huggingface/conf/pull/166
- [x] update autotrain: no need, autonlp-ui accesses the server through [a proxy on moonlanding (https://huggingface.co/proxy-datasets-preview)](https://github.com/huggingface/autonlp-ui/blob/12a7fd84cdbf9c30900e3740f6b2bbbe38ae3699/src/lib/config.ts#L57-L58)
- [x] remove the old machine
- [x] remove the `datasets-preview-backend.huggingface.tech` domain
- [x] setup monitoring: see #250 and #279
- [x] remove old webhook: asked on [Slack](https://huggingface.slack.com/archives/C023JAKTR2P/p1654005947927899)

Use a kubernetes infrastructure: As done for https://github.com/huggingface/tensorboard-launcher/ and https://github.com/huggingface/moon-landing gitaly before.

- [x] API: see https://github.com/huggingface/datasets-server/pull/227
- [x] workers: see #236
- [x] nginx reverse proxy: see #234
- [x] deploy to the prod cluster: see #249
- [x] respond to `datasets-server.huggingface.tech`: see #222
- [x] add a webhook from moonlanding to the new datasets-server URL: asked on [Slack](https://huggingface.slack.com/archives/C023JAKTR2P/p1652790074627839)
- [x] warm the cache on all the public datasets - see the state of the cache here: https://grafana.huggingface.tech/d/SaHl2KX7z/datasets-server?orgId=1
- [x] update moonlanding: see https://github.com/huggingface/moon-landing/pull/2957
- [x] update the moonlanding proxy: see https://github.com/huggingface/conf/pull/166
- [x] update autotrain: no need, autonlp-ui accesses the server through [a proxy on moonlanding (https://huggingface.co/proxy-datasets-preview)](https://github.com/huggingface/autonlp-ui/blob/12a7fd84cdbf9c30900e3740f6b2bbbe38ae3699/src/lib/config.ts#L57-L58)
- [x] remove the old machine
- [x] remove the `datasets-preview-backend.huggingface.tech` domain
- [x] setup monitoring: see #250 and #279
- [x] remove old webhook: asked on [Slack](https://huggingface.slack.com/archives/C023JAKTR2P/p1654005947927899)
