Support JWT on cookies
private conversion: https://huggingface.slack.com/archives/D030YA5BW91/p1696507761676679

When the users goes to https://huggingface.co/datasets/emrgnt-cmplxty/sciphi-textbooks-are-all-you-need, moonlanding can put a cookie on `datasets-server.huggingface.co` with name `hf_jwt_[sha256(/datasets/emrgnt-cmplxty/sciphi-textbooks-are-all-you-need)]` and with the JWT as the value

This cookie would be read on datasets server when accessing a gated dataset

Doing so would simplify a lot the code on the Hub (moonlanding) by removing the need to refresh the JWT (remove an endpoint), and avoid the logic in the frontend code that refreshes the JWT. It would be a security improvement too, because the Hub's frontend code (javascript) would no more have access to the JWT (the browser directly adds the cookie to the HTTP request)
Support JWT on cookies: private conversion: https://huggingface.slack.com/archives/D030YA5BW91/p1696507761676679

When the users goes to https://huggingface.co/datasets/emrgnt-cmplxty/sciphi-textbooks-are-all-you-need, moonlanding can put a cookie on `datasets-server.huggingface.co` with name `hf_jwt_[sha256(/datasets/emrgnt-cmplxty/sciphi-textbooks-are-all-you-need)]` and with the JWT as the value

This cookie would be read on datasets server when accessing a gated dataset

Doing so would simplify a lot the code on the Hub (moonlanding) by removing the need to refresh the JWT (remove an endpoint), and avoid the logic in the frontend code that refreshes the JWT. It would be a security improvement too, because the Hub's frontend code (javascript) would no more have access to the JWT (the browser directly adds the cookie to the HTTP request)