Index the (text) datasets contents to enable full-text search
See an example by @ola13 on the ROOTS corpus: https://huggingface.co/spaces/bigscience-data/roots-search.

Implementation details are here: https://huggingface.co/spaces/bigscience-data/scisearch/resolve/main/roots_search_tool_specs.pdf

Internal discussion: https://huggingface.slack.com/archives/C0311GZ7R6K/p1667586339997379
Index the (text) datasets contents to enable full-text search: See an example by @ola13 on the ROOTS corpus: https://huggingface.co/spaces/bigscience-data/roots-search.

Implementation details are here: https://huggingface.co/spaces/bigscience-data/scisearch/resolve/main/roots_search_tool_specs.pdf

Internal discussion: https://huggingface.slack.com/archives/C0311GZ7R6K/p1667586339997379