Prevent DoS when accessing some datasets
For example: https://huggingface.co/datasets/allenai/c4 script is doing 69219 output requests **on every received request**, which occupies all the CPUs. 

```
pm2 logs
```

```
0|datasets | INFO:     3.238.194.17:0 - "GET /configs?dataset=allenai/c4 HTTP/1.1" 200 OK
```

```
Check remote data files:  78%|███████▊  | 54330/69219 [14:13<03:05, 80.10it/s]
Check remote data files:  79%|███████▊  | 54349/69219 [14:13<03:51, 64.14it/s]
Check remote data files:  79%|███████▊  | 54364/69219 [14:14<04:44, 52.14it/s]
Check remote data files:  79%|███████▊  | 54375/69219 [14:14<04:48, 51.38it/s]
Check remote data files:  79%|███████▊  | 54448/69219 [14:15<02:37, 93.81it/s]
Check remote data files:  79%|███████▉  | 54543/69219 [14:15<01:56, 125.60it/s]
Check remote data files:  79%|███████▉  | 54564/69219 [14:16<03:22, 72.33it/s]
```
Prevent DoS when accessing some datasets: For example: https://huggingface.co/datasets/allenai/c4 script is doing 69219 output requests **on every received request**, which occupies all the CPUs. 

```
pm2 logs
```

```
0|datasets | INFO:     3.238.194.17:0 - "GET /configs?dataset=allenai/c4 HTTP/1.1" 200 OK
```

```
Check remote data files:  78%|███████▊  | 54330/69219 [14:13<03:05, 80.10it/s]
Check remote data files:  79%|███████▊  | 54349/69219 [14:13<03:51, 64.14it/s]
Check remote data files:  79%|███████▊  | 54364/69219 [14:14<04:44, 52.14it/s]
Check remote data files:  79%|███████▊  | 54375/69219 [14:14<04:48, 51.38it/s]
Check remote data files:  79%|███████▊  | 54448/69219 [14:15<02:37, 93.81it/s]
Check remote data files:  79%|███████▉  | 54543/69219 [14:15<01:56, 125.60it/s]
Check remote data files:  79%|███████▉  | 54564/69219 [14:16<03:22, 72.33it/s]
```