Determine if a string column is string or category by proportion of unique values
...instead of by an absolute value. Will solve https://github.com/huggingface/datasets-server/issues/1953
Set to 0.2.

UPD: i also added the maximum value of categories to 1000 (for example, there are ~700 languages in the-stack-v2). Yes, it makes no sense for the viewer to display 1k labels but i think it should be fixed in the frontend - for example by displaying only top-N categories and the number of unique values. Because computing histograms for classes also doesn't make any sense, especially if column is language  code and they are all of the same length. 
Also if there are <= num_histogram_bins (currently 10) - treat as category independently of the proportion value because the viewer will display this well for sure.


Ideally this should be fixed by manual configuration (hope to start working on this soon!) and for now i think it's okay like this.


Determine if a string column is string or category by proportion of unique values: ...instead of by an absolute value. Will solve https://github.com/huggingface/datasets-server/issues/1953
Set to 0.2.

UPD: i also added the maximum value of categories to 1000 (for example, there are ~700 languages in the-stack-v2). Yes, it makes no sense for the viewer to display 1k labels but i think it should be fixed in the frontend - for example by displaying only top-N categories and the number of unique values. Because computing histograms for classes also doesn't make any sense, especially if column is language  code and they are all of the same length. 
Also if there are <= num_histogram_bins (currently 10) - treat as category independently of the proportion value because the viewer will display this well for sure.


Ideally this should be fixed by manual configuration (hope to start working on this soon!) and for now i think it's okay like this.

