stukenov/sozkz-corpus-balanced-kk-gpt2-v1
Viewer
• Updated
• 480k • 42
Soz: Kazakh Language Models from Scratch
Building foundational language models for Kazakh — models, tokenizers, and training corpora.
Note Legacy tokenized corpus (v1, domain-balanced)
stukenov/sozkz-corpus-tokenized-kk-llama50m-v1
Viewer • Updated • 5.9M • 70Note Legacy tokenized corpus for LLaMA experiments (32K BPE)
stukenov/sozkz-core-llama-30m-kk-base-v1
Text Generation • 33.5M • Updated • 22Note LLaMA 30M — modern arch (RoPE, SwiGLU, RMSNorm)
stukenov/sozkz-core-llama-50m-kk-base-v1
Text Generation • 50.3M • Updated • 7Note LLaMA 50M — early from-scratch experiment
stukenov/sozkz-core-llama-150m-kk-base-v1
Text Generation • 0.2B • Updated • 64Note LLaMA 150M — largest early model
stukenov/sozkz-core-llama-50m-kk-balanced-v1
Text Generation • 50.3M • Updated • 10Note LLaMA 50M on balanced corpus
stukenov/sozkz-core-llama-150m-kk-balanced-v1
Text Generation • 0.2B • Updated • 15Note LLaMA 150M on balanced corpus
stukenov/sozkz-core-pythia-14m-kk-dapt-v1
Text Generation • 14.1M • Updated • 14Note Pythia 14M DAPT — first Kazakh LM experiment (proof of concept)
-
stukenov/sozkz-core-llama-50m-kk-base-v2
50.6M • Updated • 32 • 2 -
stukenov/sozkz-vocab-bpe-32k-kk-base-v1
Text Generation • Updated -
stukenov/sozkz-corpus-clean-kk-pretrain-v2
Viewer • Updated • 1.02M • 73 -
stukenov/sozkz-corpus-clean-kk-text-v2
Viewer • Updated • 19M • 41