Post
8
IBM Granite 4.1 series
New models came up, here is how they compare to models in the same size:
Brainwaves
Gemma-4
Qwen3.5
Metrics for the 30B will be available soon.
Right out of the gate, IBM delivered models with better starting metrics than both Gemma and Qwen. Training these should be fun :)
Here is the Nightmedia collection of Granite models
https://huggingface.co/collections/nightmedia/ibm-granite-41
-G
New models came up, here is how they compare to models in the same size:
Brainwaves
quant arc arc/e boolq hswag obkqa piqa wino
granite-4.1-8b
mxfp8 0.486,0.666,0.875,0.636,0.450,0.766,0.631
granite-4.1-3b
mxfp8 0.406,0.581,0.821,0.484,0.434,0.712,0.559Gemma-4
quant arc arc/e boolq hswag obkqa piqa wino
gemma-4-E4B-it
mxfp8 0.480,0.656,0.797,0.608,0.400,0.755,0.665
mxfp4 0.455,0.607,0.851,0.585,0.402,0.744,0.651
gemma-4-E2B-it
mxfp8 0.376,0.464,0.743,0.490,0.378,0.709,0.622
mxfp4 0.380,0.451,0.762,0.494,0.374,0.699,0.594Qwen3.5
quant arc arc/e boolq hswag obkqa piqa wino
Qwen3.5-9B
mxfp8 0.417,0.458,0.623,0.634,0.338,0.737,0.639
mxfp4 0.419,0.472,0.622,0.634,0.352,0.739,0.644
Qwen3.5-4B
mxfp8 0.392,0.441,0.627,0.601,0.360,0.739,0.590
mxfp4 0.371,0.444,0.632,0.585,0.356,0.732,0.548Metrics for the 30B will be available soon.
Right out of the gate, IBM delivered models with better starting metrics than both Gemma and Qwen. Training these should be fun :)
Here is the Nightmedia collection of Granite models
https://huggingface.co/collections/nightmedia/ibm-granite-41
-G