Update README.md
Browse files
README.md
CHANGED
|
@@ -193,6 +193,10 @@ add the following to the command line to start the "llamacpp server" (CLI):
|
|
| 193 |
|
| 194 |
(no quotes, where "6" is the number of experts to use)
|
| 195 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 196 |
When using "API", you set the "num_experts_used" in the JSON payload (this maybe different for different back ends).
|
| 197 |
|
| 198 |
CREDITS:
|
|
|
|
| 193 |
|
| 194 |
(no quotes, where "6" is the number of experts to use)
|
| 195 |
|
| 196 |
+
FOR QWEN MODELS:
|
| 197 |
+
|
| 198 |
+
"--override-kv qwen3moe.expert_used_count=int:6" (where 6 is the number of experts per token).
|
| 199 |
+
|
| 200 |
When using "API", you set the "num_experts_used" in the JSON payload (this maybe different for different back ends).
|
| 201 |
|
| 202 |
CREDITS:
|