Instructions to use norecyc/lastbox-gemma4-e2b-v6-toolprior with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="norecyc/lastbox-gemma4-e2b-v6-toolprior")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("norecyc/lastbox-gemma4-e2b-v6-toolprior")
model = AutoModelForImageTextToText.from_pretrained("norecyc/lastbox-gemma4-e2b-v6-toolprior")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="norecyc/lastbox-gemma4-e2b-v6-toolprior",
	filename="lastbox-gemma4-e2b-v6-bf16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16
# Run inference directly in the terminal:
llama-cli -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16
# Run inference directly in the terminal:
llama-cli -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16
# Run inference directly in the terminal:
./llama-cli -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16

Use Docker

docker model run hf.co/norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16

LM Studio
Jan

vLLM

How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "norecyc/lastbox-gemma4-e2b-v6-toolprior"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/lastbox-gemma4-e2b-v6-toolprior",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16

SGLang

How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "norecyc/lastbox-gemma4-e2b-v6-toolprior" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/lastbox-gemma4-e2b-v6-toolprior",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "norecyc/lastbox-gemma4-e2b-v6-toolprior" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/lastbox-gemma4-e2b-v6-toolprior",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with Ollama:
```
ollama run hf.co/norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16
```

Unsloth Studio new

How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for norecyc/lastbox-gemma4-e2b-v6-toolprior to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for norecyc/lastbox-gemma4-e2b-v6-toolprior to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for norecyc/lastbox-gemma4-e2b-v6-toolprior to start chatting

Docker Model Runner
How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with Docker Model Runner:
```
docker model run hf.co/norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16
```

Lemonade

How to use norecyc/lastbox-gemma4-e2b-v6-toolprior with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull norecyc/lastbox-gemma4-e2b-v6-toolprior:BF16

Run and chat with the model

lemonade run user.lastbox-gemma4-e2b-v6-toolprior-BF16

List all available models

lemonade list

LastBox v6 — SFT-warmup checkpoint for 72 % tool emission on RPi 5

Post-deadline iteration of lastbox-gemma4-e2b-sft-v3. A 12-minute targeted SFT pass on 1 034 tool-only pairs took the agentic score from 0.016 → 0.608 (38×) — a result two GRPO iterations couldn't reach in 3 h of GB10 time. Same 2B params, same Q4_K_M quant, same 8 GB Raspberry Pi 5 deployment.

GitHub: https://github.com/agentsmill/lastbox
Live site: https://agentsmill.github.io/lastbox/
Submission snapshot: v1-submission tag

What changed vs v3

The v3 SFT trained on full dialogs that contained tool calls but the inference-time prompt structure suppressed them. Two fixes unlocked tool emission:

SFT warmup on tool-only pairs (train_v2_toolonly.jsonl, 1 034 [user, assistant_tool_call] pairs only). Sets the prior on first-turn tool emission.
Eval prompt fix — the original eval was passing 733-char SYSTEM_PROMPT_EN alone; training prompts had a 4 233-char system block with the 7-tool JSON whitelist embedded. Without that block the model correctly inferred "no tool defs ⇒ don't emit tool calls". Eval now uses process_v2._build_system_prompt_with_tools().
Non-streaming POST + 2-retry in eval — fixes the 13/25 streaming-SSE completion floor that was masquerading as a quality issue.

Three independent fixes, three independent metrics each unlocked.

Eval (golden_en, n=25, tool-allowed system + non-streaming POST)

Metric	v3 SFT (this base)	v6 (this checkpoint)	Δ
tool_emission_rate	~0%	72%	+72 pp
tool_accuracy	0%	64%	+64 pp
arg_validity	4%	56%	+52 pp
agentic_score	0.016	0.608	38×
byte_compliance	0.48	1.000	+0.52
format_ok	0.52	1.000	+0.48
persona_ok	0.52	1.000	+0.48
response_quality	0.506	1.000	+0.494
completed / 25	14	25	perfect

What we tried before this (and why it didn't work)

v4 GRPO (reward v1): 0.5·tool_match + 0.3·format + 0.2·byte_cap. 200 steps, num_generations=4. Reward climbed 0.5 → 0.7 in training but tool emission collapsed at eval. The reward gave equal credit to "emit correct tool" and "skip tool, give direct answer" — the no-tool path is easier to reach so GRPO converged there.
v5 GRPO (reward v2): tool-correct +1.0, no-tool-when-expected −0.5, spurious-tool 0.0. Stronger gradient toward tool emission. Same plateau — with KL β=0.04 the model cannot move from p(tool)≈0 to p(tool)≈1 in 200 steps; the KL term blocks the required policy shift.

The lesson: when SFT base has p(behaviour)≈0, targeted SFT to set the prior + GRPO for shaping is the right pipeline. A pure-GRPO assault on a behaviour the base never exhibits is fundamentally bounded by KL.

Files

File	Size	Purpose
`lastbox-gemma4-e2b-v6-q4_k_m.gguf`	3.2 GB	Quantized inference weights
`lastbox-gemma4-e2b-v6-bf16.gguf`	8.7 GB	bf16 source for quantization
`model.safetensors`	9.6 GB	Merged HF format
`lora/adapter_model.safetensors`	49 MB	LoRA adapter (on top of v3 base)
`tokenizer.json` + chat template + config	–	Required for inference

Quickstart — llama.cpp server

docker run --rm -p 11436:8080 \
  -v $(pwd):/models:ro \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/lastbox-gemma4-e2b-v6-q4_k_m.gguf \
  --host 0.0.0.0 --port 8080 \
  --ctx-size 4096 --threads 4 --parallel 1 \
  --jinja --chat-template-file /models/chat_template.jinja \
  --no-mmap --mlock

Training recipe

base:               norecyc/lastbox-gemma4-e2b-sft-v3 (HF format)
framework:          Unsloth FastModel + TRL SFTTrainer
LoRA:               r=8, alpha=8, dropout=0, bias="none"
target modules:     language layers + attention + MLP, no vision
chat template:      gemma-4 (no-thinking)
hardware:           1× NVIDIA GB10 (Grace Blackwell, 121 GB unified, CUDA 13)
data:               1 034 tool-only pairs (norecyc/lastbox-survival-dialogues, train_v2_toolonly)
epochs:             1 (65 optimizer steps)
lr:                 2e-4, cosine, warmup 10 steps
batch / grad accum: 4 / 4 (effective 16)
training time:      12 min on GB10
final train loss:   0.018

Important: matching the training prompt at inference time

The model emits <tool_call> only when given the full training-time system prompt with the 7-tool definitions JSON block. To reproduce the 72% tool emission rate at inference:

from gemma4.scripts.process_v2 import _build_system_prompt_with_tools
system = _build_system_prompt_with_tools()
# pass `system` as the system role; do NOT use the shorter SYSTEM_PROMPT_EN alone.

Without that block, the model degrades to plain text answers (which it still does well — byte/format/persona stay at 1.000).

Dataset

norecyc/lastbox-survival-dialogues — specifically the train_v2_toolonly config.

License

Model weights are subject to the Gemma Terms of Use. Training scripts and LoRA adapter weights are Apache 2.0 (see GitHub repo).

Citation

@misc{lastbox_gemma4_v6_2026,
  title  = {LastBox v6: SFT-warmup checkpoint with 72 % tool emission on Raspberry Pi 5},
  author = {Mateusz Pawelczuk},
  year   = {2026},
  url    = {https://huggingface.co/norecyc/lastbox-gemma4-e2b-v6-toolprior}
}

Downloads last month: 608

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for norecyc/lastbox-gemma4-e2b-v6-toolprior

Base model

google/gemma-4-E2B

Finetuned

google/gemma-4-E2B-it

Finetuned

unsloth/gemma-4-E2B-it

Adapter

norecyc/lastbox-gemma4-e2b-sft-v3