---
license: apache-2.0
base_model: Qwen/Qwen3-Reranker-0.6B
library_name: node-llama-cpp
pipeline_tag: text-ranking
quantized_by: giladgd
tags:
- node-llama-cpp
- llama.cpp
---

# Qwen3-Reranker-0.6B-GGUF

Static quants of [`Qwen/Qwen3-Reranker-0.6B`](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B).


## Quants
| Link | [URI](https://node-llama-cpp.withcat.ai/cli/pull) | Quant | Size |
|:-----|:--------------------------------------------------|:------|-----:|
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q2_K.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q2_K` | Q2_K | 296.0MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q3_K_S.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q3_K_S` | Q3_K_S | 322.8MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q3_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q3_K_M` | Q3_K_M | 346.9MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q3_K_L.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q3_K_L` | Q3_K_L | 368.3MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q4_0.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q4_0` | Q4_0 | 381.3MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q4_K_S.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q4_K_S` | Q4_K_S | 383.0MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q4_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q4_K_M` | Q4_K_M | 396.5MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q5_0.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q5_0` | Q5_0 | 436.4MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q5_K_S.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q5_K_S` | Q5_K_S | 436.4MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q5_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q5_K_M` | Q5_K_M | 444.2MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q6_K.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q6_K` | Q6_K | 494.9MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q8_0.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0` | Q8_0 | 639.2MB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.F16.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:F16` | F16 | 1.2GB |

> [!TIP]
> Download a quant using `node-llama-cpp` ([more info](https://node-llama-cpp.withcat.ai/cli/pull)):
> ```bash
> npx -y node-llama-cpp pull <URI>
> ```


# Usage
## Use with [`node-llama-cpp`](https://node-llama-cpp.withcat.ai) (recommended)
Ensure you have node.js installed:
```bash
brew install nodejs
```

### CLI
Chat with the model:
```bash
npx -y node-llama-cpp chat hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0
```

### Code
Use it in your project:
```bash
npm install node-llama-cpp
```

```typescript
import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";

const modelUri = "hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0";


const llama = await getLlama();
const model = await llama.loadModel({
    modelPath: await resolveModelFile(modelUri)
});
const context = await model.createContext();
const session = new LlamaChatSession({
    contextSequence: context.getSequence()
});


const q1 = "Hi there, how are you?";
console.log("User: " + q1);

const a1 = await session.prompt(q1);
console.log("AI: " + a1);
```

> [!TIP]
> Read the [getting started guide](https://node-llama-cpp.withcat.ai/guide/) to quickly scaffold a new `node-llama-cpp` project


## Use with [llama.cpp](https://github.com/ggml-org/llama.cpp)
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp
```

### CLI
```bash
llama-cli -hf giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0 -p "The meaning to life and the universe is"
```

### Server
```bash
llama-server -hf giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0 -c 2048
```