---
license: apache-2.0
base_model: Qwen/Qwen3-Reranker-8B
library_name: node-llama-cpp
pipeline_tag: text-ranking
quantized_by: giladgd
tags:
- node-llama-cpp
- llama.cpp
---

# Qwen3-Reranker-8B-GGUF

Static quants of [`Qwen/Qwen3-Reranker-8B`](https://huggingface.co/Qwen/Qwen3-Reranker-8B).


## Quants
| Link | [URI](https://node-llama-cpp.withcat.ai/cli/pull) | Quant | Size |
|:-----|:--------------------------------------------------|:------|-----:|
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q2_K.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q2_K` | Q2_K | 3.1GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q3_K_S.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q3_K_S` | Q3_K_S | 3.5GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q3_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q3_K_M` | Q3_K_M | 3.9GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q3_K_L.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q3_K_L` | Q3_K_L | 4.2GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q4_0.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q4_0` | Q4_0 | 4.4GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q4_K_S.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q4_K_S` | Q4_K_S | 4.5GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q4_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q4_K_M` | Q4_K_M | 4.7GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q5_0.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q5_0` | Q5_0 | 5.3GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q5_K_S.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q5_K_S` | Q5_K_S | 5.3GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q5_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q5_K_M` | Q5_K_M | 5.4GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q6_K.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q6_K` | Q6_K | 6.2GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.Q8_0.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:Q8_0` | Q8_0 | 8.0GB |
| [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-8B-GGUF/resolve/main/Qwen3-Reranker-8B.F16.gguf) | `hf:giladgd/Qwen3-Reranker-8B-GGUF:F16` | F16 | 15.1GB |

> [!TIP]
> Download a quant using `node-llama-cpp` ([more info](https://node-llama-cpp.withcat.ai/cli/pull)):
> ```bash
> npx -y node-llama-cpp pull <URI>
> ```


# Usage
## Use with [`node-llama-cpp`](https://node-llama-cpp.withcat.ai) (recommended)
Ensure you have node.js installed:
```bash
brew install nodejs
```

### CLI
Chat with the model:
```bash
npx -y node-llama-cpp chat hf:giladgd/Qwen3-Reranker-8B-GGUF:Q4_K_M
```

### Code
Use it in your project:
```bash
npm install node-llama-cpp
```

```typescript
import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";

const modelUri = "hf:giladgd/Qwen3-Reranker-8B-GGUF:Q4_K_M";


const llama = await getLlama();
const model = await llama.loadModel({
    modelPath: await resolveModelFile(modelUri)
});
const context = await model.createContext();
const session = new LlamaChatSession({
    contextSequence: context.getSequence()
});


const q1 = "Hi there, how are you?";
console.log("User: " + q1);

const a1 = await session.prompt(q1);
console.log("AI: " + a1);
```

> [!TIP]
> Read the [getting started guide](https://node-llama-cpp.withcat.ai/guide/) to quickly scaffold a new `node-llama-cpp` project


## Use with [llama.cpp](https://github.com/ggml-org/llama.cpp)
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp
```

### CLI
```bash
llama-cli -hf giladgd/Qwen3-Reranker-8B-GGUF:Q4_K_M -p "The meaning to life and the universe is"
```

### Server
```bash
llama-server -hf giladgd/Qwen3-Reranker-8B-GGUF:Q4_K_M -c 2048
```