Quantized Qwen3.5
Collection
Verified models. Compatible with Transformers v5.3 and vLLM v0.16.1rc1 (nightly). Under evaluation. • 9 items • Updated • 9
This is Qwen/Qwen3.5-27B quantized with AutoRound to NVFP4 with linear-attention layers kept in 16-bit. The model is compatible with vLLM (tested: v0.16.1rc1). Tested with an RTX Pro 6000. Currently under evaluation.
- Developed by: The Kaitchup
Instructions
uv pip install vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly
uv pip install git+https://github.com/huggingface/transformers.git
vllm serve [this model ID] --max-model-len 262144 --reasoning-parser qwen3
Acknowledgments
Thank you Verda for providing the needed compute. I used their B200s for this. Verda is a European, AI-focused cloud and GPU infrastructure provider with sovereignty, sustainability, data privacy, and performance at its core. Check them out if interested.
- Downloads last month
- 1,133
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for kaitchup/Qwen3.5-27B-autoround-NVFP4-linearattn-BF16
Base model
Qwen/Qwen3.5-27B