Upload folder using huggingface_hub

README.md

@@ -21,30 +21,57 @@ Omni Senter is a trained version of Qwen2.5-Omni-3B that can:
 
 Perfect for building voice-controlled phone assistants!
 
-## Quick Start
+## Download & Setup (Recommended)
 
-### Running with llama.cpp
+We've included a download script that automatically gets the base model and applies our LoRA adapter:
 
 ```bash
-llama-server \
-  -m /path/to/Qwen2.5-Omni-3B-Q4_K_M.gguf \
-  --mmproj /path/to/mmproj-Qwen2.5-Omni-3B-Q8_0.gguf \
-  --lora omni-senter-3b.gguf \
-  --ctx-size 8192 \
-  --n-gpu-layers -1 \
+# 1. Clone this repo
+git lfs install
+git clone https://huggingface.co/sovthpaw/Omni-Senter-3B
+
+# 2. Run the setup script
+cd Omni-Senter-3B
+chmod +x download_omni_senter.sh
+./download_omni_senter.sh
+
+# 3. Run the model
+llama-server -m ~/.cache/llama.cpp/models/omni-senter-merged.gguf \
+  --mmproj ~/.cache/llama.cpp/models/mmproj-Qwen2.5-Omni-3B-Q8_0.gguf \
   --port 8107
 ```
 
-### Merging with HF Model
+The script will:
+1. Download Qwen2.5-Omni GGUF base model (~4GB)
+2. Download mmproj for vision/audio (~1GB)
+3. Download our LoRA adapter (46MB)
+4. Apply the LoRA and create a merged model
 
-```python
-from transformers import AutoModelForCausalLM
-from peft import PeftModel
+## Quick Start (Manual)
 
-base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Omni-3B")
-model = PeftModel.from_pretrained(base, ".")
-merged = model.merge_and_unload()
-merged.save_pretrained("omni-senter-3b-merged")
+### Option 1: Use LoRA with llama.cpp
+
+```bash
+# Download base model
+huggingface-cli download ggml-org/Qwen2.5-Omni-3B-GGUF Qwen2.5-Omni-3B-Q4_K_M.gguf
+huggingface-cli download ggml-org/Qwen2.5-Omni-3B-GGUF mmproj-Qwen2.5-Omni-3B-Q8_0.gguf
+
+# Download LoRA adapter from this repo
+huggingface-cli download sovthpaw/Omni-Senter-3B senter-lora-500.gguf
+
+# Apply LoRA
+llama-export-lora -m Qwen2.5-Omni-3B-Q4_K_M.gguf -o omni-senter.gguf --lora senter-lora-500.gguf
+
+# Run
+llama-server -m omni-senter.gguf --mmproj mmproj-Qwen2.5-Omni-3B-Q8_0.gguf --port 8107
+```
+
+### Option 2: Use Full HF Model (8.8GB)
+
+```bash
+# Download full merged model
+git lfs install
+git clone https://huggingface.co/sovthpaw/Omni-Senter-3B-full
 ```
 
 ## Input Formats
@@ -129,11 +156,19 @@ Model: <speak>I hear you! You asked me to open the browser.</speak>
 
 ## Files
 
-- `omni-senter-3b.gguf` - LoRA adapter (apply to base GGUF model)
-- `adapter_model.safetensors` - LoRA weights (for HF merge)
+- `download_omni_senter.sh` - Automated download & setup script
+- `senter-lora-500.gguf` - LoRA adapter (46MB) - apply to base GGUF model
+- `adapter_model.safetensors` - LoRA weights for HF merge
 - `adapter_config.json` - LoRA configuration
 - `tokenizer*` - Tokenizer files
 
+## Full Model Download
+
+For the fully merged model (8.8GB, no LoRA needed):
+```bash
+git clone https://huggingface.co/sovthpaw/Omni-Senter-3B-full
+```
+
 ## Requirements
 
 - Qwen2.5-Omni-3B GGUF model

download_omni_senter.sh

@@ -0,0 +1,52 @@
+#!/bin/bash
+# Download and setup Omni Senter 3B
+# This script downloads the base Qwen2.5-Omni GGUF and applies our LoRA adapter
+
+set -e
+
+MODEL_DIR="$HOME/.cache/llama.cpp/models"
+mkdir -p "$MODEL_DIR"
+
+echo "=== Omni Senter 3B Setup ==="
+
+# Download base model (Qwen2.5-Omni GGUF)
+BASE_MODEL="$MODEL_DIR/Qwen2.5-Omni-3B-Q4_K_M.gguf"
+if [ ! -f "$BASE_MODEL" ]; then
+    echo "Downloading Qwen2.5-Omni-3B GGUF model..."
+    huggingface-cli download ggml-org/Qwen2.5-Omni-3B-GGUF Qwen2.5-Omni-3B-Q4_K_M.gguf \
+        --local-dir "$MODEL_DIR" \
+        --token HF_TOKEN
+else
+    echo "Base model already exists: $BASE_MODEL"
+fi
+
+# Download mmproj
+MMPROJ="$MODEL_DIR/mmproj-Qwen2.5-Omni-3B-Q8_0.gguf"
+if [ ! -f "$MMPROJ" ]; then
+    echo "Downloading mmproj..."
+    huggingface-cli download ggml-org/Qwen2.5-Omni-3B-GGUF mmproj-Qwen2.5-Omni-3B-Q8_0.gguf \
+        --local-dir "$MODEL_DIR" \
+        --token HF_TOKEN
+else
+    echo "mmproj already exists: $MMPROJ"
+fi
+
+# Download LoRA adapter
+LORA_DIR="$HOME/.cache/llama.cpp/omni-senter"
+mkdir -p "$LORA_DIR"
+echo "Downloading Omni Senter LoRA adapter..."
+huggingface-cli download sovthpaw/Omni-Senter-3B senter-lora-500.gguf \
+    --local-dir "$LORA_DIR" \
+    --token HF_TOKEN
+
+LORA_FILE="$LORA_DIR/senter-lora-500.gguf"
+
+# Apply LoRA and create merged model
+MERGED_MODEL="$MODEL_DIR/omni-senter-merged.gguf"
+echo "Applying LoRA adapter..."
+./llama-export-lora -m "$BASE_MODEL" -o "$MERGED_MODEL" --lora "$LORA_FILE"
+
+echo ""
+echo "=== Setup Complete! ==="
+echo "Run with:"
+echo "  llama-server -m $MERGED_MODEL --mmproj $MMPROJ --port 8107"