Hugging Face's logo

mlabonne
/
gemma-3-12b-it-abliterated-GGUF

Image-Text-to-Text
Transformers
GGUF
autoquant
conversational
Model card Files
xet
 Community
5

Any tips on running this? It's just yapping nonsense at me...

#2
by deleted - opened
Discussion
deleted

I'm having trouble making this AI do anything efficiently, I don't know why, it's just saying a lot of nonsense, most of it unrelated to any message I send... are there recommended settings to be used please?
mlabonne
 Owner

Make sure you're using quants with high bitrates (>=4) and that the software you use is up-to-date.

I recommend using these generation parameters: temperature=1.0, top_k=64, top_p=0.95.
deleted

Sadly I was using the latest KCPP with q8 :( Maybe it's just not good at what I ask it to do...

Sign up or log in to comment