Prompt
Hey there any Prompt for general conversation?
Hey there any Prompt for general conversation?
There should be are you having issues with running the model?
No not a problem really i run the IQ1_S Quant in LM Studio and the output is very inconsistent but i think that is to be expected with that quantization?
No not a problem really i run the IQ1_S Quant in LM Studio and the output is very inconsistent but i think that is to be expected with that quantization?
Yes that is expected. Lower quants tend to perform worse the lower they get. Try increasing until you find one that works for you. Feedback is welcome on what you find. I do tweak quantisation based on feedback
I have queued it to be converted again as it is actually using the older quantization method. I am interested to see how the new method performs on this model.
Nice! I was curious how the smallest avail quantization works. I will tinker and let you know how its going! Thx for your work anyways.