Infinite loop IQ1_S
I have loaded the IQ1_S, but it gets stuck in an infinite answer loop at the end. Im using ollama and openwebui
no issues with IQ2_M
Yes, I too get the infinite loop with IQ1_S using llama.cpp with this command line: E:\llamacpp\bin\llama-cli.exe -m E:\llamacpp\models\minimax-m2.1-PRISM-IQ1_S.gguf -ngl 30 -c 8192 -cnv --temp 0.8 -p 0.9. Also tried E:\llamacpp\bin\llama-cli.exe -m E:\llamacpp\models\minimax-m2.1-PRISM-IQ1_S.gguf -ngl 20 -c 8192 -cnv --temp 0.7 --dry-multiplier 0.8 --dry-base 2.5. Always repeats answers and won't come up for air.
Yes solution also works for me - IQ2 (https://huggingface.co/Ex0bit/MiniMax-M2.1-PRISM/blob/main/MiniMax-M2.1-PRISM-IQ4_NL.gguf) works fantastic on a 4090 with 128gb ram with these parameters: E:\llamacpp\bin\llama-cli.exe -m E:\llamacpp\models\minimax-m2.1-PRISM-IQ2_M.gguf -ngl 18 -c 8192 -cnv --temp 0.7 --top_p 0.95 --top_k 40 --dry-multiplier 0.5 --dry-base 2.0 at around 11 t/s. Very exciting!