bartowski

/

zai-org_GLM-4.6V-Flash-GGUF

like 14

I'll try it again but no it didn't manage to generate for some reason

Vision model without vision support is like a pirate without parrot.

Yeah I was surprised to see it convert at all but then not having vision, I assume there was something complicated, I'll dig around if I remember later

this model is supported text only now
details https://github.com/ggml-org/llama.cpp/pull/14823
pull request for vision https://github.com/ggml-org/llama.cpp/pull/16600

Yeah I was surprised to see it convert at all but then not having vision, I assume there was something complicated, I'll dig around if I remember later

Hey thanks for trying. Hopefully they fulfill the pull request for vision.

Thanks for jumping on this so fast! I take it that you'll be doing the 4.6V large model, too? You're the best!

Also, anyone else having trouble getting this to load with mmproj files? I can load the bare model without the mmproj files in TextGenWebUI/Oobabooga. But I've tried the bartowski BF16 mmproj as well as the ggml Q8 mmproj, and neither would allow the model to load.

???

Also, anyone else having trouble getting this to load with mmproj files? I can load the bare model without the mmproj files in TextGenWebUI/Oobabooga. But I've tried the bartowski BF16 mmproj as well as the ggml Q8 mmproj, and neither would allow the model to load.

???

Your LLM client needs at least this backend build: llama.cpp b7429 https://github.com/ggml-org/llama.cpp/releases/tag/b7429
I don't know if you can manually update llama.cpp in Oobabooga; otherwise, you'll need to wait until it's updated.

Also, anyone else having trouble getting this to load with mmproj files? I can load the bare model without the mmproj files in TextGenWebUI/Oobabooga. But I've tried the bartowski BF16 mmproj as well as the ggml Q8 mmproj, and neither would allow the model to load.

???

Your LLM client needs at least this backend build: llama.cpp b7429 https://github.com/ggml-org/llama.cpp/releases/tag/b7429
I don't know if you can manually update llama.cpp in Oobabooga; otherwise, you'll need to wait until it's updated.

Thanks! The latest Oobabooga was updated two days ago. So it didn't include these most recent changes.

So I just went to the link you posted... and then I downloaded 'llama-b7429-bin-win-cuda-12.4-x64.zip' and 'cudart-llama-bin-win-cuda-12.4-x64.zip' ... then I extracted their contents and copied all files to the Oobabooga llama.cpp bin folder which manually updates Oobabooga. And yes, the model instantly loaded with the mmproj file and vision capability 😎

Thanks for your help!

if anyone was using the bf16 mmproj there was apparently a bug with it, resolved on master here:

https://github.com/ggml-org/llama.cpp/pull/18124

pushed the new mmproj files here and to GLM-4.6V

Yeah I learned that the hard way. The BF16 mmproj wouldn't load for me. But I already had their Q8 mmproj, and I also grabbed your F16 mmproj just to see if there was a difference. I think yours was a little "smarter," but there's was a little more human-like. Idk, maybe it's just me tho? Anyway, it works! Thanks again for being so on top of this. It's a great model.