high hallucinations
#1
by
krustik
- opened
I've tried it with that special branch of llama and got very high hallucinations with code test. It just stuck in reasoning forever loop trying to determine the correct notes melody.
I will quantize myself the original model into Q8 version and try that, but 96 model shards are not so user friendly to download, it fact the main problem to get it from huggingface at all, git in xet or lfs (no matter) clonning they disconnect for some reason all the time.
Q4 uses ~150Gb RAM