ben, 6 months ago @simon is there a llamafile for llava 1.5 13B? Your blog post is awesome, and now I'm hoping to try the larger model.
@simon is there a llamafile for llava 1.5 13B? Your blog post is awesome, and now I'm hoping to try the larger model.
freakazoid, 6 months ago @ben @simon What format? There are lots. https://huggingface.co/models?sort=trending&search=llava+1.5+13b
@ben @simon What format? There are lots. https://huggingface.co/models?sort=trending&search=llava+1.5+13b
ben, 6 months ago @freakazoid @simon specifically Justine Tunney's llamafile format: https://simonwillison.net/2023/Nov/29/llamafile/
@freakazoid @simon specifically Justine Tunney's llamafile format: https://simonwillison.net/2023/Nov/29/llamafile/
freakazoid, 6 months ago @ben @simon Oh, neat! It appears to be a self-executing GGUF file. There are instructions on that page for creating a llamafile from a GGUF file. I found one GGUF quantization of it. I'd recommend using the Q5_K_M version. https://huggingface.co/PsiPi/liuhaotian_llava-v1.5-13b-GGUF/tree/main
@ben @simon Oh, neat! It appears to be a self-executing GGUF file.
There are instructions on that page for creating a llamafile from a GGUF file. I found one GGUF quantization of it. I'd recommend using the Q5_K_M version. https://huggingface.co/PsiPi/liuhaotian_llava-v1.5-13b-GGUF/tree/main
freakazoid, 6 months ago @ben @simon Err, actually I guess the instructions for making a llamafile are in the llamafile README. https://github.com/Mozilla-Ocho/llamafile
@ben @simon Err, actually I guess the instructions for making a llamafile are in the llamafile README. https://github.com/Mozilla-Ocho/llamafile
ben, 6 months ago @freakazoid @simon double thanks
@freakazoid @simon double thanks
simon, 6 months ago @ben @freakazoid you should be need to make a new llamafile to try it out if you download the GGUF and run it like this: https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-trying-other-models
@ben @freakazoid you should be need to make a new llamafile to try it out if you download the GGUF and run it like this: https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-trying-other-models
Add comment