chikim

chikim, 4 months ago

@FreakyFwoof You can use Llava, but Llava is more designed for processing image in language. I't's not going to be as good as other regular LLMs specifically designed for chat like openhermes, solar, neural-chat, zephyr, etc.

chikim, 4 months ago

@FreakyFwoof You can read while it's streaming. You don't have to wait for it to finish. Just shift tab and read. Only prolem is cursor gets reset every time a new word arrives. You can command+a and command+c and paste somewhere too. lol

chikim, 4 months ago

@FreakyFwoof If you want to expose your llama to another machine on the network, type this in terminal and quit Ollama from menu extras and open it again.
launchctl setenv OLLAMA_HOST "0.0.0.0"

chikim, 4 months ago

@FreakyFwoof @FreakyFwoof I believe it'll persist, but not sure 100%.

chikim, 4 months ago

@FreakyFwoof Also different models produce different things, so you might want to try different ones. Some are even specifically designed not to avoid engaging in NSFW chat. lol

chikim, 4 months ago

@FreakyFwoof That's interesting. I didn't know Zephyr was not censored. lol

chikim, 4 months ago

@FreakyFwoof Yeah, I thought about it, so it might happen.

chikim, 4 months ago

@FreakyFwoof @vick21 Close VOLlama, copy a model: ollama cp zephyr andre. Open VOLlama, talk to andre first. Then talk to something else. With VOLlama open, delete andre: ollama rm andre. Then if you try to talk to andre in VOLlama, you should get an error.

chikim, 4 months ago

@FreakyFwoof @vick21 Maybe Ollama or llama.cpp that ollama uses have caching. Unfortunately I don't have control over behind what's going on.

chikim, 4 months ago

@FreakyFwoof Re caching: did you clear the previous message before asking the same question to another model? If not, the newly selected model will receive all the messages, including responses from the previous model you used before the switch. For the newly selected model, it will seem as though it has already answered the questions, and you are asking the exact question again.

chikim, 4 months ago

@FreakyFwoof Ah, you used the new chat. I also discovered that bug in alpha.2 while implementing saving history. lol

chikim, 4 months ago

@FreakyFwoof I think You'd have alot of fun with alpha.2. You can set system message like You're a funniest comedian. Make every response as funny as possible. lol Also you can copy model and name it Alex. You can also play with adjusting bunch of other parameters in modelfile like temperature that makes model more or less creative/wild. https://github.com/ollama/ollama/blob/main/docs/modelfile.md

chikim, 4 months ago

@FreakyFwoof Tell your wife my apollogy for ugly UI. I have no idea how interface looks visually. I have to ask my wife and fix things. lol

chikim, 4 months ago

@FreakyFwoof There are many web based clients out there with variety of accessibility issues. lol

chikim, 4 months ago

@FreakyFwoof If you go to their library, click model, and click tags. It'll show you the size.

chikim, 4 months ago

@FreakyFwoof Also you'll see like differetn quantized model with different sizes. Higher quantization means more accurate but bigger size. However, 7Bq8 is less accurate than 13Bq4. Parameter count like 7B 13B matters more. Also I wouldn't use less than Q4 unless it's absolutely necessary.

chikim, 4 months ago

@FreakyFwoof They have 34B. Slower but more accurate. You have to decide how much youre willing to tolerate the slow speed.

chikim, 4 months ago

@FreakyFwoof Also that reminds me... they updated their model recently to v1.6, so I would update it with ollama pull llava:13b.

chikim, 4 months ago

@pitermach No it's cross platform mac and windows. Mac is easier to get going, because Ollama is easier to setup on Mac. Anyways, Dropbox said Already too many people downloaded, so they disabled. the link. It can't be that many people. Ha, What a BS! I'll uploaded to github momentarily.

chikim, 4 months ago

@FreakyFwoof @ZBennoui You'll be fine since you're used to how QT interfaces behave with VoiceOver. :)

chikim, 4 months ago

@FreakyFwoof @vick21 I don't need to change anything. Also if you have multiple multimodal odels, VOCR will ask you to choose which one to use. However, it's interesting Llava 34B is based on newer Llava v1.6 architecture, and Llama.cpp doesn't support it yet. There's a PR for partial support.

chikim, 4 months ago

@FreakyFwoof @vick21 Worth to try it! It's on their website, so why not. Make sure to update Ollama though because Llava v1.6 has new architecture. FYI, 34B need over 20GB. Make sure you have enough space for both SSD and Memory.