hayesstw, 8 months ago

@killyourfm Will it run on Windows XP?

killyourfm, 8 months ago

@hayesstw I honestly don't know as I couldn't find hardware requirements at a glance. Might be a good question to ask the developers.

ReverseModule, 8 months ago

deleted_by_author

killyourfm, 8 months ago

@ReverseModule My pleasure! (I miss sharing cool open source apps...)

bill88t, 8 months ago

@killyourfm You may also wanna take a look at koboldcpp which is a jack of all trades.
It pretty much supports every model type context size and setting you could imagine.
Doesn't play nice with cuda 12 atm, but runs well with just cpu.
Also plays very nicely inside of docker (even with gpu acceleration).

plwt, 8 months ago

@killyourfm Strongly recommend this for a weekend project - I have tried it and it is good fun.

paninid, 8 months ago

@killyourfm FWIW, it will eat up processor bandwidth when running. Multi-tasking not an option.

killyourfm, 8 months ago

@paninid That's interesting to know, thanks! I haven't taken a look at its performance footprint yet.

radasbona, 8 months ago

@killyourfm could this be the beginning for an "open source Siri" on android?

killyourfm, 8 months ago

@radasbona I'm not a developer or an AI expert, so I'd hesitate to give any kind of concrete answer.

But I do have hope! Mozilla AI is working on some special things that use this as its foundation.

(Also, knowing that Firefox Translations runs locally and is only about 17MB inspires even more hope)

razze, 8 months ago

@killyourfm did you compare with localAI? That's integrated with nextcloud.

killyourfm, 8 months ago

@razze No I haven't. I'm just barely starting to drink from the open-source AI firehose.

razze, 8 months ago

@killyourfm was just curios if you could report

FrayJay, 8 months ago

@killyourfm @razze ouf that's a deep rabbit hole. Recommend looking at orca and wizzardlm models ;)

shipp, 8 months ago

deleted_by_author

killyourfm, 8 months ago

@shipp It also sounds like they're about to get sued into the ground on multiple fronts.

I guess this is where profit-driven goals and greed without any oversight gets you, huh?