@chikim@mastodon.social avatar

chikim

@chikim@mastodon.social

Love music, technology, accessibility! Faculty at Berklee College of Music 👨🏻‍💻🎹🐕‍🦺

This profile is from a federated server and may be incomplete. Browse more on the original instance.

chikim, to random
@chikim@mastodon.social avatar

Well Logic Pro on Ipad now has built in stem separation like Demucs. lol

macrumors, to random
@macrumors@mastodon.social avatar

Apple Event Live Blog: New iPads and Accessories Expected https://www.macrumors.com/2024/05/07/apple-event-live-blog-may-2024/?utm_source=dlvr.it&utm_medium=mastodon

chikim,
@chikim@mastodon.social avatar

@macrumors Hi, do you guys have MacRumorsLive on Mastodon?

bryansmart, to random

@chikim I've really been enjoying VOLlama. Nice work! Would be nice to be able to switch between OpenAI and local models without going in to API prefs. More accelerator keys for menu options would be good, too. Could maybe a blank line be inserted in the log between each entry? Last, can you trap key-down on the Control key to stop the system voice? I know it's a hobby project, so no idea how much time you have for any of that, but just throwing them out there.

chikim,
@chikim@mastodon.social avatar

@bryansmart Thanks. What do you mean by blank line between each entry? Like user: bla bla blank line llama: bla bla blank line? This would be easiest request to implement. Switching between platform is little tricky bc I have to keep track of what model you used with which platform. I'm sure there's a way, but catching modifier by itself will be also tricky. Pause/resume will be significantly more work because how each system implements api call for tts. Also I'm using threading to feed text.

chikim,
@chikim@mastodon.social avatar

@bryansmart the Alternative is press alt or option+up to go into the edit mode, and it'll paste only one message at a time into the prompt field.

chikim,
@chikim@mastodon.social avatar

@bryansmart Alt or option+up/down will let you edit the history context. It'll show one message at a time into the prompt field and lets you edit. If you want to just review it without editing it, just escape or alt/option+dwn all the way to the bottom when you're done reading.

chikim,
@chikim@mastodon.social avatar

@bryansmart Hmm, what kind of error? @vick21 also told me he got out of memory error during creating embedding.

chikim,
@chikim@mastodon.social avatar

@bryansmart @vick21 ah, you forgot to download the embedding model. ollama pull nomic-embed-text

chikim,
@chikim@mastodon.social avatar

@bryansmart @vick21 No problem. I totally get it. I hate writing manual/user guide/readme as well as reading them. lol

chikim,
@chikim@mastodon.social avatar

@vick21 @bryansmart That's actually great idea! VOLLlama has only few files, so I should try feeding everything and ask to write manual for it. hahahaha

chikim,
@chikim@mastodon.social avatar

@bryansmart Did you start your question with /q?

chikim,
@chikim@mastodon.social avatar

@bryansmart Also, you can't ask general questions like summarize that require reading the entire thing. The way it works it compares your question against documents through embedding, and retrieve and feed few chunks that might be relevant to answer your question.

chikim,
@chikim@mastodon.social avatar

@bryansmart If you want to see which chunks are fed into the model in order to answer your question, you can check the only checkbox in RAG settings. lol I don't think it's labeled in mac due to wx widget bug on Mac.

chikim,
@chikim@mastodon.social avatar

@bryansmart Embedding model is only used during indexing. The quality of the answer depends on the model you are chatting with. Because it will read the chunks in text and give you the answer. It really depends on if the LlamaIndex was able to retrieve the relevant chunks or not. You can increase number of chunks and chunk length, but you might end up feeding chunks that are not related to your question. Also there's threshold you can change to filter out below certain simularity score.

chikim, to llm
@chikim@mastodon.social avatar

I created a multi-needle in a haystack test where a randomly selected secret sentence was split into pieces and scattered throughout the document with 7.5k tokens in random places. The task was to find these pieces and reconstruct the complete sentence with exact words, punctuation, capitalization, and sequence. After running 100 tests, llama3:8b-instruct-q8 achieved a 44% success rate, while llama3:70b-instruct-q8 achieved 100%! https://github.com/chigkim/haystack-test

chikim, to llm
@chikim@mastodon.social avatar

VOLlama v0.1.0, an open-source, accessible chat client for OLlama
Unfortunately, many user interfaces for open source large language models are either inaccessible or annoying to use with screen readers, so I decided to make one for myself and others. Non screen reder users are welcome to use it as well.
I hope that ML UI libraries like Streamlit and Gradio will become more friendly with screen readers in the future, so making apps like this is not necessary!

https://chigkim.github.io/VOLlama/

chikim, to llm
@chikim@mastodon.social avatar

Wow, Private LLM locally runs Llama-3-8B on iOS. No idea how accessible with VoiceOver though. https://privatellm.app/en

chikim, to random
@chikim@mastodon.social avatar

For those of you who have used VOLlama, thank you for testing my hobby project! I'm considering moving it out of pre-release and marking the latest build as the first public release. Any thoughts on its stability or issues with bugs? Is it fairly stable or unstable? Of course, like all my other projects, it'll be free open source! @vick21 @technowitch @FreakyFwoof @kaveinthran @pixelate @ppatel

chikim,
@chikim@mastodon.social avatar

@vick21 @technowitch @FreakyFwoof @kaveinthran @pixelate @ppatel I'm sure too. Also all the ridiculous complaints from entitled people. haha

chikim,
@chikim@mastodon.social avatar

@vick21 @technowitch @FreakyFwoof @kaveinthran @pixelate @ppatel Have you disable the smart quotes from system settings > Keyboard?

chikim,
@chikim@mastodon.social avatar

I disabled the single file package for VOLlama 0.1.2. The file size got bigger, but it loads much much faster because it doesn't have to unpack itself every time when you open it.
https://chigkim.github.io/VOLlama/
@vick21 @technowitch @FreakyFwoof @kaveinthran @pixelate @ppatel

chikim, to llm
@chikim@mastodon.social avatar

Tired of neutral responses from LLMs? Llama-3 seems great at following system prompts, so try this system prompt for an opinionated chatbot.
"You are a helpful, opinionated, decisive assistant. When asked a yes/no question, begin your respond with one word answer: yes or no. For open-ended or complex questions, adopt a firm stance. Justify your views with well-reasoned arguments, robust evidence, and succinct explanations, ensuring clarity and confidence in every response."

chikim, to llm
@chikim@mastodon.social avatar

Mark Zuckerberg on Llama 3: Apparently Meta stopped training Llama-3-70b before convergence and decided to move onto Llama-4. Meaning they could have kept training and made it smarter! Also llama3-70b multimodal as well as multilingual and bigger context window are coming. https://youtu.be/bc6uFV9CJGg

chikim,
@chikim@mastodon.social avatar

@kellogh They're on a race with other companies, so I guess it makes sense. You want to move onto next quicly and get better.

chikim,
@chikim@mastodon.social avatar

@kellogh Giving AI community free opensource at the same time having to answer the board and investors, I can understand his decision though. If you don't like his style, you can move onto other open source models. :)

chikim, to llm
@chikim@mastodon.social avatar

Start saving money for that M4 Ultra with 500GB! Maybe this could be the first open source that could surpass GPT-4! AIatMeta: "Llama 3 8B & 70B models are just the beginning of what we’re working to release for Llama 3. Our largest models currently in the works are 400B+ parameters and while they’re still in active development, we’re excited about how this work is trending." https://twitter.com/AIatMeta/status/1780997414071181370

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • anitta
  • Durango
  • ngwrru68w68
  • thenastyranch
  • magazineikmin
  • hgfsjryuu7
  • DreamBathrooms
  • Youngstown
  • slotface
  • vwfavf
  • PowerRangers
  • everett
  • kavyap
  • rosin
  • normalnudes
  • khanakhh
  • tacticalgear
  • InstantRegret
  • cubers
  • mdbf
  • ethstaker
  • osvaldo12
  • GTA5RPClips
  • cisconetworking
  • tester
  • Leos
  • modclub
  • provamag3
  • All magazines
    •