@xcjs@programming.dev avatar

xcjs

@xcjs@programming.dev

This profile is from a federated server and may be incomplete. Browse more on the original instance.

Advice - Getting started with LLMs

I’m new to the field of large language models (LLMs) and I’m really interested in learning how to train and use my own models for qualitative analysis. However, I’m not sure where to start or what resources would be most helpful for a complete beginner. Could anyone provide some guidance and advice on the best way to get...

xcjs,
@xcjs@programming.dev avatar

No offense intended, but are you sure it’s using your GPU? Twenty minutes is about how long my CPU-locked instance takes to run some 70B parameter models.

On my RTX 3060, I generally get responses in seconds.

xcjs,
@xcjs@programming.dev avatar

Unfortunately, I don’t expect it to remain free forever.

xcjs,
@xcjs@programming.dev avatar

Ok, so using my “older” 2070 Super, I was able to get a response from a 70B parameter model in 9-12 minutes. (Llama 3 in this case.)

I’m fairly certain that you’re using your CPU or having another issue. Would you like to try and debug your configuration together?

xcjs,
@xcjs@programming.dev avatar

It should be split between VRAM and regular RAM, at least if it’s a GGUF model. Maybe it’s not, and that’s what’s wrong?

xcjs,
@xcjs@programming.dev avatar

Good luck! I’m definitely willing to spend a few minutes offering advice/double checking some configuration settings if things go awry again. Let me know how things go. :-)

xcjs, (edited )
@xcjs@programming.dev avatar

I think there was a special process to get Nvidia working in WSL. Let me check… (I’m running natively on Linux, so my experience doing it with WSL is limited.)

docs.nvidia.com/cuda/wsl-user-guide/index.html - I’m sure you’ve followed this already, but according to this, it looks like you don’t want to install the Nvidia drivers, and only want to install the cuda-toolkit metapackage. I’d follow the instructions from that link closely.

You may also run into performance issues within WSL due to the virtual machine overhead.

xcjs,
@xcjs@programming.dev avatar

We all mess up! I hope that helps - let me know if you see improvements!

maegul, to fediverse
@maegul@hachyderm.io avatar

Nice demonstration of why mastodon's dominance is problematic

See the conversions here:
https://github.com/LemmyNet/lemmy/pull/4628
and
https://socialhub.activitypub.rocks/t/federating-the-content-of-posts-note-articles-and-character-limits/4087

AFAICT, mastodon's decisions, which are arguably problematic (on which see: https://lemmy.ml/post/14973403) are literally trickling down to other platforms and infecting how they federate with each other as they dance around mastodon's quirks in different ways.

It seems like masto is ruining "the standard" with its gravity.


@fediverse

xcjs, (edited )
@xcjs@programming.dev avatar

It’s a W3C managed standard, but there are tons of behavior not spelled out in the specification that platforms can choose to impose.

The standard doesn’t impose a 500 character limit, but there’s nothing that says there can’t be a limit.

xcjs,
@xcjs@programming.dev avatar

Or maybe just let me focus on who I choose to follow? I’m not there for content discovery, though I know that’s why most people are.

xcjs,
@xcjs@programming.dev avatar

I was reflecting on this myself the other day. For all my criticisms of Zuckerberg/Meta (which are very valid), they really didn’t have to release anything concerning LLaMA. They’re practically the only reason we have viable open source weights/models and an engine.

xcjs,
@xcjs@programming.dev avatar

That’s the funny thing about UI/UX - sometimes changing non-functional colors can hurt things.

xcjs,
@xcjs@programming.dev avatar

Exactly why I surmised they may not want to keep it there: programming.dev/comment/8019112

xcjs, (edited )
@xcjs@programming.dev avatar

At some point, you lose productivity and reduced work weeks have shown increases in productivity can happen.

xcjs,
@xcjs@programming.dev avatar

My go-to solution for this is the Android FolderSync app with an SFTP connection.

How to distribute an open source font in a git repository?

Hello, for a few days I have been looking at how to make a git repository to store a free font so that everyone can use it, modify it and redistribute it, the thing is that many have different ways of distributing the source code, and it is not very clear how. Do it in a way that is similar to an open source program. The...

xcjs, (edited )
@xcjs@programming.dev avatar

I’m not familiar with creating fonts specifically, but you’ll want to commit any resources necessary to recreate the font file, including any build scripts to help ease the process and instructions specifying compatible versions of tooling (FontForge in this case). Don’t include FontForge in the repository, of course.

The compiled font files should be under releases in GitHub for the repository.

Git isn’t generally meant for binary resources but as long as they’re not too large, they’ll be fine. You just may not have meaningful ways to compare changes easily.

xcjs,
@xcjs@programming.dev avatar

Systemd was created to allow parallel initialization, which other init systems lacked. If you want proof that one processor core is slower than one + n, you don’t need to compare init systems to do that.

xcjs,
@xcjs@programming.dev avatar

I mean, sysvinit was just a bunch of root-executed bash scripts. I’m not sure if systemd is really much worse.

xcjs,
@xcjs@programming.dev avatar

The Nuzu repository is already wiped.

xcjs,
@xcjs@programming.dev avatar

Correction: migrated to GitLab, but I don’t expect they’ll want to keep it there.

xcjs,
@xcjs@programming.dev avatar

With UI decisions like the shortcut bar, they really don’t. I switched to another SMS app because I couldn’t stand it.

xcjs,
@xcjs@programming.dev avatar

I already left the Messages app after the Shortcut Bar annoyance. This would have been another death knell to me that the app is past its prime.

xcjs,
@xcjs@programming.dev avatar

I’ve been disappointed in general with the XPS line in recent years. Dell has made some keyboard changes that I am not a fan of:

  • A touch bar instead of function keys? Why? Did they not learn from Apple’s mistake?
  • Changing Fn + Left to Page Down instead Home? And Fn + Right to Page Up instead of End? Once again, why?

I’ve been purchasing the XPS line of laptops since 2013, but I stopped as soon as those changes landed and the Developer Edition of their laptops shipped with inferior hardware compared to the Windows ones.

xcjs,
@xcjs@programming.dev avatar

On Android, it moved SMS messages from the shared SMS store upon receipt and to Signal’s own database, which was slightly more secure.

xcjs,
@xcjs@programming.dev avatar

I tried to take hourly snapshots of an already-large Minecraft world using Git, but after a few years of snapshots, the repository became corrupted.

One of the issues was that regardless of any player-based changes that occurred, the spawn regions were always different as they were always loaded in memory.

xcjs,
@xcjs@programming.dev avatar

I…do not miss XP, but I understand the nostalgia attached to it.

I learned a lot of technical skills on XP, but that’s what made me appreciate the architectural decisions behind UNIX-likes all the more.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • megavids
  • khanakhh
  • ngwrru68w68
  • mdbf
  • tacticalgear
  • Youngstown
  • rosin
  • slotface
  • Durango
  • kavyap
  • everett
  • thenastyranch
  • InstantRegret
  • DreamBathrooms
  • JUstTest
  • magazineikmin
  • anitta
  • GTA5RPClips
  • modclub
  • cisconetworking
  • cubers
  • osvaldo12
  • ethstaker
  • provamag3
  • normalnudes
  • Leos
  • tester
  • lostlight
  • All magazines