@kellogh@hachyderm.io
@kellogh@hachyderm.io avatar

kellogh

@kellogh@hachyderm.io

I'm a software engineer and sometimes manager. Currently #Raleigh but also #Seattle. Building ML platform for a healthcare startup. Previously, built an IoT platform for one of "those" companies.

Open source: dura, fossil, Jump-Location, Moq.AutoMock, others

Do I have other interests? No, but I do have kids and they have interests. I think that counts for something. I can braid hair and hunt unicorns!

I put the #rust in frustrate

He/Him

#metal #science #python

This profile is from a federated server and may be incomplete. Browse more on the original instance.

kellogh, to random
@kellogh@hachyderm.io avatar

i wish “type checking for infrastructure” was a thing

my code declares that there should be a S3 bucket that’s different from that other S3 bucket, etc. —> spin up the type checker, it reads APIs and verifies, “yep, this code should run fine”

kellogh, to ai
@kellogh@hachyderm.io avatar

a crazy development over the last couple years is the revival of computer architecture

for most of my career, CPUs and GPUs just worked, and while there were some wild ideas, the problems hadn’t yet emerged

with , we’ve moved so far from the low level, that the existing chips aren’t working all that well anymore. I’d argue that computer architecture is now a skill needed for AI engineers these days

kellogh, to random
@kellogh@hachyderm.io avatar

sperm whales have a complex language! they use complex patterns of clicks to communicate

it turns out that the hardest part of the search for intelligent life is recognizing what you’re looking at

https://www.npr.org/2024/05/07/1249546255/sperm-whale-communication-ai-language

kellogh,
@kellogh@hachyderm.io avatar

@jimfl i bet it was an FFT

meganL, to random
@meganL@mas.to avatar

Community participation:

What are your favorite non-monopoly places to shop online? Bonus if they're actually also really cool businesses?

Help folks discover non-Amazon, non-Walmart stores!

Ones I use include:

American Science & Surplus: https://sciplus.com/

Stonemountain & Daughter https://stonemountainfabric.com/

Davis Food Co-op (not mail order, but you can still do "curbside" pick-up) https://davisfood.coop/

kellogh,
@kellogh@hachyderm.io avatar

@WuMargaret @meganL i tried other brands of barefoot shoes years ago, and my general takeaway was that they’re a great way to mess up your feet and calves. i’m sure you can do it safely, but it’s not intuitive and there’s a whole ton of ways to seriously hurt yourself

kellogh, to random
@kellogh@hachyderm.io avatar

the kids are arguing

8yo: yes! honeybees sting!

3yo: no they don’t! when honeybees come over they say hi

kellogh,
@kellogh@hachyderm.io avatar

@amyedge she must have gotten it from somewhere, not sure where though

kellogh,
@kellogh@hachyderm.io avatar

@amyedge lol, apparently it’s something my wife says

kellogh, to ai
@kellogh@hachyderm.io avatar

is there a framework like STRIDE for evaluating ethics? seems like something @DAIR would have

kellogh, to llm
@kellogh@hachyderm.io avatar

there’s a big need for something stronger than , but more flexible and cheaper than a giant all-knowing .

A great part about RAG is that it’s just a database. You just issue INSERT/UPDATE and yeah, that’s how you maintain knowledge. No million dollar training process

kellogh,
@kellogh@hachyderm.io avatar

i just had a case where perplexity (gpt-3.5 + whole internet RAG) failed miserably but gpt-4 did great.

i’m imagining it would be somehow where the embedding + lookup is somehow tightly integrated into the model itself, not just text.

kellogh,
@kellogh@hachyderm.io avatar

it’s one of those things that OpenAI probably won’t do because it doesn’t get us closer to AGI (RAG doesn’t help symbolic reasoning), even though it’s extremely useful and would make for overwhelmingly practical use cases

kellogh,
@kellogh@hachyderm.io avatar

@mako the example i gave — GPT4 got less confused than perplexity — RAG isn’t everything, and the main places it falls apart is in its text interface. it got confused because the context was partitioned across messages, so the embedding used for RAG didn’t have enough information.

the reason we do smaller chunks for embeddings is because, as the chunk size increases, it continues to add information and it gets harder for the embedding to represent the main themes

kellogh,
@kellogh@hachyderm.io avatar

@mako i guess we just need an innovation in embedding models — prompt-able embedding models. give it the full context, but only generate an embedding for the last n-words. I’m not entirely sure that hasn’t been done already…

kellogh,
@kellogh@hachyderm.io avatar

@chromosundrift i suppose it doesn’t, but it’s very hard to imagine it without it. seems like at some point you need precision

nicola, to ADHD
@nicola@fosstodon.org avatar

I read ADHD—A Lifelong Struggle today. It’s the best thing I’ve read on ADHD so far. Its advice is grounded in experience and applies to everyone: those who think ADHD is a crock or, worse, an excuse, those who suffer from it, and those who have family members or friends who suffer from it.

https://gekk.info/articles/adhd.html

kellogh,
@kellogh@hachyderm.io avatar

@nicola this is best book i’ve read on the subject. it’s got a good explanation of what it is, how it happens, who it happens to, what it looks like, and what to do about it. Everyone i’ve recommended it to has read it more than once

https://a.co/d/2zZSggx

futurebird, to random
@futurebird@sauropods.win avatar

Politicians and political commentators whine about young people not buying homes and starting families (not getting married) and then do NOTHING to help the young people (who do exist) who would love to do one or more of these things but can't because it's too damn expensive.

Build housing and coops where young people want to live. Protect their jobs with unions. Make health care not an issue.

Do not make some marriages impossible because you don't like the genders.

Not rocket science.

kellogh,
@kellogh@hachyderm.io avatar

@futurebird i had this same conversation with my dad after he went off about some population crisis. “dad, do you know how expensive it is to have children these days? the whole world is working against you and your family”

kellogh, to random
@kellogh@hachyderm.io avatar

there’s no 3d printing in space

kellogh, to random
@kellogh@hachyderm.io avatar

actually yeah, now that all the white moms and dads are smoking pot, might as well make it more legal

kellogh,
@kellogh@hachyderm.io avatar

@wagesj45 next up: noodling

skinnylatte, to random
@skinnylatte@hachyderm.io avatar

literally everything i do today for work and money comes directly from 'me dabbling at random stuff at a computer 20 years ago that no one else around me understood' so i truly feel sorry for kids of helicopter parents who are like 'no screens'

kellogh,
@kellogh@hachyderm.io avatar

@skinnylatte for my kids, they should be creating stuff. doesn’t matter what or how, or even if it’s good. they knit, draw, paint, use dall-e, 3d print, mix beats on my phone, … and they do consume also, podcasts, youtube, books, … but i figure as long as they’re taken with creating, they’ll be okay

andrew, to random
@andrew@esq.social avatar

You’d think Microsoft would have negotiated for some kind of exclusivity for search with Bing.

It just became semi relevant and it’s going to be relegated to third banana again. https://mastodon.social/@ricmac/112416562753787073

kellogh,
@kellogh@hachyderm.io avatar

@andrew not worth it. they win either way. you can look at it as waste, or genetic algorithms

futurebird, to random
@futurebird@sauropods.win avatar

I love the fedi only more for being the only place I know full of exasperated FORTRAN defenders. How can we insult the original word? Yea, when it was first written we were not yet even born! 🥰

kellogh,
@kellogh@hachyderm.io avatar

@futurebird ha, BLAS and LAPACK are written in fortran and almost all linear algebra libraries use them for array operations, e.g. numpy. so there’s a chance several of the programs you use are running fortran, especially anything ML/AI related

smach, to LLMs
@smach@masto.machlis.com avatar

“The general problem of mixing data with commands is at the root of many of our computer security vulnerabilities.” Great explainer by security researcher Bruce Schneier on why large language models may not be a great choice for tasks like processing your emails.
https://cacm.acm.org/opinion/llms-data-control-path-insecurity/

kellogh,
@kellogh@hachyderm.io avatar

@smach yay! i had the same thought a while ago. if you can separate the data & control, you can make it safe

https://timkellogg.me/blog/2024/01/11/application-phishing

kellogh,
@kellogh@hachyderm.io avatar

@smach after writing that, i found out about control vectors, which is sort of close, but the control still goes through the same channel as data https://vgel.me/posts/representation-engineering/#Control_Vectors_v.s._Prompt_Engineering

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • JUstTest
  • GTA5RPClips
  • DreamBathrooms
  • InstantRegret
  • magazineikmin
  • khanakhh
  • Youngstown
  • ngwrru68w68
  • slotface
  • everett
  • rosin
  • thenastyranch
  • kavyap
  • tacticalgear
  • megavids
  • cisconetworking
  • normalnudes
  • osvaldo12
  • ethstaker
  • mdbf
  • modclub
  • Durango
  • tester
  • provamag3
  • cubers
  • Leos
  • anitta
  • lostlight
  • All magazines
    •