it occurs to me that transformers are a lot like microprocessors, in that there’s a high level management decision up front to decide how big you want to make it (transistor count vs parameter count) that dictates the capability of the end product. where those transistors/parameters are allocated is a lower level series of decisions that impacts efficiency, but overall capability is decided mostly by the total budgeted size
my 3yo exhibits this thing i call stochastic walking, where her exact walking path is wholly unpredictable, except that (1) it’s always in my way and (2) if i step into her probabilistic path cloud, her actual path will definitely converge with mine with absolute certainty
i’ve seen a lot in my career, but until now, i’ve never seen religious fervor like there is against AI. there’s a lot of religion around AI in most directions, but crypto had that too
what’s crazy about this phase is that if people took time to understand what accelationists see, they wouldn’t come to the same conclusions, and we’d have far more interesting and productive conversations about it
but instead we’re caught in tired arguments about “it’s just linear algebra”
@kellogh an apt analogy here might be the discovery of a very, very effective compression algorithm, which doesn’t actually expand what was put in, but might fool you that it does. Since it is so effective at compressing, every drive maker on the planet is adding it to their drives, so that they can claim the compression rates of their competitors
@kellogh The real issues, even the most obvious ones of bias-in, bias-out and real-world harms, have been totally drowned out by competition for meme-worthy phrasings of specious nonsense. It’s dismaying.
one thing i love about #LLMs is asking it “how tf do i do X” and it responds with 5 ideas, four of which are terrible but one is far better than anything i’d thought of. or their all terrible but one makes me realize i’ve been thinking about the problem wrong
also, #LLMs cause me to think a lot about the multifaceted nature of intelligence. we used to over-weight language skill, but now that LLMs have that in spades, it’s apparent that there’s more going on
for example, spontaneity. if it had an ounce of sponteity, it could suggest approaching the problem differently
talking to my conservative family, they’re completely baffled that there’s an election interference angle to the #trump#conviction. my dad consumes a metric ton of news media, had no idea
they say #northcarolina is a purple state, but it’s more like a bimodal “flaming red” AND “ice blue” state depending on where you are
imo the blue parts feel far more liberal than any part of #seattle. i debate if that’s because the south is just more superficial about everything, or if people in seattle simply don’t talk to literally anyone so we don’t actually know if it’s a liberal or conservative area
@kellogh It's pretty well urban vs rural/exurban with suburbs split everywhere. Even Texas is largely bright blue throughout the main urban areas, with the exception of Ft Worth that's purple
when learning about computers, i can’t imagine how people do it without “hacking” — tearing things down to the most basic components, building them back up in quirky ways
i had a CS prof that demanded everyone use Notepad.exe as their exclusive code editor and i kinda dig that mindset. you don’t need to efficiently write code when you’re learning
@kellogh When I was first learning I forced myself to use notepad (or equivalent) because I wanted the muscle memory, not to be prompted. Granted, this was like Win32 API programming with Petzold and a .hlp file, but the mentality stuck with me all of these years.
@kellogh I have been wondering, too. Esp when I’ve realized that my kids’ first exposure to programming was Python at school… and they could reverse a string without having any idea what a CPU was. Still can’t figure out whether that’s insane or totally fine.
this is great! the last few years we’ve finally internalized how important bacteria is to our health, so broad spectrum antibiotics that wipe out all bacteria are sometimes more harmful than we wish.
this new antibiotic targets only Gram-negative pathogens, which commonly infect hospital patients and have frequently become resistant to antibiotics. This could save a lot of lives
it’s funny that “legalistic” means a very strict interpretation, but in 2024 general society has become so saturated with computers and technology that actual legal interpretation seems vague, muddy and subjective
those days were wild. people would say, “what if we…” and then outline the plot to a black mirror episode
one of the side effects of working in software is you most often observe software in a failing state, so replacing fundamental societal structures with code seems like a really bad idea