“A new computer model to predict the weather built by Google and powered by artificial intelligence consistently outperforms and is many times faster than government models that have existed for decades and involved hundreds of millions of dollars in investment, according to a study published Tuesday.” - Dan Stillman, Washington Post
Any good sources on what the outputs of the attention blocks in a transformer represent? I expected that for "The bank of the plane took it around the savings bank on the bank of the river", the vectors corresponding to "bank" would diverge -- "rotation things/money things/rivery things" -- but AFAICT that doesn't clearly happen. Here are the dot prods of the normalized vectors (aka "cosine similarity") against themselves after embedding layer and attention block 5: #ML#Transformer
There's a submarine movie that's popular with Dads where the bad guy climactically launches a sound-seeking torpedo that circles back and blows up the sub from which it was fired. In #ML that's called "the alignment problem": there's a gap between what you want the system to do ("blow up the other guy") and how it pursues that goal ("seek the loudest sound and go boom"). The #1 area for #AI concern should be bias and immediate harms. But, IMO, the "alignment problem" is worth thinking about.
Woot, extended 2 more job offers today, one Jr. and one Sr. I suspect there are at least 3 more people in the pipeline I will be hiring. All from this one post.
That leaves at least 10 out of 15 positions open for anyone still looking for a job!
We also just donated another $5K to our open-source fund. This will go to paying open-source contributors on our projects bounties for completing certain tasks. Very excited by that as well. We are even using it on interviews for code projects so in a sense we are paying people to take interviews with us even if they dont get the job through those bounties. I am getting some interesting reactions with that one.
Picked up an old Tesla P100 accelerator card off eBay quite cheap, managed to get it working in my home server.
Had to print up some ducts to add active cooling (they're usually cooled by the airflow within the server they're installed in), added a little fan smoother/adjuster and a few brackets to hold the (massive) card in place, then whipped up a little helper utility that reads from nvidia-smi GPU temp data to set the 12v fan speed as I was having issues getting fancontrol to read nvidia's i2c values in a useful way.
It's still a bit too noisy when under load, the fan I've got doesn't rotate until you apply at least 40% of the PWM cycle as it's a big-boi, so might look at getting a couple of smaller fans perhaps, just want to make sure the thing doesn't cook.
years ago, the “language of machine learning” was split between #R and #python but it’s been steadily shifting toward python. At this point, after all the #LLM developments, i think it’s clearly python. i don’t see much R in the LLM world at all. And increasingly, i’m seeing #rust being the “systems language of #ML” #rustlang#LLMs
Anybody out there looking for an ML or software engineer with >30 years total experience and ~20 years in the industry?
I have extensive experience with #Python and #ML frameworks, particularly #TensorFlow, and I've worked on #NLP and #ImageProcessing both in the workplace and in personal open source projects. My resume is available here:
I knew it. Getty's complaint about #OpenAI wasn't about protecting artists. They cared that they weren't making money.
#Getty partners with Nvidia to launch Generative AI by Getty Images, which lets users create legally protected images using Getty's library of licensed photos.
Hey #classicalmusic and #ai fans:
If one trained a suitable #ml model on all Bach fugues, it should be able to generate passable fugues itself, I would think. Is that a reasonable hypothesis?
#ChatGPT consumes up to an estimated 500ml of water for every five to 50 prompts; #Microsoft reported its water use spiked 34% YoY in 2022, #Google 20%.
Just found something interesting on a Bing SERP: they were not only showing individual images as usual, but were combining several images in one frame.
In the close-up image you can see current products as well as their predecessors. Meaning: these older images are not at all displayed on the website.
My guess is that the Bing algorithm groups images together, which have something in common - as long as they are to be found on that webserver.
These days I find #openaccess academic research that uses #ML is mostly all show, no tell. Few spill the beans on the full workflow (pre-processing, model and model weights). You get fancy maps, poor uncertainty characteristics and no way to reproduce results (even worse when drivers are hidden behind "platforms").
If #web2 has taught us anything it is that platforms are another way of gatekeeping, which is perceived to be sharing but mostly benefits the platform itself. Thusfar, #openscience.