Jack Clark’s Import AI newsletter is always fun; this one especially because it explores the shared interest Meta and the CCP have in preventing fine-tuning of their models.
If you’d like to make it possible for people to follow you from Bluesky — and you’re in a domain that permits it — all you have to do is follow @bsky.brid.gy@bsky.brid.gy.
I think these connections have the potential to strengthen everyone. We need critical mass!
How can we evaluate the factual accuracy of long answers from LLMs? Researchers from DeepMind / Stanford demonstrate a strategy that uses LLMs + search to assess factuality: it's more accurate than human evaluation and 20x cheaper. h/t Marc Lanctot on Threads arxiv.org/abs/2403.18802
IRL, if all particle accelerators started generating weird results and scientists saw a flashing countdown clock superimposed on their vision, 100% of them would assume it was a grant deadline. https://www.threads.net/@tedunderwoodillinois/post/C41478vtflW
It’s fine for the bridge to Bluesky to be opt-in. But however it’s constructed, I am looking forward to sharing some of the fine content here in a space where algorithmic discovery is easy and uncontroversial.
@TedUnderwood the problem could be faceted. This platform offers features that are up to instance administrators to activate, and I understand algorithmic feed is one of these.
The instance I landed into provides it, and I hated it in the beginning, but in the end it does a better job at following contents I care about (better than filters I designed myself) plus others moderately outside of my bubble.
"Open social media" is a place where mileage can vary greatly. Which can be seen as positive
@ideaferace Yes, faceted and local has advantages and is okay with me.
But personally, I prefer a space where algorithmic discovery (and search) are just default norms. I’m glad I’ll be able to base myself there and connect to (parts of) the multiverse.
@muzicofiel@TedUnderwood Well, when this goes through I can shut down my Bsky account (once I figure out how to transfer my follow list to Mastodon), but I think I'll keep @dilbert going over there for shits and giggles ...
The School of Information Sciences at IL Urbana-Champaign is seeking postdoctoral researchers for AY 2024-25, renewable for a second year. We're interested in fields from children's literature through computational social science; your doctorate doesn't have to be in IS.
Applications are due March 15. It's a good idea to reach out to potential faculty mentors in advance.
Computer-assisted studies of fiction tend to count words and trace themes. It's harder to measure the things that really keep readers turning pages: mystery, suspense, and surprise.
@Virginicus yes, although the range between outliers in Hound may also be partly just having more data points — I had to do four passes on it versus one in the other illustration
Very cool. I think there's lot of promise in having LLMs "read" texts like this and using it to analyze the text itself.
Can I ask why you went with the distance in the embedding space? The model itself already produces probabilities, so you could check the probability of the truth under the model distribution (if this is available in the GPT API), or just the entropy of the predicted part (alhtough NNs are notoriously bad an producing well-calibrated uncertainty).
@robux4 he indicated being able to take followers out of Threads to other Fediverse platforms, which is I think the part that would be hardest for Meta to swallow
Doing a bibliometric study of political science, history, economics and looking for journals that have been relatively central to English-language scholarship over a long print run. Need to have a few that were central early on (1930-1950). Recommendations? #history#polisci#econ
I've read enough of The Ends of Knowledge (ed @SethRudy & Rachael Scarborough King) to recommend it to people in #DH, literary studies, and #18thC. Not just "interdisciplinary," it aims to produce a Diderot/d'Alembert scale overview! I found Mark A-H's chapter on DH esp. illuminating. https://www.bloomsbury.com/us/ends-of-knowledge-9781350242302/
TLDR: 1) if we're teaching students how to do what has already been done, do it offline.
2) We also need advanced projects so students learn how to do what hasn't yet been done. It's okay if they use AI for those; it's not cheating.
3) If sorting things into piles 1 and 2 is painful, it may be that we haven't been honest with ourselves about how much is 1.
@TedUnderwood@ebrandom To be honest, the problem with students getting help from a computer on a take-home assignment isn’t so different from the problem of students getting help from their parents. The only way to be sure it’s their own work is to have them write it in pen in front of you.
@TedUnderwood I'm between being angry that he's doing so much damage and glee that he's making such a cock up of it. More than that I want him to leave the building via the fire exit with a night bag and never be seen again.
@christof@Transportist@Dreamwieber I don’t really know, except that my impression is, academics are more willing to move than industry people. And industry is really important in that field right now.