@BenjaminHan@sigmoid.social
@BenjaminHan@sigmoid.social avatar

BenjaminHan

@BenjaminHan@sigmoid.social

Working on natural language, knowledge, reasoning, machine learning, and AI at a fruity company.

Husband, father, runner, German learner, piano player. A curious soul living in #PacificNorthwest (WA US).

๐Ÿ‘Ÿ 05/25/18-05/24/24 (dist # time pace/m date src):

5K 647 21:34 6'56" 4/20/24 Strava
10K 100 48:59 7'52" 5/16/24 Strava
15K 4 1:16:05 8โ€™10โ€ 5/19/24 Strava
HM 25 1:48:25 8โ€™16โ€ 5/19/24 Strava
M 7 3:44:58 8'35" 3/24/24 AppleW

#nlp #nlproc #knowledgeGraphs #ai #running #classicalMusic

This profile is from a federated server and may be incomplete. Browse more on the original instance.

lebelge, to classicalmusic
@lebelge@mathstodon.xyz avatar

@classicalmusic

Lars-Erik Larsson (1908 โ€“ 1986)
https://en.wikipedia.org/wiki/Lars-Erik_Larsson

"String Quartets"
[String Quartet No. 1 in D minor, Op. 31;
String Quartet No. 2 โ€œQuartetto Alla Serenataโ€, Op. 44;
String Quartet No. 3, Op. 65;
Intima Miniatyrer, Op. 20]
Helsingborgs Strรฅkkvartett
(Big Ben Phonogram 1987)
https://songwhip.com/helsingborgsstrakkvartett/larsson-string-quartets


BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@lebelge @classicalmusic https://classical.music.apple.com/us/recording/lars-erik-larsson-1908-pp59-571330059

BenjaminHan, to Fonts
@BenjaminHan@sigmoid.social avatar

Monaspace - An innovative superfamily of for https://monaspace.githubnext.com/

jbigham, to random
@jbigham@hci.social avatar

yesterday at UIST, we received the "Lasting Impact Award" for the original VizWiz paper -- it was super cool to receive the award, along with Robin Miller who 14 years ago was an undergraduate researcher on the project!

VizWiz was an iPhone app, released shortly after iOS added VoiceOver -- users could take a photo, ask a question, and get an answer back in a few tens of seconds from MTurk workers -- several novel things stitched together.

https://www.cs.cmu.edu/~jbigham/pubs/pdfs/2010/vizwiz.pdf

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@jbigham Jeez itโ€™s already been 14 years? Feeling my ageโ€ฆ

Congratulations!

BenjaminHan, to science
@BenjaminHan@sigmoid.social avatar

โ€œโ€ฆthe experiment demonstrated that plants can grow on the moon despite the intense radiation, low gravity, and prolonged intense light.โ€

Good to know Matt Damon can really the sh*t out of this.

China set up a tiny farm on the moon in 2019. How did it do?
https://phys.org/news/2023-10-china-tiny-farm-moon.html

BenjaminHan, to apple
@BenjaminHan@sigmoid.social avatar

Our team ( ) is looking for an working on data quality!

quality assurance is a highly important task that requires deep insights into how we integrate , , human-in-the-loop, system tooling and into a cohesive solution!

Please respond to the hiring manager via this post: https://www.linkedin.com/posts/benjaminhan_our-team-is-looking-for-talented-interns-activity-7122271517077368832-GCcl

GreatDismal, to random
@GreatDismal@mastodon.social avatar

Back here courtesy of the Mona app, or rather of the friend who wisely recommended it to me.

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@GreatDismal Yes โ€” @MonaApp is the only app I know that does syncing over iOS iPadOS and macOS. Love it.

BenjaminHan, to ArtificialIntelligence
@BenjaminHan@sigmoid.social avatar

1/

Our paper on automatic knowledge-graph-aligned dataset construction is out! The main points [1]:

  1. We showed cyclic evaluation โ€” the process of training GTG (graph-to-text-to-graph) or TGT using a graph-aligned dataset (screenshot 1) โ€” reflects faithfully the same trend a unidirectional evaluation does (screenshot 2-3). It is therefore a better way to assess data quality because it does not rely on knowing ground truth matches!

image/jpeg
image/jpeg

jbigham, to random
@jbigham@hci.social avatar

an advantage machines have over humans in reading is that humans generally have to perceive words, whereas machines just have the words sitting there already in their memory.

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@jbigham Word also are already sitting in humans memory?

jbigham, to random
@jbigham@hci.social avatar

a challenge i have as a runner is that all the races are early in the morning and i do not like to wake up early. probably folks should drop everything and work on this important problem.

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@jbigham Completely agreed. I can understand if the race is held in summer (to avoid sun). Another reason might be traffic control/road closure.

jbigham, to random
@jbigham@hci.social avatar

i'm going to say something that i think isn't controversial -- if a NYTimes reporter has to spend 3 days trying every possible way to "trick" your LLM into saying the thing they're desperately asking it to sayโ€ฆ that's not a safety concern.

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@jbigham 3 days by a single person.

BenjaminHan, to llm
@BenjaminHan@sigmoid.social avatar

1/

If a powerful #LLM is told that โ€œDaphne Barrington is the director of A Journey Through Timeโ€, it would surely be able to answer the question โ€œWho is the director of A Journey Through Time?โ€, right? Well, according to a recent paper [1], not quite (screenshot).

#NLProc #NLP #KnowledgeGraph #Reasoning #Papers

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@kellogh Thank you โ€” I enjoy writing them too!

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@indieterminacy Thatโ€™s a great excerpt! Read the book many many years ago and donโ€™t remember this. Thank you for bringing this up!

BenjaminHan, to Health
@BenjaminHan@sigmoid.social avatar

Time to train!

How to Get Strong https://www.nytimes.com/article/how-to-build-muscle-strength.html

jbigham, to random
@jbigham@hci.social avatar

ran the pittsburgh great race this weekend! largest 10k in pennsylvania! it was okay, i ran slower than last year (35:15), but it's always a great time!

jen (wife) won, again, and is quoted in the article belowโ€ฆ lol :)

https://www.post-gazette.com/local/city/2023/09/24/pittsburgh-great-race-runners/stories/202309240148

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@jbigham Pace 5โ€™40โ€?!!! Thatโ€™s freaking unbelievable!

jbigham, to random
@jbigham@hci.social avatar

20 years ago I started as a PhD student at UW CSE

10 years ago I started as a professor at CMU

5 years ago I started at Apple

It's been quite a ride! ๐Ÿš˜

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@jbigham Congratulations to the multi-anniversaries!

BenjaminHan, to Weather
@BenjaminHan@sigmoid.social avatar

Is it , or ?

Extreme switched off South American's sink | Corporate | University of Leeds https://www.leeds.ac.uk/news-environment/news/article/5391/extreme-el-ni-o-weather-switched-off-south-american-s-carbon-sink

BenjaminHan, to math
@BenjaminHan@sigmoid.social avatar

Is just symbol pushing?

When Computers Write Proofs, What's the Point of Mathematicians? https://youtu.be/3l1RMiGeTfU?si=sQMFAK7tzkS4ODZp

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

@dhinojosa Me too (via this video)! Very intriguing! https://leanprover.github.io

BenjaminHan, to generativeAI
@BenjaminHan@sigmoid.social avatar

1/ In this age of LLMs and generative AI, do we still need knowledge graphs (KGs) as a way to collect and organize domain and world knowledge, or should we just switch to language models and rely on their abilities to absorb knowledge from massive training datasets?

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

4/ But is that all? A recent paper revisits this question and offers a different take [2]. The authors believe just testing isolated fact retrieval is not sufficient to demonstrate the power of KGs.

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

5/ Instead, they focus on more intricate topological and semantic attributes of facts, and propose 9 benchmarks testing modern LLMsโ€™ capability in retrieving facts with the following attributes: symmetry, asymmetry, hierarchy, bidirectionality, compositionality, paths, entity-centricity, bias and ambiguity (screenshots).

image/png

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

6/ In each benchmark, instead of asking LLMs to retrieve masked words from a cloze statement, it also asks the LLMs to retrieve all of the implied facts and compute scores accordingly (screenshot).

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

7/ Their result shows that even #GPT4 achieves only 23.7% hit@1 on average, even when it scores up to 50% precision@1 using the earlier proposed LAMA benchmark (screenshot). Interestingly, smaller models like BERT can outperform GPT4 on bidirectional, compositional, and ambiguity benchmarks, indicating bigger is not necessarily better.

#KnowledgeGraphs #GenerativeAI #LLMs #NLP #NLProc #Paper

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

8/ There are surely other benefits of using KGs to collect and organize knowledge. They do not require costly retraining to update, therefore can be updated more frequently to remove obsolete or incorrect facts. They allow more trackable reasoning and can offer better explanations. They make fact editing more straightforward and accountable (think of GDPR) compared to model editing [3].

#KnowledgeGraphs #GenerativeAI #LLMs #NLP #NLProc #Paper

BenjaminHan,
@BenjaminHan@sigmoid.social avatar

9/ But LLMs can certainly help in bringing in domain-specific or commonsense knowledge in a data-driven way. In conclusion: why not both [4]? :-)

  • All
  • Subscribed
  • Moderated
  • Favorites
  • โ€ข
  • anitta
  • kavyap
  • DreamBathrooms
  • InstantRegret
  • magazineikmin
  • cubers
  • GTA5RPClips
  • thenastyranch
  • Youngstown
  • rosin
  • slotface
  • tacticalgear
  • ethstaker
  • modclub
  • JUstTest
  • Durango
  • everett
  • Leos
  • provamag3
  • mdbf
  • ngwrru68w68
  • cisconetworking
  • tester
  • osvaldo12
  • megavids
  • khanakhh
  • normalnudes
  • lostlight
  • All magazines
    •