Das Jahr neigt sich dem Ende zu und damit sind wir auch schon beim vorerst letzten Vortrag des #DigitalHistoryOFK 2023 angekommen:
Wir freuen uns Miriam Schirmer (Uni Regensburg) begrüßen zu dürfen, die ihr Promotionsprojekt vorstellt, in dem sie verschiedene #NLP-Verfahren für die Genozidforschung einsetzt und mit qualitativen Ansätzen kombiniert.
does anyone here know how tokenizers are built in #NLP & #LLMs? in my head they seem like literal huffman coding calculated from the training dataset (on the character level instead of bit level)
In this lecture, Andrej explains the process behind training, inference, and finetuning LLMs, and talks about the future of LLMs and security. Super interesting.
Natural language processing has brought us enormous benefits, but with it challenges and concerns surrounding #privacy. These issues are tackled in "Privacy in #NLP: Are we There yet?", a talk by @habernal of Paderborn University. Join us online on 6 December at 18:30 CET: https://www.ofai.at/events/2023-12-06habernal
If you are looking for an NLP course, the University of Austin offers its master-level NLP course online for free. The course by Prof Greg Durrett focuses on the foundations of NLP, and covers:
✅ Intro and Linear Classification
✅ Multiclass and Neural Classification
✅ Word Embeddings
✅ Language Modeling and Self-Attention
✅ Transformers and Decoding
✅ Modern Large Language Models
FreeCodeCamp released a crash course for PaLM 2 API today by Ania Kubow. The PaLM 2 is Google's advanced language model, and the course focuses on creating applications with the API. That includes the following topics:
✅ Getting started with the PaLM 2 API
✅ Developing Chatbot interface
✅ Chatbot functionality
✅ Chatbot debugging and testing
Mid-month newsletter has the new esoteric/weird section of news for the paid subs, but the intro article is open -- this time on creepy Cornish folk horror film "Enys Men" (about a woman alone on a rocky island and a flower). News bits include collage tools, game agents, hallucination in NLPs... #ai#nlp#folklorehttps://arnicas.substack.com/p/titaa-485-stone-tape-on-a-stone-island
With the greatest respect, I submit that the problem is that it is considered something to be taught as a history of black people or white people. For many reasons, I believe the concept of #slavery is best taught as part of the history of people.
Man if anybody has a lead.... all the entry level jobs want #internships and all the internships want me to be still enrolled in school for the next school year. I returned to school too late for the summer internship hiring season last year and now I'm too close to graduation for anything coming up.
If you have a lead or a referral for an entry level position that requires no professional experience, or please hit me up!
⚡ Having fun with it — using #AI to grok policy about AI. With language models, I calculated embeddings against President #Biden's executive order on Artificial Intelligence. I used that to facilitate retrieval-augmented generation against the document based on my query, and then pass the relevant passages to a locally-running #LLM to summarize. Here I asked the model to tell me what the order said about copyright issues. Seems to work well!
Here it is in #Jupyter + Colab form — a #python Notebook demonstrating how I used text embeddings and a #LLM (LLaMA 2) to ask questions of #Biden's Executive Order on Artificial Intelligence. No API keys required, just some good ol' #OpenSource libraries and models. I learned a lot!
Very excited to share a substantially updated version of our preprint “Language models show human-like content effects on reasoning tasks!” TL;DR: LMs and humans show strikingly similar patterns in how the content of a logic problem affects their answers. Thread: 1/10 #LanguageModels#lms#AI#cogsci#machinelearning#nlp#nlproc#cognitivescience
After creating a simple example of natural language to SQL translator with the OpenAI API, this weekend, I extended my POC to open-source models with the Hugging Face API. The good news is that it is straightforward and simple to work with the API and get decent output. On the other hand, it is very slow to run locally. Any tips, articles, or examples on how to improve the model performance when running it locally? Thx!
Create a Large Language Model from Scratch with Python Tutorial 👇🏼
Another fun tutorial from freeCodeCamp, focusing on building LLM model from scratch with Python. It covers topics such as:
✅ Handling and processing text
✅ Core PyTorch functions for text
✅ Basic language models
✅ Advance methods
✅ Working with GPUs
I have just released a preprint with my postdoc advisor Maryellen MacDonald in which we argue that constraint satisfaction theories of sentence comprehension -- and in particular ambiguity resolution -- have been (partially) implemented in the probabilistic learning objective of (some) large language models. We are very open to comments and discussion and we hope you like the work! https://osf.io/kjd63
I am working on a new tutorial for creating a natural language for SQL code generator with LLM. So far, I have played with the Open AI API with Pandas and DuckDB, and it is impressive how simple it is to build and with accurate (so far) results. A notebook with example (WIP) is available in the below repo 👇🏼
i wish i knew more about comparing #embeddings. anyone have resources? one thing i’ve wondered is how to convert an embedding from a “point” to an “area” or “volume”. e.g. an embedding of a 5 paragraph essay will occupy a single point in embedding space, but if you broke it down (e.g. by paragraph), there would be several points and the whole would presumably be at the center. is there a way to trace the full space a text occupies in #embedding space? #LLMs#LLM#AI#NLP