Andrej Karpathy released today a tutorial for reproducing GPT-2 from scratch. OpenAI released GPT -2 in 2019, and it is a 124M parameters model. This four-hour tutorial covers setting up the GTP-2 network and then training and optimizing its parameters.
It looks like a really cool tutorial; I hope to get the bandwidth to watch it in the coming weeks!
(1/2) I am excited to present at the useR!2024 conference on July 2nd!
I am going to run a virtual workshop about deployment and monitoring data and ML pipelines using free and open-source tools. This includes setting pipelines using GitHub Actions, Docker 🐳, R, and Quarto 🚀.
If you are looking for a #PhD and are interested in working on #OpticalComputing for #MachineLearning (and to spend some time in the UK and some time in Australia), contact me!
Got the funding but the official advert is not out yet. Will update when it is (but the deadlines are going to be short).
I am excited to present at the Dev AI conference in Paris on June 19!
I am going to run a workshop about the deployment and monitoring of ML pipelines with free and open-source tools. This includes using tools such as GitHub Actions and Pages, Docker, Python, Quarto, etc.
In this week's #PeopleOfACM, we interview Nesime Tatbul, a Senior Research Scientist at Intel Corporation ’s Parallel Computing Lab (PCL) and Massachusetts Institute of Technology ’s Computer Science and Artificial Intelligence Lab (CSAIL).
In her interview, Tatbul discusses her work in large-scale data management systems, including how we can improve data systems through #machinelearning and observability.
Before I head off on a trip to various parts of not-Barcelona, I thought I’d share a somewhat provocative paper by David Hogg and Soledad Villar. In my capacity as journal editor over the past few years I’ve noticed that there has been a phenomenal increase in astrophysics papers discussing applications of various forms of Machine Leaning (ML). This paper looks into issues around the use of ML not just in astrophysics but elsewhere in the natural sciences.
The abstract reads:
Machine learning (ML) methods are having a huge impact across all of the sciences. However, ML has a strong ontology – in which only the data exist – and a strong epistemology – in which a model is considered good if it performs well on held-out training data. These philosophies are in strong conflict with both standard practices and key philosophies in the natural sciences. Here, we identify some locations for ML in the natural sciences at which the ontology and epistemology are valuable. For example, when an expressive machine learning model is used in a causal inference to represent the effects of confounders, such as foregrounds, backgrounds, or instrument calibration parameters, the model capacity and loose philosophy of ML can make the results more trustworthy. We also show that there are contexts in which the introduction of ML introduces strong, unwanted statistical biases. For one, when ML models are used to emulate physical (or first-principles) simulations, they introduce strong confirmation biases. For another, when expressive regressions are used to label datasets, those labels cannot be used in downstream joint or ensemble analyses without taking on uncontrolled biases. The question in the title is being asked of all of the natural sciences; that is, we are calling on the scientific communities to take a step back and consider the role and value of ML in their fields; the (partial) answers we give here come from the particular perspective of physics
arXiv:2405.18095
P.S. The answer to the question posed in the title is probably “yes”.
I was looking for examples of interactive data visualization for a gradient descent algorithm, and I found this app by Lili Jiang. This desktop app is based on C++ and enables simulation and visualization of different gradient descent algorithms, such as momentum, AdaGrad, RMSProp, and Adam. The app enables to compare different methods simultaneously.
Open your calendar, NumPy 2.0 is going to be out on June 16th 🚀
This is the first major release since 2006. The release includes breaking changes in the library API, and therefore, if you are planing to adopt it, some code refactoring may required.
The release includes new features, performance improvement 🏎️, improvements on the C API, and more.
(1/2) Shiny Apps for demystifying statistical models and methods 🚀
This is a cool website that explains different statistical concepts with the use of interactive Shiny Apps. Ben Prytherch made this website from the Department of Statistics at Colorado State University.
The Machine Learning for Beginners by Microsoft Developer is an introductory course for classical machine learning. This crash course mainly focuses on regression analysis with Python 🐍, and it covers topics such as:
✅ General setup
✅ Cleaning data
✅ Data visualization
✅ Regression models
✅ Polynomial regression
✅ Logistic regression
Hello Mastodon, I know that a lot of you discuss the high environmental cost (such as energy use and water use) of AI and I hope that some of you could reply with authoritative publications/links regarding this problem! I want to try to convince an environmental science colleague #climatechange#AI#chatgpt#energy#technology#machinelearning#llm
Here is a short e-book with a sequence of tutorials on the scientific Python ecosystem for beginners. This includes topics such as:
✅ Working with numerical data using NumPy
✅ Data visualization with Matplotlib
✅ Scientific computing with SciPy
✅ Statistics with Python
✅ Machine learning with scikit-learn