#AI#Algorithms#ML#ResponsibleAI: "Machine learning and algorithmic systems are useful tools whose potential we are only just beginning to grapple with—but we have to understand what these technologies are and what they are not. They are neither “artificial” or “intelligent”—they do not represent an alternate and spontaneously-occurring way of knowing independent of the human mind. People build these systems and train them to get a desired outcome. Even when outcomes from AI are unexpected, usually one can find their origins somewhere in the data systems they were trained on. Understanding this will go a long way toward responsibly shaping how and when AI is deployed, especially in a defense contract, and will hopefully alleviate some of our collective sci-fi panic.
This doesn’t mean that people won’t weaponize AI—and already are in the form of political disinformation or realistic impersonation. But the solution to that is not to outlaw AI entirely, nor is it handing over the keys to a nuclear arsenal to computers. We need a common sense system that respects innovation, regulates uses rather than the technology itself, and does not let panic, AI boosters, or military tacticians dictate how and when important systems are put under autonomous control." https://www.eff.org/deeplinks/2024/03/how-avoid-ai-apocalypse-one-easy-step
#LLMs have really created a paradigm shift in machine learning. It used to be so that you would train an #ML model to perform a task by collecting a dataset reflecting the task, with task output labels, and then using supervised learning to learn this task by doing.
Now a new paradigm has emerged: Train by reading about the task. We have such generalist models that we can let them learn about the domain by reading all the books and other content about it, and then utilize that learned knowledge to perform the task. Note that task labels are missing. You might need those to measure the performance but you don't need those for training.
Of course if you have both example performances as task labels and lots of general material about the topic, you can actually use both to get even better performance.
Here is a good example of training the model not by example performances, but by general written knowledge about the topic. #GPT4 surpasses the quality levels of previous state-of-the-art despite not having been trained for this task.
This is the power of generalist models; they unlock new ways to train them, which for example allow us to surpass human-level by side-stepping imitative objectives. This isn't the only way to train skills these models enable, there are countless other ways, but this is an uncharted territory.
The classic triad of supervised learning, unsupervised learning and reinforcement learning are going to have an explosion of new training methodologies to become their peers because of this.
Delighted to be able to publicise a paper that was presented at the @ALTAnlp 2023 Workshop at the end of last year, co-authored with my #PhD supervisor, Associate Professor @eltwilliams, and written as part of my research at #ANU School of Cybernetics.
Titled "Right the docs: Characterising voice dataset documentation practices used in machine learning", it combines both exploratory interviews and documentation analysis to characterise how large voice datasets - e.g. #LibriSpeech, @mozilla's #CommonVoice, and several others, document their #metadata.
Unsurprisingly, it finds that the #dataset#documentation practices seen currently do not meet the needs of the #ML practitioners who use these datasets.
We show, once again, in the words of Nithya Sambasivan - "everyone wants to do the model work, but nobody wants to do the data work" ...
Reid, K., Williams, E.T., 2023. Right the docs: Characterising voice dataset documentation practices used in machine learning, in: Muresan, S., Chen, V., Casey, K., David, V., Nina, D., Koji, I., Erik, E., Stefan, U. (Eds.), Proceedings of the 21st Annual Workshop of the Australasian Language Technology Association. Association for Computational Linguistics, Melbourne, Australia, pp. 51–66.
Maybe we have an open source competitor for ElevenLabs? Check out their demo which they switch between original and synthesized. I can't tell. lol Apparently they're going to fully open source codebase and model weights. #TTS#AI#MLhttps://jasonppy.github.io/VoiceCraft_web/
Grok is a LLM from Elon Musk's xAI, and it's 638GB in fp16! Running on a consumer hardware will be pretty impossible anytime soon even with quantized. Maybe Mac Studio with 192GB. #LLM#AI#MLhttps://huggingface.co/hpcai-tech/grok-1
Continuing our #EverythingOpen Schedule Highlights, we present Faisal Masood of #AWS who will talk about the #ML life-cycle of #data preparation, model #training, testing and deployment, and the role that #automation and #monitoring tools play.
Faisal shows you how to build a model workflow where all team members can collaborate to create a #CI and delivery pipeline for ML models.
Concentration of measures:
Talagrand's "work illustrates the idea that the interplay of many random events can, counter-intuitively, lead to outcomes that are more predictable, and gives estimates for the extent to which the uncertainty is reigned in."
I don't think the tech nerds out there understand how upsetting generative AI is to artists. Not because it will replace them, but because there will be a generation of soulless creation devoid of humanity.
Also, how many children are looking at the progress and thinking 'what's the point of becoming an artist?'. Or how many school directors are thinking 'what's the point of a fine art budget'.
"The issue is that [Google] trained up the [Gemini] foundation model on the polluted ocean and now they're trying to stop the pollution from getting out with a filter, and that doesn't work," he said. "These models were built by drinking a data ocean without cleaning it first. And we have to do better than that." And Microsoft has the same problem, he added. #MLsec#ML#AI#LLM
Prompt "engineering" boils my blood. Can you imagine if you were working on a stream prediction system and the quality of the output depended on prepending a stream of magic numbers? You'd disdain anyone claiming that was a sustainable solution for a business. (I mean, I can imagine it, because that's exactly the kind of crap you see in consulting.) #ML