After months of work and $10 million, Databricks has unveiled DBRX - the world's most potent publicly available open-source large language model.
DBRX outperforms open models like Meta's Llama 2 across benchmarks, even nearing the abilities of OpenAI's closed GPT-4. Novel architectural tweaks like a "mixture of experts" boosted DBRX's training efficiency by 30-50%.
Two footmen dressed in white approach the vehicle as it arrives. One opens the rear door. #Guo#Ping, one of #Huawei's rotating chairmen, steps forward and extends a hand as the guest emerges.
After walking a red carpet, the two men enter the magnificent marble-floored building, ascend a stairway, and pass through French doors to a palatial ballroom.
Several hundred people arise from their chairs and clap wildly.
The guest is welcomed by Huawei's founder, #Ren#Zhengfei, whose sky-blue blazer and white khakis signify that he has attained the power to wear whatever the hell he wants.
After some serious speechifying by a procession of dark-suited executives, Ren
—who is China's Bill Gates, Lee Iacocca, and Warren Buffett rolled into one
—comes to the podium.
Three young women dressed in white uniforms enter the room, swinging their arms military style as they march to the stage, then about-face in unison as one holds out a framed #gold#medal the size of a salad plate.
Embedded with a red Baccarat crystal, it depicts the Goddess of Victory and was manufactured by the Monnaie de Paris. Ren is almost glowing as he presents the medal to the visitor.
This #honored#guest is not a world leader, a billionaire magnate, nor a war hero. He is a relatively unknown Turkish academic named #Erdal#Arıkan.
Throughout the ceremony he has been sitting stiffly, frozen in his ill-fitting suit, as if he were an ordinary theatergoer suddenly thrust into the leading role on a Broadway stage.
Arıkan isn't exactly ordinary.
Ten years earlier, he'd made a major discovery in the field of information theory.
Huawei then plucked his theoretical breakthrough from academic obscurity and, with large investments and top engineering talent, fashioned it into something of value in the realm of commerce.
The company then muscled and negotiated to get that innovation into something so big it could not be denied:
the basic #5G#technology now being rolled out all over the world.
Huawei's rise over the past 30 years has been heralded in China as a triumph of smarts, sweat, and grit. Perhaps no company is more beloved at home
—and more vilified by the United States.
That's at least in part because Huawei's ascent also bears the fingerprints of China's nationalistic industrial policy and an alleged penchant for intellectual property theft;
the US Department of Justice has charged the company with a sweeping conspiracy of misappropriation, infringement, obstruction, and lies.
As of press time, Ren Zhengfei's #daughter was under house arrest in Vancouver, fighting extradition to the US for allegedly violating a ban against trading with Iran.
The US government has banned Huawei's 5G products and has been lobbying other countries to do the same. Huawei denies the charges; Ren calls them political.
Huawei is settling the score in its own way. One of the world's great technology powers, it nonetheless suffers from an inferiority complex.
Despite spending billions on research and science, it can't get the respect and recognition of its Western peers. Much like China itself.
So when Ren handed the solid-gold medal
—crafted by the French mint!
—to Erdal Arıkan, he was sticking his thumb in their eye.
ERDAL ARIKAN WAS born in 1958 and grew up in Western Turkey, the son of a doctor and a homemaker.
He loved science.
When he was a teenager, his father remarked that, in his profession, two plus two did not always equal four.
This fuzziness disturbed young Erdal; he decided against a career in medicine. He found comfort in engineering and the certainty of its mathematical outcomes.
“I like things that have some precision,” he says. “You do calculations and things turn out as you calculate it.”
Arıkan entered the electrical engineering program at Middle East Technical University. But in 1977, partway through his first year, the country was gripped by political violence, and students boycotted the university.
Arıkan wanted to study, and because of his excellent test scores he managed to transfer to #CalTech, one of the world's top science-oriented institutions, in Pasadena, California.
He found the US to be a strange and wonderful country. Within his first few days, he was in an orientation session addressed by legendary physicist #Richard#Feynman. It was like being blessed by a saint.
The field was still young, launched in 1948 by #Claude#Shannon, who wrote its seminal paper while he was at Bell Labs;
he would later become a revered MIT professor.
Shannon's achievement was to understand how the hitherto fuzzy concept of information could be quantified, creating a discipline that expanded the view of communication and data storage.
By publishing a general mathematical theory of information
—almost as if Einstein had invented physics and come up with relativity in one swoop
—Shannon set a foundation for the internet, mobile communications, and everything else in the digital age.
The subject fascinated Arıkan, who chose #MIT for graduate studies.
There was one reason: “#Bob#Gallager was there,” he says.
Robert Gallager had written the textbook on information theory. He had also been mentored by Shannon's successor.
In the metrics of the field, that put him two steps from God.
“So I said, if I am going to do information theory,” Arıkan says, “MIT is the place to go.”
By the time Arıkan arrived at MIT, in 1981, Gallager had shifted his focus and was concentrating on how data networks operated.
Arıkan was trembling when he went to Gallager's office for the first time. The professor gave him a paper about packet radio networks.
“I was pushing him to move from strict information theory to looking at network problems,” Gallager says.
“It was becoming very obvious to everyone that sending data from one place to another was not the whole story
—you really had to have a system.”
What's going on is that Anthropic "prompt engineers" have redefined self-awareness to mean 'has contextual information.' That the system is using language then allows them to delude themselves into universalizing their definition.
Saw a similar problem in AI research in the 80s: researchers might define a "frame" holding contextual info, & when their program produced solutions that referenced the frame, construed that as a form of self-awareness. #AIHype#Claude
The wife and I had a good laugh today while I was experimenting with #LLM's. Feeding #Claude a complex prompt that analyzes forum posts, it decided that her career in pharmacy is a crime. 🤣
That said #Claude is still really good, I just need to find the right approach to make it act consistently. I doubt my problem is math, but it is mixing up numbers (0 is 10 is 0 is...). 🤔
I'd assume GPT-4 will work better, but the point is I'm trying to do stuff without OpenAI.
Right now I'm exploring if my idea is viable (it looks promisimg), then I'll move on to recreating it on #Llama and fine tuning.
Any of my American colleagues that have tried #Claude 2.1? No access in EU, but supposedly it outperforms #ChatGPT when it comes to academic writing. @debivort@Pool_Lab@schoppik
i chuckled when i saw the #claude 2.1 launch today. like, “we were shooting for 2.5, but this #openai thing happened and we had to launch something so our engineers compromised by calling it 2.1”
ChatGPT defaults to APA style, but you can use it to format or reformat citations into other styles. And even with a relatively high error rate, it can save you a lot of tedious work.
Also, I have a doozy of a podcasting AI debacle for you in today's newsletter. Let it be a cautionary tale and an example you use when making the case for keeping humans in the workflow.
This week, I highlight how often the different tools like ChatGPT and Claude give bad results. They have different failure rates that also vary depending on what you're trying to do with them. (The range is 3% to 70%).
The ongoing need to check facts is why you must include humans in the process.
Anyway, here are some great anecdotes with hard numbers you can use to make your case!
I've been hearing from a lot of writers and editors who are afraid they are going to lose work to tools like ChatGPT.
I think some jobs will go away, but it won't be as bad as some people fear.
In my newsletter today, I outlined why people will still need writers and editors and how you can change your marketing to be sure your clients understand your value.
If you aren't getting output in the tone you want from AI, give it an example of the kind of writing you do want, and ask it to analyze the tone. And then use those descriptors in your prompts.
It's amazing. I entered two very different pieces of my writing, and it described them better than I could have myself. I have examples in my newsletter this morning.
"AI models don’t contain reality. They rely on the complex statistical abstraction of digital data. This limits their real-world creative significance and their capacity to produce “eureka” moments.
To differentiate AI-driven creativity from old-fashioned creativity, I have proposed a new term: generic, or g-type, creativity. It formalises the fact that while AI models are capable of provoking new thought, they are limited by the underlying data they have been trained on."
"Primärer Cloud-Anbieter": KI-Firma Anthropic und Amazon vereinbaren Kooperation
Nvidia scheffelt dank des KI-Hypes Milliarden. Mit der Milliardeninvestition in Anthropic will Amazon nun auch die eigenen Chips für das KI-Training bewerben.
#AI#GenerativeAI#ChatGPT#Claude#Chatbots: "In conclusion, Claude is the better choice when you have longer content to analyze, but ChatGPT might be better for shorter content. Both tools, however, can expand your curiosity and fill your mind with questions. This curiosity about the world around you can make you more attentive and present."
Tales from the jar side: Live Stream with Baruch Sadogursky @jbaruch, AI tools for Java developers, I am become Barbie, Destroyer of Worlds, and more Tweets / Toots / Skeets