What You May Have Missed #29
Research (5% US adults find ChatGPT very useful, Falcon, Voyager, MMS, CoDi, QLoRA) / Products (Copilot, Firefly) / Essays (Best: Altman vs EU) / Talks (State of GPT) / Misc (The ChatGPT lawyer case)
Research
A majority of Americans have heard of ChatGPT, but few have tried it themselves. (Emily A. Vogels on Pew Research Center): “About six-in-ten U.S. adults (58%) are familiar with ChatGPT” but “Just 14% of U.S. adults have tried [it].” Among that 14%, only 15% have found it “extremely useful” for work, education, or entertainment. That’s 2% of all US adults. 20% have found it “very useful.” In total, people who find ChatGPT significantly useful are 1 in 20. 5%. This is a lot of people but given the amount of attention and coverage generative AI—and ChatGPT—receives, it feels quite too little. Maybe we’re calling it a revolution too soon? And if it’s too soon, why say it at all?
Falcon-40B: Falcon “is the best open-source model currently available. Falcon-40B outperforms LLaMA, StableLM, RedPajama, MPT, etc. … features an architecture optimized for inference ... [and] it is made available under a license allowing commercial use.” Here’s the Hugging Face Open LLM Leaderboard.
Jim Fan: “GPT-4 unlocks a new paradigm: “training” is code execution rather than gradient descent. “Trained model” is a codebase of skills that Voyager iteratively composes, rather than matrices of floats. We are pushing no-gradient architecture to its limit.”
Introducing speech-to-text, text-to-speech, and more for 1,100+ languages. (META blog): Yann LeCun: “MMS: Massively Multilingual Speech. Can do speech2text and text speech in 1100 languages. Can recognize 4000 spoken languages. Code and models available under the CC-BY-NC 4.0 license. half the word error rate of Whisper.”
CoDi: Any-to-Any Generation via Composable Diffusion. (GitHub) “We present Composable Diffusion (CoDi), a novel generative model capable of generating any combination of output modalities, such as language, image, video, or audio, from any combination of input modalities … This allows CoDi to freely condition on any input combination and generate any group of modalities, even if they are not present in the training data.”
The False Promise of Imitating Proprietary LLMs (arXiv): “…we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs.”