What You May Have Missed #36
Top 5 picks: How do we know how smart AI systems are? / The low-paid workers behind AI / Inside Google’s big AI shuffle (with Demis Hassabis) / GPT-4 demystified / The FTC is investigating ChatGPT
Top 5 Picks
How do we know how smart AI systems are? (Melanie Mitchell on Science): “AI systems, especially generative language systems like GPT-4, will become increasingly influential in our lives, as will claims about their cognitive capacities. Thus, designing methods to properly assess their intelligence—and associated capabilities and limitations—is an urgent matter. To scientifically evaluate claims of humanlike and even superhuman machine intelligence, we need more transparency on the ways these models are trained, and better experimental methods and benchmarks.”
Margaret Mitchell’s chronological thread of articles that have “shone a light on who the low-paid workers behind AI are. Hint: They're not in Silicon Valley.”
Inside Google’s big AI shuffle—and how it plans to stay competitive, with Google DeepMind CEO Demis Hassabis (Nilay Patel on The Verge). Demis Hassabis: “…chatbots and those kinds of systems, ultimately, they will become these incredible universal personal assistants that you use multiple times during the day for really useful and helpful things across your daily lives … I think we know what’s missing: things like planning and reasoning and memory, and we are working really hard on those things. And I think what you’ll see in maybe a couple of years’ time is today’s chatbots will look trivial by comparison to I think what’s coming in the next few years.”
GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Vision, MoE (Dylan Patel and Gerald Wong on SemiAnalysis): “We have gathered a lot of information on GPT-4 from many sources, and today we want to share. This includes model architecture, training infrastructure, inference infrastructure, parameter count, training dataset composition, token count, layer count, parallelism strategies, multi-modal vision adaptation, the thought process behind different engineering tradeoffs, unique implemented techniques, and how they alleviated some of their biggest bottlenecks related to inference of gigantic models. The most interesting aspect of GPT-4 is understanding why they made certain architectural decisions.”
The FTC is investigating whether ChatGPT harms consumers (Cat Zakrzewski on the Washington Post): “The Federal Trade Commission has opened an expansive investigation into OpenAI, probing whether the maker of the popular ChatGPT bot has run afoul of consumer protection laws by putting personal reputations and data at risk … Among the information the FTC is seeking from Open AI is any research, testing or surveys that assess how well consumers understand “the accuracy or reliability of outputs” generated by its AI tools … extensive details about its products and the way it advertises them … a detailed description of the data that OpenAI uses to train its products … The agency also asked OpenAI to describe how it refines its models to address their tendency to “hallucinate…” Sam Altman’s response.