weekend ai reads for 2023-03-31

🏗️ FOUNDATIONS

• via Rahim, "Bill on a Microsoft podcast talking about ai...  Sounds like it was recorded right before the GPT 4 announcement." (55:47) Microsoft, YouTube

• Nobody’s on the ball on AGI alignment For Our Prosperity

  • Best thing I read this week -- on scalable alignment -- byLeopold Aschenbrenner:

• via Patrick: how do we get large models to accomplish reasoning, especially in math?

  • Math reasoning has traditionally followed symbolic architectures: that is, translating queries into math formulas and using computational solvers – essentially really good calculators. These are verifiable and excellent for high-precision applications. LLMs do not use this approach, but instead use statistical neural designs, which produce outputs that sound good but are not verifiable. The synthesis of traditional symbolic approaches to computation with newer neural approaches is promising; this is called a “neural symbolic architecture”. The creator of one of the most important symbolic solvers wrote about how these designs complement each other. Here is another example (editor: "(MRKL, pronounced "miracle")" ... um, it's clearly not). Stephen Wolfram, arXiv

  • This study benchmarks ChatGPT against a new mathematics dataset (aq: "if your goal is to use [ChatGPT, pre-GPT4] to pass a university exam, you would be better off copying from your average peer!"). Also interesting studies here and here and a recent workshop on this topic (editor: here are videos from the workshop; I haven't watched them). arXiv

  • This article describes how users can put reasoning examples directly into the input prompts for the LLM to understand how to do that exact form of reasoning Tech Talks

  • A related question: “How do LLMs gain the ability to perform complex reasoning using chain-of-thought? @Francis_YAO_ argues it's a consequence of training on *code* - the structure of procedural coding and OOP [object oriented programming] teaches it step-by-step thinking and abstraction.” Yao Fu, Notion

🎓 EDUCATION and AI

• How to recognize AI snake oil Arvind Narayanan, Associate Professor of Computer Science at Princeton University

• Data science tools to study local politics and policy-making in the United States, including school board meetings. It's neat but I hope it gets better. Localview

• related: Census GPT, to explore census data much more easily.

📊 DATA & TECHNOLOGY

• How could AI impact developing economies? Daniel Björkegren, Assistant Professor of Economics at Brown University

• Edge GPT is here (which is why we’re not linking to the “Elon letter”)

  • “Stanford's Alpaca AI performs similarly to the astonishing ChatGPT on many tasks – but it's built on an open-source language model and cost less than US$600 to train up,” by Loz Blain. New Atlas

  • “GPT4All 7B param fine-tuned curated set of 400k ChatGPT assistant style generations;” runs on laptops. Github

• The plug-ins were cool but this is even cooler: ChatGPT + Code Interpreter = Magic Andrew Mayne, Science Communicator at OpenAI

🎉 FUN and/or PRACTICAL THINGS

• I have used regex forever, and still rely on the generosity of StackOverflow to solve 99% of my problems. This requires some knowledge of regex but still a huge step forward.

• One of my perpetual side projects involves spatial data on very large photo collections. So Prismer, the “latest vision-language AI, empowered by domain-expert models in depth, surface normal, segmentation, etc.” from Nvidia Labs, might make my life easier. The demo from HuggingFace is easy to use.

• A Kanye (I know! :-/) vocal model. Worth it for the snippet where “AI Kanye” does Kid Cudi's “Day N Night”. I can't wait for the Dolly Parton version. (Twitter; sorry)