weekend ai reads for 2023-07-14

📰 ABOVE THE FOLD: LONGREADS

• The risks of AI are real but manageable Gates Notes (3,000 words)

  • Also, Bill Gates is (or his people are) on Threads

• The Building Blocks of Generative AI — A Beginners Guide to The Generative AI Infrastructure Stack Jonathan Shriftman, Substack (5,500 words)

  • Primer on the functions and market players among the various Generative AI technical components

  • tl;dr image here

• The LLMentalist Effect: how chat-based Large Language Models replicate the mechanisms of a psychic’s con Out of the Software Crisis (5,300 words)

  • Counterpoint to GPT-is-magic, positing we’re convincing ourselves that GPT chatbots are better than they are

  • related; slightly less skeptical

    • Hallucinations in AI — everything you think you know about AI has been told to you by someone with an incentive to lie (and mostly to themselves). Pirate Wires

• Synthetic vs Real Data: Why do models perform worse when trained on synthetic data? Superfast AI (2,800 words)

  • Explainer on both how LLMs are trained and how synthetic data can make things go awry; lots of helpful diagrams to explain the concepts

  • Synthetic data is one proposed approach to bigger, better LLMs, so these challenges with these approaches are worth noting

• Generative AI and the Near Future of Work: An EdTech Example eLiterate (5,900 words)

  • Half is a summary of how systems integration and ed tech standards might be changed by the application of AI; other half is speculative about improving collaboration among institutions

  • The five main points are restated succinctly at the bottom of the article

📻 QUOTES OF THE WEEK

Kjirste Morrell, on AI versus humanity (source)

Sara Azout (source)

🏗️ FOUNDATIONS & CULTURE

• Claude, from Anthropic

  • A new LLM-based chatbot to compete with ChatGPT; has cheaper APIs, allows for easier upload of documents (up to five), and can access data through 2023 (versus 2021 for ChatGPT)

  • Requires sign-up; no cost (yet)

  • related: Poe, which lets you use multiple chatbots in one interface

• via mimi, AI Is a Lot of Work — As the technology becomes ubiquitous, a vast tasker underclass is emerging — and not going anywhere. New York Magazine 

  • related (1): Mechanical Turk workers are using AI to automate being human Tech Crunch

    • The ouroboros is perfect

  • related (2): Google’s AI Chatbot Is Trained by Humans Who Say They’re Overworked, Underpaid and Frustrated Bloomberg

  • related (3): The workers at the frontlines of the AI revolution — The global labor force of outsourced and contract workers are early adopters of generative AI — and the most at risk. Rest of World

• My A.I. Writing Robot — A new wave of artificial-intelligence startups is trying to “scale language” by automating the work of writing. I asked one such company to try to replace me. The New Yorker

• Does Artificial Intelligence Help or Hurt Gender Diversity? Evidence from Two Field Experiments on Recruitment in Tech Social Science Research Network

• An Indian politician says scandalous audio clips are AI deepfakes. We had them tested Rest of World

  • spoiler: at least one of them probably wasn’t

  • “Deepfake” is going to become the new “fake news” when politicians want to parry attacks

• Black Artists Say A.I. Shows Bias, With Algorithms Erasing Their History — Tech companies acknowledge machine-learning algorithms can perpetuate discrimination and need improvement. New York Times

🎓 EDUCATION

• Programs to detect AI discriminate against non-native English speakers, shows study The Guardian

  • The paper

• AI and the next digital divide in education The Brookings Institution

  • Short commentary exploring a potential new digital divide, which is: “The rich have access to both technology and people to help them use it, while the poor have access to technology only.”

• How AI Can Help Educators Test Whether Their Teaching Materials Work EdSurge

  • Excerpt of a podcast from the creators of Adaptive Experimentation Accelerator, which employs a “strategy of ‘adaptive experimentation’ to regularly improve their teaching materials.”

  • to note: they apparently use unpaid student labor to develop their model

• How Artificial Intelligence is Powering Education with Dr. Ann Marie Sastry (55:00) Ark Invest Podcast

  • Dr. Sastry is CEO of Amesite

• AI-related classes have doubled at Stanford over the past five years — showing just how fast it’s taking over Business Insider

📊 DATA & TECHNOLOGY

• Announcing the first Machine Unlearning Challenge Google Blog

  • A simple flow diagram showing the purpose and objective of the challenge

  • Having ML models “forget” data — for example when a user opts out — would often require retraining the model on the remaining dataset. This challenge seeks to find solutions to “use the already-trained model as a starting point and efficiently make adjustments to remove the influence of the requested data.”

• GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Vision, MoE Semianalysis

  • Apparently, the GPT-4 architecture has leaked. The some key insights, which are unconfirmed:

    • 1.8 trillion parameters, which is about 10x GPT-3

    • Trained on about 25,000 A100s for between 90 and 100 days at about 32% to 36% MFU

      • The low utilization is partially to account for the high failure rate, which in turn required restart checkpoints

    • The training costs for this run alone are estimated at about $63 million, assuming about $1 per A100 hour

      • The cost today would be about a third of this; compute cost is coming down fast.

    • Training data is secret but rumored to include Reddit, Twitter, Sci-Hub, LibGen, with a particular focus on textbooks

• Data Clean Rooms — Guidance and Recommended Practices [PDF] IAB Tech Lab

• via andy, Building Boba AI — Some lessons and patterns learnt in building an LLM-powered generative application Martin Fowler

  • Deep-dive into the development of a co-pilot tool

• The next artificial intelligence frontier: Causal AI Semafor

  • Causal AI was first mentioned on 2023-03-34

🎉 FUN and/or PRACTICAL THINGS

• via jake b, Results from AI21’s Human or Not? experiment (now closed) AI21

• New Clippy app gives us a taste of AI in Windows 11 — While we wait for Copilot, an unofficial Clippy app has launched for Windows 11, bringing ChatGPT-like features to your desktop. XDA

• Momento — make short videos out of long ones

  • No one who reads this has time to make longform videos. But if they did, this tool uses AI to identify the key moments the longform videos, and convert those moments into short videos for social media platforms

  • Someone apply this to class lectures, corporate training sessions, etc.

• via marc, AI Imagines a Typical Home in Every State All Star Home

  • Very slow webpage; load it, read all the other articles, and come back

• How to Use A.I. for Family Time — Plan meals, find gifts and create stories using generative A.I. New York Times

• Danswer is an open source Enterprise Question Answering Tool.

  • Open-source tool to ingest large sets of documents that can then be interrogated via a chatbot

  • Works surprisingly well with Google Drive; has a handful of other integrations as well

• Verble — AI speechwriting assistant that helps you master the art of verbal persuasion and storytelling.

  • Decent results but nothing some good prompts in any chatbot couldn’t get you

  • “Pro” tier is free through 01 September

• TwitterGPT — TwitterGPT Analyzes a Twitter User's Profile Using AI and Creates a Detailed Report.

  • Output feels a generic like a horoscope but scary how much of (y)our lives are available for anyone to peer into

• Two guys who were fed up with spam calls created a ChatGPT tool that trolls telemarketers and wastes their time Business Insider

  • the site: https://jollyrogertelephone.com/

🧿 AI-ADJACENT

• xAI

  • It’s Elon Musk, so it’s either vaporware like the Hyperloop, or industry-changing like SpaceX; his track record lately …

👉 ICYMI: The most-clicked link from last week’s newsletter was this post from Ethan Mollick on ChatGPT’s Code Interpreter.