- That AI Thing
- Posts
- weekend ai reads for 2024-05-10
weekend ai reads for 2024-05-10
📰 ABOVE THE FOLD: ARTISTIC EXPRESSION
This year’s Met Gala theme is AI deepfakes / Tech Crunch (4 minute read)
FKA twigs creates deepfake of herself, calls for AI regulation / Mashable (4 minute read)
She pointed out that the likes of deepfakes and AI technology can be "highly valuable tools both artistically and commercially when under the control of the artist."
AI video throwdown: OpenAI’s Sora vs. Runway and Pika / Ars Technica (5 minute read)
“I had to laugh when I watched this Runway one. There’s a bit more photorealism, but the people are walking forwards and then backwards, so it’s certainly not a believable scene.
“As an industry professional, my expectation is perfection. I am looking for realistic-quality video, and AI is probably never going to quite get there.
“At the end of the Sora video, the couple is having a conversation in a coffee shop, looking like they’re enjoying themselves. That would be a shot that we’d use to sell a commercial property space as an amenity nearby.
Nike developing AI model as part of design “step change” / Dezeen (6 minute read)
“It's a little bit of thinking about developing a private garden, of looking at our own datasets that are exclusive to Nike – so performance data from an athlete, from our laboratories, et cetera. And then kind of commingling that with some things from the public garden, but making sure that that's all contained within what we're training the model on.”
the concepts at the link are interesting in that they seem impractical but also somehow recognizably attributed to the athlete-slash-sport; except Mbappe’s … what???
animate your word! / Animate Your Word, Github (5 minute read; or 30 second skim of the examples)
We present an automated text animation scheme, termed “Dynamic Typography,” which combines two challenging tasks. It deforms letters to convey semantic meaning and infuses them with vibrant movements based on user prompts.
📻 QUOTES OF THE WEEK
Make the marketers and ad companies earn their money. Be original and real and you’ll strengthen whatever community you love.
Marcus Brown (aka Nourished By Time) (source)
12. Knowing when to stop is a form of talent.
Frank Chimero (source)
🏗️ FOUNDATIONS & CULTURE
AI Safety for Fleshy Humans: a whirlwind tour / Nicky Case (60+ minute read, but absolutely worth it)
Nicky Case is an artist, and creator of interactive media projects that explore complex topics in engaging, humorous, and thought-provoking ways. She is always great.
for example, “How to Explain Things Real Good” / Nicky Case (17 minute video)
related, Model Spec / OpenAI Blog (51 minute read)
OpenAI explains how they think about and establish guardrails
Slop is the new name for unwanted AI-generated content / Simon Willison’s Weblog (2 minute read)
But I’m increasingly of the opinion that sharing unreviewed content that has been artificially generated with other people is rude.
Slop is the ideal name for this anti-pattern.
related (1), Meet AdVon, the AI-Powered Content Monster Infecting the Media Industry / Futurism (22 minute read)
related (2), Warren Buffett says AI scamming will be the next big ‘growth industry’ / CNBC (3 minute read)
Ninety-five theses on AI / Samuel Hammond, Second Best (sorry) (12 minute read)
Within the next two years, frontier models will cross capability thresholds that even many open source advocates will agree are dangerous to open source ex ante.
related, Vico’s Singularity / Henry Farrell, Programmable Mutter (sorry) (13 minute read)
How LLMs Work, Explained Without Math / Miguel Grinberg (22 minute read)
The assumption that most people make is that these models can answer questions or chat with you, but in reality all they can do is take some text you provide as input and guess what the next word (or more accurately, the next token) is going to be.
🎓 EDUCATION
A Careful Examination of Large Language Model Performance on Grade School Arithmetic / arXiv (37 minute read)
As such, we hypothesize that the reward modelling process may have leaked information about the correct reasoning chains for GSM8k even if the problems themselves did not ever appear in the dataset. Finally, we observe that the Llema [sic] models have both high log-likelihoods and minimal overfit. These models are open-sourced alongside their training data, and the authors report finding a very small number of GSM8k examples in the training corpus. Nevertheless, they also find (and our study supports) that these few instances do not lead to overfitting. The existence of these outliers suggests that overfitting on GSM8k is not purely due to data contamination, but rather may be through other indirect means, such as model builders collecting data similar in nature to benchmarks as training data or selecting final model checkpoints based on performance on benchmarks, even if the model itself may have not seen the GSM8k dataset at any point via training. Conversely, the reverse is also true: small amounts of data contamination do not necessarily lead to overfitting
so good; not only a look at how LLMs currently perform against benchmarks, but a great analysis of some of the implications of “public” benchmarks.
one takeaway is that private test sets with public results or leaderboards, like Scale AI’s GSM1k, are one way to make sure that test data are not leaked into training data, and is one way to prevent overfitting
How we use generative AI tools / University of Cambridge (9 minute read)
more institutions and organizations should have this level of clarity in their AI statements
Co-Intelligence: AI in the Classroom with Ethan Mollick / Global Silicon Valley, YouTube (25 minute video)
when an author has a new book coming out, watching their talks or listening to their podcast appearances is a great way to get the gist of the book without reading it
New Canva Research Reveals Amid Burnout, Teachers Are Ready to Embrace AI / press release (6 minute read)
35% of all K-12 educators experienced burnout daily
66% report working beyond contractual hours
92% of teachers using AI found it helpful in addressing pain points at work
📊 DATA & TECHNOLOGY
Machine Unlearning in 2024 / Ken Ziyu Liu, Stanford University (41 minute read)
Machine unlearning can be broadly described as removing the influences of training data from a trained model. At its core, unlearning on a target model seeks to produce an unlearned model that is equivalent to—or at least “behaves like”—a retrained model that is trained on the same data of target model, minus the information to be unlearned.
related (1), Microsoft, Google and Meta Bet on Fake Data to Train AI Models — Top artificial intelligence companies are experimenting with a different approach to meet their insatiable need for data. / Bloomberg (8 minute read)
related (2), In-Context Learning with Long-Context Models: An In-Depth Exploration / arXiv (62 minute read)
related (3), llm-datasets — High-quality datasets, tools, and concepts for LLM fine-tuning. / mlabonne, GitHub (12 minute read)
Building Models That Learn From Themselves — AI developers are hungry for more high-quality training data. The combination of agentic workflows and inexpensive token generation could supply it. / The Batch, Deep Learning (4 minute read)
related, How to Beat Proprietary LLMs With Smaller Open Source Models — Building your AI applications around open source models can make them better, cheaper, and faster / Aiden Cooper, Impromptu Engineer (14 minute read)
How to build a robust AI Agent stack [CrewAI + YouTube API + Ollama + Groq + AgentOps] / The How-to Guy, YouTube (36 minute video)
In this video, we'll discuss how to create AI agents that interact with the YouTube Data API to extract comments from any given video and generate actionable insights. Based on user feedback, these agents can help you understand and create better content.
have not worked all the way through this, but promising approach to agentic workflows
🎉 FUN and/or PRACTICAL THINGS
Geospy — Photo location prediction using Al
upload a photo and it will guess where it was taken
eerily accurate, in our brief testing
Randy Travis Used AI to Record His First Song Since Near-Fatal Stroke — Travis, who lost his speech after suffering a stroke in 2013, used an AI voice clone on “Where That Came From” / Rolling Stone (5 minute read)
the song: Where That Came From (Lyric Video) / Randy Travis, YouTube (3 minute video)
Rizemail — Forward your email to us, we'll return a summary
obviously, don’t submit any private information
🧿 AI-ADJACENT
The AI Hardware Dilemma — Why new devices are flopping—and how they might succeed / Napkin Math, Every (8 minute read)
⋄