weekend ai reads for 2025-08-29

When we added safety mitigations to autonomous mode, we reduced the attack success rate of 23.6% to 11.2%, which represents a meaningful improvement over our existing Computer Use capability (where Claude could see the user’s screen but without the browser interface that we’re introducing today).

when users ask it to “Summarize this webpage,” Comet feeds a part of the webpage directly to its LLM without distinguishing between the user’s instructions and untrusted content from the webpage. This allows attackers to embed indirect prompt injection payloads that the AI will execute as commands. For instance, an attacker could gain access to a user’s emails from a prepared piece of text in a page in another tab.

This kind of thing should make everybody stop and really think before deploying any AI agents. We simply don’t know to defend against these attacks. We have zero agentic AI systems that are secure against these attacks. Any AI that is working in an adversarial environment—and by this I mean that it may encounter untrusted training data or input—is vulnerable to prompt injection. It’s an existential problem that, near as I can tell, most people developing these technologies are just pretending isn’t there.

Anthropic’s approach here is the most open-eyed I’ve seen yet but it still feels doomed to failure to me.

A significant portion of LLM-generated research ideas appear novel on the surface but are actually skillfully plagiarized in ways that make their originality difficult to verify.

It found that employment for workers ages 22 to 25 in the most AI-exposed jobs, including software development and customer service, has dropped by 13% relative to less-exposed roles.

It also creates an unsustainable creative ecosystem. Today’s generative AI models are built on the world’s creative output—mostly uncompensated. If we don’t value and sustain human creativity now, future creativity will be nothing but AI-remixed leftovers.

Good design doesn’t mean shoving every bell and whistle into a product. (This is one way I see many startups struggle, particularly in vertical AI: skeptical lawyers and construction workers and restauranteurs don’t want a product overloaded with state-of-the-art features from the latest model; they want a simple, intuitive product that gets the job done).

Some educators are automating grading; others are deeply opposed

In our Claude.ai data, faculty used AI for grading and evaluation less frequently than other uses, but when they did, 48.9% of the time they used it in an automation-heavy way (where the AI directly performs the task). That’s despite educator concerns about automating assessment tasks, as well as our surveyed faculty rating it as the area where they felt AI was least effective.

Get a detailed college audit with personalized profile recommendations, a day-to-day planner, and Cori, the AI counselor to help you manage your tasks & get into your dream schools.

Overall, we’re seeing a ~30% increase in speed from the feature proposal to when it hits prod. This is huge for us.

TL;DR: Always start with a solid design doc and architecture. Build from there in chunks. Always write tests first.

Gemini’s new AI image model is designed to make more precise edits to images — based on natural language requests from users — while preserving the consistency of faces, animals, and other details, something that most rival tools struggle with.

And even as it consumes those who use it, even as the scammers become their own marks, even as it is sustained by exploited workers slotted in as human filters for algorithmic abuse – some people want to have a little, as a treat. As a joke. Just to make fun of it, just for the busywork, because it’s good enough, right? You understand.

What I’m really worried about is those companies and services and products disappearing altogether—because they’re replaced by more efficient companies and products that do the same thing but better and with fewer people.