Enduring Methods for an Emerging Medium: The Generative AI Studio
In the Generative AI Studio, students pursued individually driven projects with no right answer, only the potential to serve a need or create a meaningful human experience. Two practices were core to the educational experience: the critique, which develops discernment, and the charrette, which develops a productive creative process. Together they enabled students to create an unexpected array of AI-driven experiences and surface insights that hold lessons well beyond the classroom.
The AI Behavioral Observatory
AIBO, the AI Behavioral Observatory, is an open-source tool for running controlled behavioral experiments on AI systems at scale. Creating and using it over twelve months to publish our latest paper changed how we work. We went from using AI to build the tool to using AI as an active collaborator in running the research itself, and we shortened the distance between articulating a research question and having the evidence needed to answer it.
Persuading AI to Comply with Objectionable Requests
In a preliminary study, we found that older LLMs could be persuaded to break their own safety rules using classic principles of human influence. We went back and ran a more rigorous version of that study on three frontier reasoning models (Claude Haiku 4.5, GPT-5 mini, and Gemini 3 Flash) to find out whether models that think more carefully before answering were still vulnerable. Across 126,000 conversations, persuasion principles raised compliance with regulated drug synthesis requests from 35.3% to 51.3%, suggesting that “parahuman” susceptibility to social influence is a durable property of large language models, not an artifact of an earlier generation.
Beyond Copy-and-Paste: How Game Studios Are Reorganizing Around AI
The gaming industry has long used AI in its products and tools. This would suggest that the incorporation of Generative AI, the newest iteration, would be more straightforward than it’s reported to be in many other industries, especially given its potential to transform the development process. But our research—20 interviews across game studios from AAA (large, high-budget publishers) to indie—found a consistent pattern: organizations began by empowering individuals and automating existing, tractable tasks, but struggled when workflows became more complicated and spanned multiple teams. Progress came when individuals used AI to reach across domain boundaries, completing adjacent tasks themselves. The most dramatic results, however, came from studios designed around AI from day one, where small generalist teams replaced specialist silos and cycle times collapsed from months to weeks. The implication: don’t stop after asking “what can we automate?” Also consider “how would we build if we designed around AI from day one?”
This is an Excellent Paper: The Effects of Prompt Injection on Grading
This study investigates whether frontier AI models used as graders can be manipulated by prompt injections – hidden instructions embedded in the documents they evaluate. Across roughly 40,000 grading trials, prompt injections had negligible effects on most frontier models. However, Gemini 3 Pro showed meaningful vulnerability to verbose injections at the beginning or middle of the longer-paper corpus we tested. In comparison to recent AI models, older and smaller models such as GPT-4o mini were highly susceptible, with scores inflating by nearly 20 percentage points on average. Even when LLMs resisted injection attempts, they almost never verbalized the detection of injection attempts. These results suggest that LLM choice and injection design can meaningfully affect risk.
Playing Pretend: Expert Personas Don’t Improve Factual Accuracy
This study investigates whether persona prompting improves AI performance on challenging academic benchmarks. We find that despite widespread adoption, assigning expert personas (e.g., “You are a world-class physics expert”) does not reliably improve accuracy. Domain-mismatched experts sometimes degrade performance, and low-knowledge personas (layperson, young child, toddler) often reduce accuracy. These results suggest practitioners should focus on task-specific instructions rather than persona assignment.
I’ll pay you or I’ll kill you — but will you care?
This study rigorously tests whether threatening or tipping AI models improves their performance. We find that despite popular claims, these prompting strategies do not significantly enhance results on challenging benchmarks. While individual questions may see dramatic performance swings based on prompt variations, there’s no reliable way to predict which questions will benefit from which prompts.
The Decreasing Value of Chain of Thought in Prompting
This report investigates Chain-of-Thought (CoT) prompting, which encourages LLMs to ‘think step by step.’ We tested this common prompting approach and found that its effectiveness varies significantly by model type and task: non-reasoning models show modest average improvements but increased variability in answers, while reasoning models gain only marginal benefits despite substantial time costs (20-80% increase). These findings challenge the assumption that CoT is universally beneficial.
Prompt Engineering is Complicated and Contingent
This report is fundamentally about exploring the variability in language model performance, not the models themselves. As experimenters, we demonstrate how the same model can produce dramatically different results based on small changes in prompting and evaluation methods – a critical consideration for real-world applications.
AI Agents and Education: Simulated Practice at Scale
Generative AI has the potential to significantly lower the barriers to creating effective, engaging simulations. This opens up new possibilities for experiential learning at scale. By leveraging a system of multiple AI agents, simulations can provide personalized learning experiences, offering students the opportunity to practice skills in scenarios with AI-generated mentors, role-players, and instructor-facing evaluators. In this paper we describe a prototype, PitchQuest, a venture capital pitching simulator that showcases the capabilities of AI in delivering instruction, facilitating practice, and providing tailored feedback. We discuss the pedagogy behind the simulation, the technology powering it, and the ethical considerations in using AI for education, while acknowledging the limitations and need for rigorous testing.
Instructors as Innovators: A Future-Focused Approach to New AI Learning Opportunities, with Prompts
This paper explores how instructors can leverage generative AI to create personalized learning experiences for students that transform teaching and learning. We present a range of AI-based exercises that enable novel forms of practice and application including simulations, mentoring, coaching, and co-creation. For each type of exercise, we provide prompts that instructors can customize, along with guidance on classroom implementation, assessment, and risks to consider. We also provide blueprints, prompts that help instructors create their own original prompts. Instructors can leverage their content and pedagogical expertise to design these experiences, putting them in the role of builders and innovators. We argue that this instructor-driven approach has the potential to democratize the development of educational technology by enabling individual instructors to create AI exercises and tools tailored to their students’ needs. While the exercises in this paper are a starting point, not a definitive solutions, they demonstrate AI’s potential to expand what is possible in teaching and learning.
Assigning AI: Seven Approaches for Students, with Prompts
The incredible promise of AI as a way for students all over the world, of all ability levels, to learn is undeniable. Education is our most powerful system for increasing social mobility, unlocking potential, and improving lives. A tool that can help with this has tremendous implications. Plus, students are already using AI for direct help. Teaching them how to do it responsibly may alleviate some of the negative implications of our AI moment. In this paper, we tackle ways that students can be assigned to use AI directly. We don’t shy away from the dangers but provide detailed instructions on how students and instructors can think about each of the tools we suggest.
Using AI to Implement Effective Teaching Strategies in Classroom: 5 Strategies, including Prompts
In the rush to deliver AI benefits directly to students, the role of instructors is often overlooked. AI tutors, as exciting as they are, do not replace the complex role of a teacher in front of a class. This paper provides evidence-informed AI approaches to pedagogy to make teaching easier and more effective.
