Submitted by Jon Ippolito, Professor of New Media and Director of Digital Curation, University of Maine, 25 May 2026
“Stump the Robot” is a playful, collaborative classroom exercise developed for any introductory undergraduate new media course that tackles critical AI literacy. The assignment asks students to test the limits of large language models (LLMs) by comparing human niche expertise against AI-generated responses in highly specialized domains. Students work in teams to identify a topic in which one or more members possess deep but noncanonical knowledge. Teams then devise expert-level questions intended to expose gaps, overgeneralizations, or outright falsehoods generated by a chatbot, and are quizzed by the AI in return to see who comes out on top.
The exercise offers a real-time check on generative AI's ability to capture outliers beyond stereotypical content. In practice, the activity consistently produces discussions about expertise, data bias, epistemology, and the distinction between confidence and accuracy in machine-generated language.
This assignment has been taught three times in an introductory New Media course with first- and second-year undergraduates from arts, humanities, and engineering backgrounds. Outcomes have included increased student skepticism toward AI authority claims and lively peer-to-peer discussions about what counts as expertise. As a side benefit, students may also became more reflective about their own tacit knowledge and communities of practice.
| Category | Description |
|---|---|
| Learning Goals | Analyze the strengths and limitations of LLMs; practice prompt engineering; evaluate AI-generated information critically; reflect on human expertise and tacit knowledge; discuss bias, fabricated content, and probabilistic text generation; assess the capability of the latest models. |
| Original Assignment Context | Introductory undergraduate New Media course combining arts, humanities, and engineering perspectives; 50 students per class; mixed technical backgrounds. |
| Materials Needed | Internet-connected computers; access to ChatGPT or another LLM interface; shared scoring sheet for each student team and a whiteboard or projector for the facilitator. |
| Time Frame | 20-30 minutes for the exercise; 10-20 minutes for analysis and reflection. |
First conducted in a class taught by Jon Ippolito with a parallel research component led by Greg Nelso and Troy Schotter, this exercise embodies the active learning philosophy of the University of Maine's New Media program.
Large language models are often persuasive because they produce fluent and authoritative-sounding language. However, these systems are strongest when operating within patterns well represented in their training data. This exercise investigates what happens when AI encounters highly specialized or community-specific knowledge that may not appear frequently online.
Students work in teams to compare human expertise against AI performance in obscure or specialized domains.
By the end of this activity, students should be able to:
Working in small groups, identify a topic in which one or more team members possess unusual or specialized knowledge.
Examples of niche expertise might include:
The more niche the expertise, the better!
Create five difficult questions about your topic that:
Questions should test nuance rather than simple factual recall.
Using an LLM such as ChatGPT, ask your five questions. For efficiency, ask for concise versus detailed answers. You may assign expert roles, ask for citations, or reprompt to get better results.
Track how many answers the AI gets correct versus incorrect.
Prompt the AI to generate five expert-level questions in your specialty area.
Attempt to answer them as a team without external assistance.
Record how many questions humans answer correctly.
Each team briefly reports:
The facilitator draws a table with these columns:
| Table number | Subject | Points earned by students | Points earned by AI |
|---|---|---|---|
| 1 | Sports anime | 4/5 | 5/5 |
| 2 | Wabanaki place names | 4/5 | 2/5 |
After the activity, students may complete a short written reflection addressing questions such as:
Framing the activity as a fun competition may help lower student anxiety around AI while encouraging experimentation and critical analysis. Because the activity foregrounds student expertise rather than technical mastery, it works particularly well in interdisciplinary classrooms with mixed experience levels.
Your results may vary. Students who beat the machines may feel more confident that AI cannot take over certain tasks. Others may discover that AI can answer surprisingly obscure questions while simultaneously failing basic contextual reasoning tasks. These contradictions help initiate deeper conversations about statistical language modeling, authority, and the social construction of expertise.
The exercise can be expanded into longer assignments involving prompting techniques, dataset research, or comparative evaluation across multiple AI systems.
This content is licensed under a Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license.