Nick Proctor
Executive Director
Reacting Consortium
This post is excerpted from the Winter Newsletter that will be sent in January. It has been posted early here because of its relevance to current trends and discussions in the Reacting Community.
Early this December, discussions of ChatGPT, the language assembly AI, dominated our Facebook group. As usual, the discussion was passionate and informed. Several people posted especially thoughtful comments.
One of the best was on December 8th when Jamie Lerner-Brecher shared the results of her prompt, “write a speech as Thrasybulus talking about why Athens should become a democracy.” In ten seconds, the AI produced a capable, well-organized five paragraph essay. One paragraph read as follows,
“First of all, democracy is not chaos. On the contrary, it is a form of government that is based on the rule of law and the protection of individual rights. In a democracy, the people are free to express their opinions and to participate in the decision-making process, but they must do so within the bounds of the law. This means that a democracy can be orderly, fair, and just.”
As I contemplated Jamie’s post, I was struck by the degree to which the AI relied on 21st century concepts. If a player presented ChatGPT’s work as a speech, I wondered what might happen when other players started posing questions.
You say that in democracy "the people are free to express their opinions," does this mean you are criticizing the Reconciliation Agreement?
What do you mean by the "rule of law"? Don't you mean the will of the people?
Is Athenian democracy really about the "protection of individual rights"? Does this mean we can't put Socrates on trial?
Someone who did not write the paper would be hard pressed to answer. Of course, this would require other players to listen and to think. Two things that do not always happen.
Regardless, I was feeling pretty good when Javier Hidalgo shared some more essays. These were sharper because they used a point of view. They were also meatier because he asked the AI to include quotations from appropriate sources. I think these would still fall apart under questioning. This was echoed by William H. Campbell who commented, “The more we lean into informed viva voce debate rather than writing, the harder it will be for students to use AI.”
Javier agreed, but expressed understandable concern about relying on debate, which often moves very quickly, for evaluation. I agree. I’m confident when I’m marking essays, but I don’t know that I assess classroom engagement very well. If I was looking at that paper while a player floundered with her answers, I would probably think that she had just gotten flustered.
As I was mulling this over, my colleague, Rebecca Livingstone, asked me to visit her Vietnam Memorial game as a special guest star. Her students had created an aesthetic disaster and she wanted a Reagan administration official to press them on their dubious decision-making. It was here that I started to see the limitations of ChatGPT more clearly (at least in its current form).
After I left Becca’s class, I spent the next hour engaging it by asking and then refining questions like, “What would H. Ross Perot think about Maya Lin’s design if it included seven marble statues representing ideas like ‘suffering’ and ‘racism’”? If you glanced at the role sheet or the gamebook, you would know the answer in an instant, but ChatGPT was stymied. The flat, grammatically flawless answers that it produced did not begin to understand the question. It understood all the elements, but it could not combine them.
I think there are two reasons for this. First, I was asking it for a distinct point of view. Javier had better luck with this, but it struggled with all the ones I tried for Vietnam Memorial. Second, the situation was weird and unprecedented. Lacking a good baseline from which to respond, the responses were always off the mark. Mostly, I received reassurances that different people’s responses would depend on their life experiences.
As I contemplated ways in which Reacting might respond to this very new set of challenges, it struck me that they are, in many ways, not new at all. As he tells the tale, Mark Carnes was faced with a similar dilemma all those years ago when he walked across Broadway, headed to a class full of very smart students who would give him grammatically flawless answers to questions that have been asked many times before. His solutions then provide us with many of our solutions now: give students points of view, get them talking to each other, and when things get weird, embrace it. They are only human, after all.
About the Author
Nicolas Proctor is a history professor who writes games and enjoys teaching classes. As a history professor, he likely has a deep understanding of the subject and is able to convey complex ideas in an engaging and accessible way. His passion for teaching is evident in the time and effort he puts into creating games, which can be a fun and interactive way for students to learn about history. Whether in the classroom or through his games, Nicolas Proctor's love for teaching and history shines through. Thanks, ChatGPT!
Blog Author Questionnaire
One word to describe faculty: Dedicated
Two words to describe (your) school: Tidily Midwestern
Three words to describe students: Generally stressed out
Four words to describe favorite games: They make players think
Five words to describe Reacting: Mark shared his toys, yay!
