ChatGPT Created My Course – Now It’s Teaching It!
The more hands-on experience I have working with ChatGPT with students, the more I realize how much of a game changer AI is in education
Several weeks ago I was asked to write an article for Slate Future Tense on my new course at ASU on developing professional skills in using ChatGPT. What piqued my editor’s interest was the degree to which ChatGPT has been integral to creating and teaching the course.
That article has just been published, and if you’re interested in getting a glimpse into how we’re pushing the boundaries of how AI chatbots can be used in higher education, I’d encourage you to read it:
However, I thought I’d also post the first full draft of article here — the one prior to my editor working their magic! It’s something I’ve done a few times now as early drafts often include insights and perspectives that don’t make the final cut, and are worth reading despite their often-raggedy edges.
Plus, it’s sometimes interesting to see the evolution of a piece from initial idea to edited article — especially when your editor helps sharpen thinks up a lot!
(And if you’re interested in the course content, much of it is freely available here).
ChatGPT Created My Course – Now It’s Teaching It!
I’ve just started a course that was designed by ChatGPT, uses ChatGPT, and is even assessed by ChatGPT. In principle I should be up in arms over this blatant outsourcing of education to AI. The only problem is, I’m the instructor – and I’m loving it!
It all started earlier this year when news started circulating of job openings for “prompt engineers” with salaries of over $300,000 per year. As a professor who studies and teaches advanced technology transitions, my academic ears pricked up – was there a skillset here we should be teaching our students? And if so, how should we go about it?
And so I did what any self-respecting tech savvy prof. would do – I fired up ChatGPT.
Within a couple of hours I had the outline of a course on prompt engineering using ChatGPT – complete with learning objectives, assignments, and lecture notes. And it was good – better than I could have produced on my own in the time.
It was a profound wakeup call to the power of this new tool in higher education.
Fast forward to the present, and I’m now teaching this course. Admittedly there were plenty of iterations along the way – that initial outline was good, but not perfect, and the course benefitted from some much-needed input from colleagues. There were also some learning objectives that ChatGPT didn’t initially identify which we considered important – including addressing the broader societal implications of large language models and AI chatbots. But the vast majority of the course I’m now teaching is designed, written, and executed by ChatGPT.
At its core, this is a course about giving undergraduate students from all backgrounds and abilities a unique and transferrable skillset in using ChatGPT and other AI chatbots. And while it emphasizes “prompt engineering” it puts a non-technical spin on this, defining it as “the art and skill of crafting, optimizing, and employing every-day language to effectively harness the power and capabilities of Large Language Models and AI chatbots, leveraging their potential and making them useful and accessible across a very wide range of professional and personal situations while ensuring accurate and valuable outcomes.”
(And yes, ChatGPT did help out with the definition!)
The course is designed to be taken online, with students going at their own pace. It’s built around six modules, each using a series of exercises that use ChatGPT to tutor students, expand their understanding, and even assess them. Many of these exercises were developed in collaboration with ChatGPT – in at least one case we use a ChatGPT-designed exercise with no modifications.
Out of the initial five learning objectives proposed by ChatGPT I ran with four of them: understanding large language models and their limitations; prompt formulation and refinement; developing and using prompt templates; and prompt and response evaluation. The two “human derived” additions include addressing emerging trends and exploring responsible innovation and use.
Within each of these learning objectives the course has specific skills and areas of understanding that were developed through working with ChatGPT. For instance, when addressing prompt formulation and refinement the course uses a framework of ambiguity reduction, constraint-based prompting, and comparative prompt engineering, which was suggested by the chatbot. And when exploring prompt templates the course follows ChatGPT’s lead on developing skills that support the development of reusable prompt formats.
But it was when I came to developing content around prompt and response evaluation that I began to realize just how much of a game changer ChatGPT can be.
As I was working on developing an approach for testing the usefulness of prompts and the responses they elicit, ChatGPT came up with a new framework: The RACCCA framework.
RACCCA stands for relevance, accuracy, completeness, clarity, coherence, and appropriateness. It’s an approach to testing the usefulness of prompts and responses, and iterating to improve the quality of ChatGPT outputs, that works surprisingly well. What bowled me away though is that ChatGPT even came up with the acronym!
As a result, the course now covers a novel approach to evaluating prompts and responses that was devised by ChatGPT, and that uses ChatGPT to teach it. It’s a case of AI as mentor and instructor that has me wondering when I’ll be out of a job!
Thankfully I suspect this won’t happen any time soon as the course is also demonstrating the power of human-ChatGPT collaboration. None of this would be possible without me as the “human in the loop” curating and crafting ideas and directions suggested by ChatGPT. But neither would it work without the ability of ChatGPT to augment learning in ways that empower and extend the reach of me as a mere human instructor.
This is seen to good effect in a number of places in the course where learning and assessment are led by ChatGPT using exercises that were co-created with me.
For instance from the last module in the course that covers the broader societal implications of ChatGPT we co-devised an exercise that turns ChatGPT into a highly effective personal instructor:
In a new session, provide ChatGPT (using GPT4) with the following prompt “Hi ChatGPT. My name is [include full name] and I would like you to act as my personal tutor and teach me about responsible innovation in the context of using ChatGPT. I would like you to cover the field broadly. Please start by asking me a question that helps you gauge my level of understanding. Based on my response, ask me a follow-up question that is designed to increase my understanding. Continue to do this until I show a broad understanding of responsible innovation in the context of using ChatGPT.”
While there are obvious dangers with exercises like this leading to incorrect information from ChatGPT, it’s hard to convey the transformative power of such prompts on learning – even within understood limitations – until you’ve experienced it for yourself. In this case, ChatGPT is a remarkably good tutor when it comes to responsible innovation.
The course also uses ChatGPT to assess student understanding. For example, this exercise from module 1 is a simple test of understanding of large language models:
Start a new chat with ChatGPT (making sure you are using GPT-4) and cut and paste the following prompt: “Hi ChatGPT. My name is [add your full name] and I am in a class where we are learning about the uses and limitations of LLMs. Please ask me three simple questions about the uses and limitations of LLMs to test my understanding. After each question, please wait for my answer before asking the next one. When you have all three of my answers, please provide an assessment of how good they are, and give me a grade from A to C.”
Each time the prompt is run the questions are different, and students can repeat it as many times as they like to get the grade they are looking for.
Of course, the point isn’t the letter grade but the process of question, response, feedback, and iteration, that leads to self-directed discovery and knowledge reinforcement. It’s an approach that prioritizes personalized learning and leverages the ability of ChatGPT to act in part as a class co-instructor.
So should I be looking for a new job now that ChatGPT has proved itself as an able course designer, instructor, and mentor? I don’t think so – at least not just yet. Generative AI platforms like ChatGPT are going to be transformative in education – that much is already apparent, and we’ve barely scratched the surface yet. But we still need an informed human in the loop to get the most out of these new technologies – someone who knows how to leverage AI to ensure students have the most effective learning pathways possible at their fingertips to achieve what they aspire to.
This is, though going to require professors like myself to acquire new skillsets and to recalibrate how we think about learning and education.
Over the next few weeks, I’m going to be reading well over 2,000 conversations between students and ChatGPT as part of the course. It’s a unique opportunity to see first-hand how students interact with the platform, and how this potentially sparks their curiosity and enhances their learning.
Already, I’m beginning to think differently about the power of ChatGPT to transform learning and unlock the nascent interests and abilities of our students. It's almost as if ChatGPT is fine tuning my brain to be a better instructor … And messed up as this sounds, maybe it’s a necessary step toward transforming how we approach learning in this new age of AI.
Because despite all the limitations, challenges and concerns around ChatGPT and other generative AI platforms (and there are many), these have the potential to transform lives and communities through learning at a scale that probably hasn’t been since the invention of the printing press.
And even with the potential challenges, that’s a thrilling prospect.