As part of my teaching assignment, I am responsible for teaching a course on Education Law, a three-credit elective within the Legal Studies program. The course primarily focuses on constitutional law issues for students and educators, with some statutory focus on disability law for education and employment law. I have taught this course over the last five years as an online, asynchronous class for about eighty students combined.
The design of the course included low stakes reading and homework assignments based on the publisher textbook, along with a series of pre-recorded lectures. Students were also invited to participate in various discussion boards on topics relevant to the materials. Assessment was completed through three non-cumulative exams on the chapters assigned during each unit of the course. Students also had a research paper assignment that filled out the course work for the class. The course is cross-listed and advertised to both Legal Studies and Education students and has included Baltimore County Public Schools faculty and administrators seeking continuing education credits.
In 2025, I made some design changes with the assistance of AI tools I developed for improving instruction. First, for each lecture, I created a transcript, skeletal outline, and a short quiz to test student knowledge of the content presented during the lecture using ChatGPT and AssemblyAI. Second, I organized the course content into modules that were focused on a single course objective and provided students with a checklist in the LMS with an explanation of the course work associated with that course objective. Third, I integrated an Exit Ticket into each module, which included a way for students to indicate if they were still having problems with a concept from the module or had any questions about the course content. I utilized ChatGPT to summarize this feedback, which I then used to create supplemental lectures for materials in unit 1 and unit 2 of the course.
I subsequently examined student performance in the course this summer (n=22 “treatment cohort”) with prior enrollment from 2020 through 2024 (n=57 “control cohort”) and found both a higher overall success rate (students earning an ABCD), and a higher overall pass rate on the three unit exams for the 2025 section compared with the prior sections. In particular, the treatment cohort had a 91% success rate overall in the course, compared with a 58% success rate for the control cohort.
Table 1 – Grade Distribution of Control and Treatment Groups
|
Control |
Treatment |
|
|
A |
7 |
7 |
|
B |
18 |
4 |
|
C |
5 |
6 |
|
D |
3 |
3 |
|
F |
20 |
1 |
|
W |
4 |
1 |
The Chi-square test of independence statistic is 15.53 with a p-value of 0.0083, suggesting a significant association between grade distribution between the control and treatment groups. Cramer’s V was calculated as 0.443, suggesting a strong-to-moderate association between the group and grade distribution.
In addition, students in the treatment cohort had a higher pass rate (60% or higher) compared to the control cohort on each of the three unit exams (81% compared to 50% on the unit 1 exam; 81% compared to 63% on the unit 2 exam; 90% compared to 65% on the unit 3 exam); only students that took an exam were included in each of these comparisons.
Table 2 – Exam Success Rates and Analysis
|
U1 Exam |
U2 Exam |
U3 Exam |
|
|
Control Success Rate |
63% |
50% |
65% |
|
Treatment Success Rate |
81% |
81% |
90% |
|
control n |
54 |
54 |
54 |
|
treatment n |
21 |
21 |
21 |
The Chi-square test of independence statistic for the unit 2 and unit 3 exams suggests a significant association between exam pass rates between the control and treatment groups. Cohen’s h suggests that this was a large effect for the differences in pass rates on these two exams between control and treatment groups.
My working theory is that the modular design combined with supplemental materials helped some students with success in the course, and also that providing additional feedback that was responsive to student comments during the course helped increase student engagement and motivation to complete the course.
There is some evidence in the literature that reports these types of interventions can be positively impactful on student performance and retention. McCormack discusses the positive impact of modularized course designs on student performance (McCormack, 2021). Schettig discusses the use of active learning activities paired with traditional lecture increased student success rates in college courses (Schettig, 2023). Goyal discusses that responding to student feedback as part of the course design also can positively impact student satisfaction and perceived learning by students (Goyal, 2023).
AI itself is not primarily being used to accomplish these goals, though using it to speed up the development of supporting materials and activities frees a faculty member to do the part of the work that may best explain the results of this study on student learning and engagement.
As this is only an observational study, however, other factors may explain the result such as differences in student characteristics, prior GPA, overall student motivation, etc. Further follow-up is warranted to confirm the positive results reported.