AI Assessment three case studies

AI in Assessment - Three Case Studies 

Video recording of online session

AI generated summary: AI in Assessment — Three Case Studies

• 00:20–05:22 — Human teachers must remain the masters of AI. The inaugural remarks caution that, just as language labs and ICT tools were once seen as solutions to classroom problems, AI should not be treated as a panacea. Teachers should evaluate its usefulness critically and adapt it to Indian classroom contexts.
• 20:51–30:45 — The central problem: assessment workload and AI-era assignments. The speaker argues that conventional written assignments have lost some assessment value because students can readily use generative AI. This creates a need to move from merely assessing written products toward performance-based assessment and digital portfolios, while using AI to manage the resulting workload.
• 31:08–42:40 — Pre-session survey establishes teachers' assessment challenges. Among 103 respondents, MCQs were the most common assessment method (77%), followed by descriptive essays and presentations. Teachers identified lengthy answers, meaningful feedback, identifying strengths/weaknesses, comparing performances, and the time required for assessment as major difficulties. Most participants were comfortable with AI assisting in preliminary evaluation of descriptive answers.
• 43:04–54:55 — The key conceptual shift: AI as assessment assistant, not assessor. Assessment involves reading, interpreting, applying criteria, judging, scoring, and providing feedback—not merely assigning marks. The proposed model is Student Work → AI-Assisted Analysis → Rubric-Based Evidence/Patterns → Teacher Validation → Final Judgment & Feedback. The speaker emphasizes that the rubric is the backbone: the better the assessment design and criteria, the more useful AI assistance becomes.
• 55:21–1:09:40 — Case Study 1: AI-assisted assessment of essay answers. Handwritten literature answers were evaluated using teacher-designed prompts, the BAWE corpus, and CEFR guidelines. AI identified strengths and weaknesses in textual understanding, interpretation, argumentation, evidence, academic language, organization, and critical analysis. Students could then ask AI to show how their own answer might be improved rather than simply receiving an ideal model answer. The teacher remained responsible for validating the AI's evaluation and final score.
• 1:10:11–1:29:25 — Case Study 2: Video essays and performance assessment. Students produced video essays, presentations, literary performances, and short videos. AI-assisted analysis helped examine content, argument, language, delivery, visuals, creativity, coherence, and critical thinking. Tools such as Gemini, Adobe Enhance, and NotebookLM were explored. An important unexpected outcome was that students sometimes challenged AI feedback, encouraging critical engagement with assessment itself. The speaker concludes that AI can observe evidence, but humans must interpret its significance, especially regarding creativity, authenticity, cultural context, and nuance.
• 1:29:25–1:35:47 — Case Study 3: AI-assisted digital portfolio assessment. Portfolios containing essays, videos, presentations, reflections, research, projects, blogs, and creative work were analysed for evidence, reflection, growth, and competency development. AI helped identify patterns in students' learning journeys and possible future competencies/trajectories, while the teacher provided holistic judgment.
• 1:35:47–1:39:44 — The major pedagogical shift: from assessing answers to assessing learning. The teacher's role changes from marker → assessment designer → rubric designer → evidence interpreter → feedback designer → AI validator → ethical decision-maker. The proposed AI Assessment Triangle has three essential components: Assessment Design + Rubric + AI Assistance, resting on Human Judgment. If any component is weak, assessment quality suffers.
• 1:39:44–1:55:51 — Q&A reinforces validity, reliability, bias and human oversight. Questions address free/subscription tools, AI bias, validity and reliability, rubric constraints, language learning, and accessibility. The speaker stresses that AI-generated scores should never be accepted blindly and that anonymizing students as “Student 1, Student 2…” may help reduce potential name-, gender-, or caste-associated bias. AI should be AI-assisted, not AI-determined.

Core takeaway

The session's central message can be reduced to one principle:

The future of assessment is not “Human vs AI” but “Human + AI.”

AI can automate reading at scale, pattern identification, rubric application, evidence extraction, comparison, summarization, and preliminary feedback. The teacher must retain responsibility for context, interpretation, fairness, validity, ethics, final judgment, and academic decisions. This aligns closely with recent Glasp discussions emphasizing that AI can generate signals and support evaluation, but human judgment remains essential for deciding what those signals mean and what action should follow.

The most important five questions before using AI for assessment are:

1. What exactly am I assessing?
2. What evidence demonstrates it?
3. What rubric will I use?
4. What can AI reliably assist with?
5. What must remain with human judgment?

This framework also resonates with current thinking on AI evaluation: criteria should be observable, AI judgments should be validated against human judgment, and there should always be a mechanism for human disagreement and override. 

AI-Assisted Self-Assessment of Essay Type Descriptive Answers

 

[Video recording of the live session]

Transcript for the "Essay Type Answer Writing | Error Analysis & AI-assisted Self-assessment | Academic Writing Workshop" by Merlin and organised by Chat GPT

Good afternoon, everyone, and welcome to another installment of our academic writing workshop series. These workshops have become a tradition, occurring at the close of each semester following our internal tests. It's a time when teachers meticulously review your answer scripts, pinpointing common errors for discussion. But beyond this, it's a chance for you to engage in a crucial aspect of academic growth: self-assessment.

Self-assessment involves more than just glancing over your work; it requires a deep dive into your writing, identifying both obvious mistakes and subtler nuances that often escape notice. Consider creating PDFs of your answer books, allowing for ongoing reflection and improvement, whether you're at home or in the hostel. While AI tools can assist in this process, it's essential to remember that they're just that—tools. Our ultimate goal is to maintain and enhance our own writing abilities, not to become overly reliant on technology.

As we transition into discussing the role of AI in our workshops, it's important to strike a balance. While AI offers valuable insights and can even serve as a personalized tutor, it's not without its limitations. Over-reliance on AI runs the risk of diminishing our own linguistic capabilities. Our workshops aim to harness the benefits of AI while ensuring that human judgment and creativity remain at the forefront.

In analyzing sample answers, we aim to bridge the gap between proficiency and excellence. By leveraging both AI insights and human discernment, we can identify areas for improvement and chart a course towards advanced levels of writing proficiency. Practical exercises are integral to this process, allowing you to apply feedback and refine your approach iteratively.

As we conclude today's workshop, I encourage you to continue honing your writing skills beyond these sessions. The journey towards academic excellence is ongoing, and your commitment to self-assessment and improvement will undoubtedly pay dividends in the long run. Thank you for your participation, and don't forget to complete the assigned activities.

The highlights of the session:

Mastering Essay Writing: Error Analysis & AI-powered Self-Assessment

This blog post summarizes an academic writing workshop focused on improving essay writing through error analysis and AI-assisted self-assessment.

Introduction

The workshop addresses a common challenge: the gap between internal assessments (where improvement is possible) and final exams (where it's not). It emphasizes the importance of strong writing skills and explores how AI tools can enhance self-assessment without diminishing human writing ability.

Challenges of Traditional Error Analysis

Personalized feedback can be time-consuming for teachers.

Replicating student answers with high-quality variations is difficult for humans.

AI as a Solution

Generative AI offers one-on-one tutoring through feedback suggestions.

AI excels at analyzing student writing and suggesting improvements.

Cautions and Best Practices

Overreliance on AI can hinder independent writing development.

Critical thinking skills remain essential, and AI should not replace them.

University exams typically require writing without AI assistance.

Benefits of AI-assisted Self-Assessment

Identifies areas for improvement (e.g., grammar, mechanics, clarity).

Elevates writing quality from B1/B2 to C1/C2 levels (CEFR framework).

Provides suggestions for stronger introductions, conclusions, and stances.

Workshop Activity

1. Analyze Past Errors: Review feedback from teachers on past exams.
2. Select an Answer: Choose an answer you wrote for a previous internal assessment.
3. AI Feedback: Upload a photocopy/PDF of your answer to a designated AI tool.
4. Prompt Selection: Clearly state you are a postgraduate student seeking a high academic level response.
5. Self-Assessment: Analyze the AI's suggestions and compare them to your original writing.
6. Continuous Improvement: Practice writing and self-assessment throughout the semester, aiming for excellence without AI dependence.

Conclusion

This workshop equips students with the tools and strategies to become self-sufficient, confident essay writers. By combining traditional error analysis with AI-powered feedback, students can elevate their writing skills and achieve academic success.