Creating a Sample Question Paper Using Clinical Vignettes and a Large Language Model
Introduction:
With the increasing accessibility of AI, it has become incredibly convenient, as it is now available at our fingertips. AI has the potential to enhance our skills and alleviate the cumbersome task of searching for the right sources. One field where AI has particularly proven its effectiveness is medical education. For instance, recent news highlights the introduction of the NEXT exam for postgraduate entrance in colleges like AIIMS, where teaching methods have shifted from traditional book-based approaches to more practical teaching. This change has also provided an opportunity to assess students using integrated, practical-based questions during university exams.
With the advent of AI Creating a such a question paper using clinical vignettes can be an effective way to assess students' understanding and application of knowledge in a medicine.Utilising a large language model (LLM), such as OpenAI's GPT-3, we can generate a wide range of questions, long essays, and short essays, MCQs you name it ,based on student online learning portfolio clinical vignettes can be made .
This write-up aims to guide you through the process of accessing the LLM online, selecting appropriate clinical vignettes, creating prompts for questions and answer keys, and verifying them with supervisors or professors.
Step 1: Accessing the Large Language Model (LLM):
1. Visit the OpenAI website or any other platform offering access to large language models.
Here’s the link
2. Sign up for an account or log in if you already have one.
Here’s the link to chatbot on telegram
requires telegram app ,which you download from
Play store,AppStore
3.Some AI model like mid journey would require a payment based premium subscription, obtaining an API key.Since our job only requires text based AI model ,we will not be requiring any of it .
Step 2: Selecting Clinical Vignettes:
1. Access online learning portfolios, case reports, or medical journals that provide detailed clinical vignettes.
Here’s a link to our general medicine department blogspot
https://medicinedepartment.blogspot.com/2022/02/?m=0
2. Choose vignettes that cover a wide range of medical topics and scenarios, ensuring they are relevant to the subject being assessed.
3. Ensure the vignettes are concise, with well-documented history , clinical examination ,because our Job is a cake walk after copy pasting the history into the LLM , this way a good history will make a more refined clinical vignette .
Step 3: Creating Prompts for Questions:
1. Read each clinical vignette carefully and identify key concepts or knowledge to be assessed
2. For a perfect , specific clinical vignettes , you need to be specific while asking the LLM to make a clinical vignettes out of the history you have provided
3. Through my Experience best prompt would be example :“ make a clinical vignettes from this history ( copy paste the history before typing in this ) On alcoholic liver disease , don’t mention ALD anywhere in the question ,it’s for the examinee to indentify, keep only positive history clueing towards ALD , remove all negative , non specific history “
4. If the clinical vignette is too long , or too short , u can always prompt :”to add more detail to the vignettes “ for a shorter vignettes :”vignette is too long , make it short keeping only specific positive history “
2. Once a vignette is to your liking you , Determine the type of question you wish to create (MCQ, long essay, short essay) based on the learning objectives.
3. Utilize the LLM to generate questions by providing prompts example : “ make one LAQ 10 marks ,one SAQ 5 marks and one MCQ 5 marks NEET pg level questions on ALD f, do not mention ALD anywhere in the question ,For MCQs, include multiple plausible answer choices.
4. Incase level of toughness is not upto to standard , prompt example : “ increase the level of toughness , question can be around anatomy , physiology , pathology , medicine around ALD indirectly without mentioning in ALD in question “.
Step 4: Generating Answer Keys:
1. Once the questions are generated, use the LLM to create answer keys.
2. Ensure the answer keys are accurate and comprehensive, covering all relevant aspects of the clinical vignette.
Sometimes multiple choices can be correct , careful screening is required .
prompt :”example :
45-year-old male presents with severe epigastric pain radiating to the back, accompanied by vomiting and weight loss. On examination, tenderness and guarding are noted in the upper abdomen. Serum amylase and lipase levels are elevated, and abdominal ultrasound reveals inflammation in the retroperitoneal area. Which of the following mechanisms is most likely involved in the pathogenesis of the patient's condition?
A) Increased activation of pancreatic proteolytic enzymes
B) Impaired insulin production and secretion
C) Autoimmune destruction of pancreatic acinar cells
D) Defective gallbladder contractility
E) Obstruction of the pancreatic ducts by gallstones
A ,C,E can be causes of pancreatitis
“Prompt :” option A ,C, E are all correct , change the multiple choices .
3. This way Review and refine the answer keys to eliminate any ambiguities or errors.
Step 5: Verification with Supervisors or Professors:
1. Share the sample question paper, including the clinical vignettes, questions, and answer keys, with supervisors, facilitators, or professors for verification.
2. Seek their input and feedback regarding the appropriateness of questions, clarity of prompts, and accuracy of answer keys.
3. Revise and update the question paper based on the feedback received.
Comments
Post a Comment