IPSIndian Journal of Pharmacology
Home  IPS  Feedback Subscribe Top cited articles Login 
Users Online : 10984 
Small font sizeDefault font sizeIncrease font size
Navigate Here
  Search
 
  
Resource Links
   Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
   Article in PDF (847 KB)
   Citation Manager
   Access Statistics
   Reader Comments
   Email Alert *
   Add to My List *
* Registration required (free)

 
In This Article
  Introduction
  Methods
  Results
  Discussion
  Conclusion
   References
   Article Figures

 Article Access Statistics
    Viewed678    
    Printed8    
    Emailed0    
    PDF Downloaded16    
    Comments [Add]    

Recommend this journal

 


 
 Table of Contents    
LETTER TO THE EDITOR
Year : 2023  |  Volume : 55  |  Issue : 4  |  Page : 266-267
 

Testing the knowledge and interpretation skills of ChatGPT in pharmacology examination of phase II MBBS


1 Department of Pharmacology, Medical Education, Research, Panimalar Medical College Hospital and Research Institute, Varadharajapuram, Poonamallee, Chennai, Tamil Nadu, India
2 Department of Biochemistry, Medical Education, Research, Panimalar Medical College Hospital and Research Institute, Varadharajapuram, Poonamallee, Chennai, Tamil Nadu, India

Date of Submission03-Apr-2023
Date of Decision13-Jul-2023
Date of Acceptance08-Aug-2023
Date of Web Publication11-Sep-2023

Correspondence Address:
Krishna Mohan Surapaneni
Vice Principal and Professor, Departments of Biochemistry, Medical Education, Research, Panimalar Medical College Hospital and Research Institute, Varadharajapuram, Poonamallee, Chennai - 600 123, Tamil Nadu
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/ijp.ijp_188_23

Rights and Permissions



How to cite this article:
Elango A, Kannan N, Anandan I, Surapaneni KM. Testing the knowledge and interpretation skills of ChatGPT in pharmacology examination of phase II MBBS. Indian J Pharmacol 2023;55:266-7

How to cite this URL:
Elango A, Kannan N, Anandan I, Surapaneni KM. Testing the knowledge and interpretation skills of ChatGPT in pharmacology examination of phase II MBBS. Indian J Pharmacol [serial online] 2023 [cited 2023 Oct 3];55:266-7. Available from: https://www.ijp-online.com/text.asp?2023/55/4/266/385494




Dear Editor,


  Introduction Top


ChatGPT has attracted millions of users worldwide being the most advanced and rapid information generating artificial intelligence (AI) tool that imitates human conversations. ChatGPT is developed by San Francisco-based open AI with the highly advanced generative pretrained transformer-4, a deep learning tool that was made available to the public on November 30, 2022.[1] ChatGPT is being tested in various domains, and one remarkable achievement is its clearance in the United States Medical Licensing Examination (USMLE), which is considered to be one of the most competitive and challenging medical tests.[2] Ever since this commendable performance, ChatGPT has become a topic of discussion worldwide in regard to its usability in medical education and ability of AI-based tools to give a new shape to teaching and learning methods in medical sciences.[2] Furthermore, there are rising concerns about the misuse of ChatGPT. To prevent such exploitations, we need a framework of regulations for governance.[3] Thus, to extend the available evidence about the performance of ChatGPT in medical sciences, we intended to test the knowledge and interpretation skills of ChatGPT in pharmacology examination of phase II MBBS.


  Methods Top


We evaluated the performance of ChatGPT using pharmacology question papers of the final university examination in the second professional year of undergraduate medical education which was conducted in the month of February 2023. The syllabus of pharmacology has been segregated into two parts. Part I comprises general pharmacology, autonomic nervous system, central nervous system, peripheral nervous system, autacoids, cardiovascular system, blood, respiratory system, and kidneys, whereas the gastrointestinal system, hormones, antimicrobials, chemotherapy, and miscellaneous topics come under part II. For this assessment, we used the final university examination question paper of part II. Each part is evaluated for 100 marks with 30 marks being allotted for 2 essays (15 marks each) and 50 marks for 10 short answers (5 marks for each). The remaining 20 marks are awarded for multiple-choice questions (MCQs). We tested both theoretical and MCQ question papers of part II. All the questions were typed exactly on ChatGPT, and answers were generated including the four options provided for MCQs. Regeneration of the responses was not performed. All the answers generated in the first attempt were saved and later evaluated by our faculties in the department of pharmacology. Two evaluators separately assessed the answer script to prevent any bias. The scores were later compared, and the average was taken as the final score.


  Results Top


ChatGPT generated appropriate and succinct responses to all the questions. No diagrammatic- or flowchart-based representation was provided for the mechanism of action. Uses and adverse effects mentioned in all questions were provided as points in a nutshell without much elaboration which was appreciated by the evaluators. The essays were given as case-based problems from antimicrobials chapter. The first essay was typhoid and the second was urinary tract infection with amenorrhea.

The diagnoses of both the essays were right, and answers were given in short paragraphs. The response generated by ChatGPT for case 1 is represented in [Figure 1]. For the MCQs, the option of choice along with the explanation for the same was generated for all 20 questions. Out of 20 MCQs, ChatGPT has provided the correct answer for 15 questions. Individual marks were given for each question and summed up at the last. ChatGPT has managed to obtain a distinction in pharmacology with the average total score of 76%. The overall performance of ChatGPT in pharmacology was appreciable and highly suggests the beneficial role that AI could play in transforming medical education.
Figure 1: Case interpretation by ChatGPT in pharmacology

Click here to view



  Discussion Top


Our evaluation has suggested that ChatGPT has the potential to emerge as one of the most useful tools in medical education with its ability to generate appropriate and precise information to standard well-defined inputs. In accordance with the findings of our evaluation, studies have indicated that ChatGPT could play a beneficial role in facilitating “self-directed” method of learning and help students improve their knowledge and reasoning skills. However, the information provided by ChatGPT has to be validated as it did not show the standard references, from which information has been retrieved.[4] Hence, it can be predicted that AI tools such as ChatGPT have enormous potential to significantly transform clinical pharmacology, drug information services, medical writing, and the entire field of medicine when used cautiously and under proper guidance.[5]


  Conclusion Top


AI-based large language models such as ChatGPT can be used as an effective learning tool owing to its open accessibility and high speed in generating information. However, educators and students cannot solely depend on ChatGPT for learning as there is always a possibility of misleading or incorrect information being generated by such AI tool. Thus, appropriate testing and validation is essential before successful implementation of AI in advancing medical education and research.

Acknowledgements

Authors would like to extend their gratitude to OpenAI, an American artificial intelligence research laboratory for providing free access to ChatGPT.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Introducing ChatGPT, OpenAI. Available from: https://openai.com/blog/chatgpt. [Last accessed on 2023 Mar 17].  Back to cited text no. 1
    
2.
Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health 2023;2:e0000198.  Back to cited text no. 2
    
3.
Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: Is ChatGPT a blessing or blight in disguise? Med Educ Online 2023;28:2181052.  Back to cited text no. 3
    
4.
Nisar S, Aslam MS. Is ChatGPT a Good Tool for T&CM Students in Studying Pharmacology? Available from: https://ssrn.com/abstract=4324310. [Last accessed on 2023Jan 14].  Back to cited text no. 4
    
5.
Krumborg JR, Mikkelsen N, Damkier P, Ennis ZN, Henriksen DP, Lillevang-Johansen M, et al. ChatGPT: First glance from a perspective of clinical pharmacology. Basic Clin Pharmacol Toxicol 2023;133:3-5. doi: 10.1111/bcpt.13879. Epub 2023 May 12. PMID: 37170853.  Back to cited text no. 5
    


    Figures

  [Figure 1]



 

Top
Print this article  Email this article
 

    

Site Map | Home | Contact Us | Feedback | Copyright and Disclaimer | Privacy Notice
Online since 20th July '04
Published by Wolters Kluwer - Medknow