RESEARCH ARTICLE |
[Download PDF] |
|
| Year : 2016 | Volume
: 48
| Issue : 7 | Page : 47--51 |
Introducing objective structured practical examination as a method of learning and evaluation for undergraduate pharmacology
Kirti Vishwakarma1, Mukesh Sharma2, Prithpal Singh Matreja1, Vishal Prakash Giri1,
1 Department of Pharmacology, TMMC & RC, Moradabad, Uttar Pradesh, India
2 Department of Community Medicine, TMMC & RC, Moradabad, Uttar Pradesh, India
Correspondence Address:
Kirti Vishwakarma
Department of Pharmacology, TMMC & RC, Moradabad, Uttar Pradesh
India
Abstract
Objective: Assessment method can influence student learning. Use of objective structured practical examination (OSPE) has been reported in various institutes with great benefits. We evaluated OSPE for the assessment of practical skills in pharmacology examination for undergraduate medical students and compared it with conventional practical examination (CPE).
Materials and Methods: After sensitizing the 2nd year MBBS students to OSPE, the students were divided into four batches with twenty students in each batch. Students were assessed by attending five OSPE stations, each for duration of 5 min. The effectiveness was assessed through a student’s feedback questionnaire and was checked for its reliability by Cronbach’s alpha. The result of OSPE was compared with that of CPE of the same batch.
Results: Cronbach’s alpha of the feedback questionnaire was 0.71, with high internal consistency. The feedback given was categorized into three domains: cognitive, psychomotor, and affective, and an assessment was also done for its further use. In cognitive domain, 74% of the students felt that the questions asked and the syllabus taught were well correlated. In psychomotor domain, 81% agreed that it is excellent for assessing the applied part of the subject. Seventy percent of students opined that it was associated with lesser stress than CPE. On overall assessment, 76% rated this methodology as good/satisfactory and 23% as excellent in terms of better scoring. There was a significant difference in the mean score between the results of OSPE and CPE (P < 0.001, df = 158, confidence interval = 95%).
Conclusion: OSPE is a feasible and skill enhancing tool for the assessment in pharmacology examinations for undergraduate students.
Key message:
OSPE (Objective structured Practical Examination) is a feasible, meaningful and skill enhancing tool for assessment in pharmacology for undergraduate students. There are several benefits of OSPE as an assessment method including objectivity and uniformity in assessment of students.
How to cite this article:
Vishwakarma K, Sharma M, Matreja PS, Giri VP. Introducing objective structured practical examination as a method of learning and evaluation for undergraduate pharmacology.Indian J Pharmacol 2016;48:47-51
|
How to cite this URL:
Vishwakarma K, Sharma M, Matreja PS, Giri VP. Introducing objective structured practical examination as a method of learning and evaluation for undergraduate pharmacology. Indian J Pharmacol [serial online] 2016 [cited 2023 Mar 24 ];48:47-51
Available from: https://www.ijp-online.com/text.asp?2016/48/7/47/193317 |
Full Text
It has been observed that a single written examination in undergraduate medical curriculum does not fulfill all the aspects of assessment, such as assessing knowledge, skill, and attitude. Practical and clinical skill plays a considerable role in assessing the applied part of the subject.[1] Traditional methods for assessment of practical and clinical skills lack scope to assess the psychomotor, performance, as well as communication skills of the students. The marks awarded in such methods are usually based on the overall performance of the candidate and not on their individual skills.[2] The assessment of these skills leaves a lot to be desired using traditional methods. Moreover, this method is tedious and time-consuming. In addition, the scoring is affected based on the examiner’s variability for different students, thus decreasing its reliability and/or reproducibility.[3],[4]
In contrast, objective structured practical examination (OSPE) is a globally implemented and beneficial system that is utilized for assessing students in practical examination. It was derived from the term objective structured clinical examination (OSCE) which originated in 1975 and was later modified by Harden and Gleeson in 1979.[5],[6] It is used to assess competency based on objective testing through direct observation. It includes objectivity and uniformity in questions and marking of students.[7] It comprised several “stations,” in which examinees are expected to perform a variety of practical tasks within a specified period against criteria formulated to the practical skill, thus demonstrating competency of skills and/or attitudes. OSPE has been used to evaluate those areas most critical to perform by students, such as the ability to obtain/interpret data, solve a problem, teach, and communicate. Several studies have proved that OSPE is a valid and reliable assessment tool and decreases examiner bias in various medical courses. It is now an accepted tool for the assessment of practical skills in pre- and para-clinical subjects too.[2],[3],[4],[7],[8]
Use of OSPE in pharmacology has been evaluated in some institutes of India and certain advantages were evident. For example, authors reported that OSPE covered a wide area of knowledge and skills in a short time with minimum variability. Moreover, research shows that the type of assessment method adopted can influence student learning.[9],[10],[11] Considering these facts, we introduced OSPE as a method of learning and evaluation tool for assessment of practical skills in pharmacology examinations for undergraduate students. The method was compared with the conventional method of assessment that is commonly used in India.
Materials and Methods
The 2nd year MBBS students of 2013 batch of a tertiary care teaching hospital appearing for their third semester practical examinations were included in the study, after successfully completing the designated syllabus for the same and obtaining approval from the Institutional Ethics Committee. The students were sensitized and oriented toward OSPE beforehand by explaining the OSPE pattern and discussing sample questions during their practical classes. The examination was conducted by a trained faculty of the Department of Pharmacology. The OSPE questions were constructed and validated beforehand for each station along with the answer key and checklists as applicable.
A total of eighty students were assessed. They were divided into four batches with twenty different students in each batch. The assessment was carried out for 4 days with different batches per day. Each student attended five different OSPE stations. Of these five stations, three were response stations while two were procedure stations. This was decided based on time and workforce available to conduct the examination. Each station was designed so that the task could be completed in 5 min. The OSPE was planned for 25 marks. Viva voce that evaluated theoretical knowledge was allowed a weightage of 15 marks in the practical examination. The questions were based on dose calculation, drug label, routes of drug administration, pharmacy exercises, interpretation of data and graphical records from animal experiments, checking of various parameters and interpretation of results in rabbit eye experiment, therapeutic problems, adverse drug reactions, fixed dose combinations, etc. The effectiveness of OSPE was assessed through a student’s feedback questionnaire that was prepared after the literature search and validation by the faculty in pharmacology and medical education. The questions administered were checked for coefficient of reliability by Cronbach’s alpha.
The results of OSPE were compared with the results of the conventional practical examination (CPE) of the same batch during the 4th semester. The CPE in pharmacology consists of two components, total of 40 marks, including practical exercises of 25 marks and viva voce of 15 marks. Comparison of the results was based on 25 marks allotted for OSPE and practical exercises. The questions asked were in the form of short exercises related to the topics taught during the semester to different batches such as dose calculation, prescription writing, pharmacy exercise, drug labeling, simulation exercises, and management of diseases.
Results
The study feedback questionnaire was collected from eighty students after they completed the OSPE. The questions administered were checked for coefficient of reliability by Cronbach’s alpha, and it was found to be 0.71 with high internal consistency.
The feedback given by the students were categorized into three domains, i.e., cognitive, psychomotor, and affective, and an assessment was made for its further use [Table 1].{Table 1}
Majority of the students (74%) said that the pattern of questions asked was well correlated with the syllabus taught in the class. Majority (94%) of the students rated OSPE well (71% good/satisfactory, 23% excellent) and opined that instructions given at each station were clear and understandable. A total of 96% of students opined (66% good/satisfactory and 30% excellent) that the questions asked in the examination contributed to their learning and helped them improve their knowledge [Figure 1].{Figure 1}
Twenty-five percent of students said that this pattern of examination is excellent and 72.5% said that it is good or satisfactory in testing practical skills of the students. Eighty-one percent of students agreed that it is an excellent method for assessing the applied part of the subject, i.e., applying knowledge to actual situation [Figure 2]. Seventy percent of the students opined that this pattern of examination encouraged them and created interest for learning and minimized their stress level during examination [Figure 3].{Figure 2}{Figure 3}
Seventy-six percent (76%) of the students rated this methodology of practical examination as good/satisfactory and 23% as excellent in terms of obtaining better scores. Students (67.5%) felt that time allotted at each station was good/satisfactory whereas 27.5% felt it to be excellent. 72.5% of the students felt that it would be an excellent idea to introduce this OSPE for future assessments [Figure 4].{Figure 4}
Eighty out of 100 students in the 2nd year MBBS class appeared in both the examination patterns, i.e., OSPE in the first semester and CPE in the second semester. The mean scores out of 25 were 19.94 ± 6.94 and 14.48 ± 10.98 for OSPE and CPE, respectively. There was significant difference in the mean scores between OSPE and CPE (P < 0.001, df = 158, confidence interval = 95%) [Figure 5].{Figure 5}
Discussion
OSPE has introduced for the 2nd year MBBS students and was evaluated with the help of student’s feedback questionnaire and a comparison of performance of the students in the two evaluation methods. Student’s feedback for educational methodologies is an important and useful basis for modifying and improving medical education. The basic aim of this feedback is to assess the areas of strength and/or lacunae of teaching methodologies to rectify the shortcomings and revise the curriculum suitably.[12],[13]
In our study, about 79% of the students opined that OSPE assessed the syllabus that has been taught earlier in the practical classes. Majority of the students were able to understand and follow the instructions properly, due to prior sensitization to this novel method. Students’ perceptions with regard to improvement in their practical were positive as majority of the students perceived OSPE to be good or satisfactory, suggesting that it would be acceptable to a majority of students in the event that it replaced the CPE. Previous studies also revealed similar findings in this regard.[11],[14]
Around 81% of students opined that OSPE is an excellent tool for assessing the application of knowledge. This was in tandem with a study by Yaqinuddin et al., where 74% students agreed to multiple modes of assessment to improve their knowledge and skills in anatomy.[15] In another study by Chandelkar et al., it was found that all the students accepted OSPE because it helped them improve their practical skills and application in pharmacology.[16] Results of another study indicated better effect of objective structured assessment of technical skills on learning.[17]
Any examination is a well-known source of stress and anxiety and OSPEs in particular considered as quite stressful.[18] In contrast to this, around 80% of the students agreed that OSPE is an examination with minimum stress level in our study. This could be possibly due to least or indirect interaction with the examiners as well as students undergoing limited number of stations. This finding was however similar to a study by Wadde et al., where they had limited number of stations and no direct interaction with examiners, but the students there agreed that this type of examination may be exhausting and stressful if number of stations will be increased.[19] Low level of stress can make student more alert and motivated. However, high level of stress can cause several difficulties, including reduced ability to prepare for and perform during their exams.[20]
76.2% of the students felt OSPE to be a better method of evaluation for future examinations as well. Objectivity and more uniform evaluation are the documented advantages of this method.[16],[21]
In our study, only 5% of the students were not satisfied with the time allotted for OSPE. A similar opinion was noted in a study by Chandelkar et al., where only one student was not satisfied with the time allotted for OSPE. In another study by Ranjan et al., time management in OSPE was found better than traditional practical examination.[21] Communication skill is one of the most important skills of a physician for patient’s management. The OSCE has been used extensively to assess communication skills.[22] In our study, there was no OSPE station to evaluate communication skills. This was because of the limited time and workforce in the department. This limitation could be overcome in future studies by including clinical cases, where students may be asked to give verbal instructions regarding the use of various drugs or in the form of clinical pharmacological exercises.
OSPE when compared to CPE, a significant difference (P < 0.001) was found in the mean scores of the result showing better performance of the students [Figure 5]. This was in accordance to a study by Ranjan et al., where significant (P < 0.05) improvement was observed in the mean marks obtained in OSPE as compared to CPE.[21] In another study by Nigam et al., OSPE was found to be important in competency-based performance discrimination and also in improving students’ performance quality in laboratory exercises.[23] This study confirmed the feasibility and students’ acceptability of OSPE in evaluating the pharmacology skills in the undergraduate medical curriculum.
Conclusion
The current study shows several benefits of OSPE as an assessment method. Most of participants in the study were in favor of using this assessment method in future also. In comparison to CPE, OSPE shows better performance of the students as well. Thus, it can be concluded that use of OSPE is a relevant, meaningful, and feasible tool for the assessment of practical skills in undergraduate training in pharmacology.
Financial Support and Sponsorship
Nil.
Conflicts of Interest
There are no conflicts of interest.
References
| 1 | Elango S, Jutti RC, Kandasami P, Cheong Lieng CP, Loh LC, Motila T. Assessment of basic practical skills in an undergraduate medical curriculum. International e-Journal of Science, Medicine and Education (IeJSME) 2007;1:41-5. |
| 2 | Ananthakrishnan N. Objective structured clinical/practical examination (OSCE/OSPE). J Postgrad Med 1993;39:82-4. |
| 3 | Roy V, Tekur U, Prabhu S. A comparative study of two evaluation techniques in pharmacology practicals: Conventional practical examination versus objective structured practical examination. Indian J Pharmacol 2004;36:385-9. |
| 4 | Hasan S, Malik S, Hamad A, Khan H, Bilal M. Conventional/traditional practical examination (CPE/TDPE) versus objective structured practical evaluation (OSPE)/semi objective structured practical evaluation (SOSPE). Pak J Physiol 2009;5:58-64. |
| 5 | Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J 1975;1:447-51. |
| 6 | Harden RM, Gleeson FA. Assessment of clinical competencies using an objective structured clinical examination (OSCE). In: ASME Medical Education Booklet No. 8. Vol. 64. Dundee: ASME; 1979. p. 123-5. |
| 7 | Zayyan M. Objective structured clinical examination: The assessment of choice. Oman Med J 2011;26:219-22. |
| 8 | Nayar U, Malik SL, Bijlani RL. Objective structured practical examination: A new concept in assessment of laboratory exercises in preclinical sciences. Med Educ 1986;20:204-9. |
| 9 | Nayak V, Bairy KL, Adiga S, Shenoy S, Magazine BC, Amberkar M, Kumari KM. OSPE in pharmacology: Comparison with the conventional method and students’ perspective towards OSPE. Br Biomed Bull 2014;2:218-22. |
| 10 | Malhotra SD, Shah KN, Patel VJ. Objective structured practical examination as a tool for the formative assessment of practical skills of undergraduate students in pharmacology. J Educ Health Promot 2013;2:53. |
| 11 | Deshpande RP, Motghare VM, Padwal SL, Bhamare CG, Rathod SS, Pore RR. A review of objective structured practical examination (OSPE) in pharmacology at a rural medical college. Int J Basic Clin Pharmacol 2013;2:629-33. |
| 12 | Verhoeven BH, Hamers JG, Scherpbier AJ, Hoogenboom RJ, van der Vleuten CP. The effect on reliability of adding a separate written assessment component to an objective structured clinical examination. Med Educ 2000;34:525-9. |
| 13 | Rafique S, Rafique H. Students’ feedback on teaching and assessment at Nishtar Medical College, Multan. J Pak Med Assoc 2013;63:1205-9. |
| 14 | Manjula A, Shashikala P, Nagaraj P. Student’s perception on objective structured practical examination in pathology. J Med Educ Res 2013;1:12-4. |
| 15 | Yaqinuddin A, Zafar M, Ikram MF, Ganguly P. What is an objective structured practical examination in anatomy? Anat Sci Educ 2013;6:125-33. |
| 16 | Chandelkar UK, Rataboli PV, Samuel LJ, Kamat AS, Bandodkar LV. Objective structured practical examination: Our experience in pharmacology at Goa Medical College, Bambolim Goa, India. Int J Sci Rep 2015;1:113-7. |
| 17 | Mansoorian MR, Hosseiny MS, Khosravan S, Alami A, Alaviani M. Comparing the effects of objective structured assessment of technical skills (OSATS) and traditional method on learning of students. Nurs Midwifery Stud 2015;4:e27714. |
| 18 | Goud BK, Begum S, Zaki B, Haridas S. Perceptions and performance of undergraduate medical students in objective structured practical examinations (OSPE) in biochemistry at RAK Medical and Health Sciences University (RAKMHSU), UAE. J Univ Coll Med Sci 2014;2:54-61. |
| 19 | Wadde SK, Deshpande RH, Madole MB, Pathan FJ. Assessment of III MBBS students using OSPE/OSCE in community medicine: Teachers’ and students’ perceptions. Sch J App Med Sci S 2013;1:348-53. |
| 20 | Exam Stress. Student Learning Development, University of Leicester; 2009. Available from: http://www.le.ac.uk//offices/ld/resources/study-guides-pdfs/study-skills-pdfs/exam-stress-v0.1.pdf. [Last accessed on 2016 Sep 23]. |
| 21 | Ranjan R, Jain A, Rashmi Bhujade R. OSPE in anatomy: New dimensions in assessment. Int J Anat Res 2016;4:1789-94. |
| 22 | Baig LA, Violato C, Crutcher RA. Assessing clinical communication skills in physicians: Are the skills context specific or generalizable. BMC Med Educ 2009;9:22. |
| 23 | Nigam R, Mahawar P. Critical analysis of performance of MBBS students using OSPE & TDPE – A comparative study. Natl J Community Med 2011;2:322-4. |
|
