Validating a problem-based learning process assessment tool in a Nepalese medical school

Abstract

Introductions: The newly established Patan Academy of Health Sciences (PAHS) has incorporated the measurement of non-cognitive skills and behaviors into the summative assessment in the setting of problem based learning (PBL). This study was conducted to validate a PBL process assessment tool for PAHS.

Methods: A list of 72 items of student behaviors observable in PBL tutorials was compiled from literature review. They were categorized under ten broad dimensions consistent with predefined PAHS Graduate Attributes. A series of PBL project committee meetings and expert inputs refined the list of 72 items to 47 and categorized them under eight dimensions. These 47 items, each with a 4-point rating scale, formed the Tutor Assessment of Student Tool (TAS-Tool). Twenty-four trained faculty members used the TAS-Tool to evaluate the performance of 41 senior high school students in PBL tutorials.

Results: The internal-consistency of the TAS-Tool was very high (Cronbach’s α = 0.954). Removal of two inconsistent items further increased it to 0.975. Principal components analysis with varimax rotation applied to the remaining 45 items gave seven components and explained 69.47% of the variation between the components. These seven components (% variation) were: Immersed in the Tutorial Process (20.16%); Professional (12.71%); Communicator and Team Leader (11.25%); Critical Thinker (8.77%); Reflector (6.22%); Creative (5.95%), and Sensitive (4.41%).

Conclusions: TAS-Tool was found to be reliable and valid instrument deemed applicable in formative PBL process assessment at PAHS starting with the pioneer cohort of medical students. Further validation of TAS-Tool through longitudinal study with PAHS students is required for summative purpose.