An Evaluation on Dental Students’ Performance during the First Year of Covid-19 Pandemic

As with other university programs, dental school has been affected during the Covid-19 pandemic due to physical distancing and clinical restriction. Covid-19 pandemic changes the original method of study with the swift shift to online learning, which may affect students’ performance. The objective of this study was to assess the effect of online learning methods on the final examination performance for dental undergraduate students in their clinical years in a public university in Malaysia. One-way analysis of variance (ANOVA) and post-hoc Tukey tests were conducted to compare the effect of learning methods on the mean examination score, by comparing clinical students from the year of 2018 and 2019 face-to-face (f2f) learning method as compared to fully online learning in 2020. The analysis shows that there is a statistically significant difference on final exam performance at p < 0.05 for Year 3 [F (2, 102) = 11.68] and Year 5 [F(2, 95) = 22.32]. Posthoc Tukey HSD test indicates that the mean examination result for 2019 is significantly different from 2020 and 2018 even though the same learning method was employed for the latter. There was also no statistically significant difference for when the 2020 cohort is compared to the 2018 cohort for both Year 3 and Year 5. There was no statistically significant difference in mean examination score across all cohorts for Year 4 students. Although limited in scope, the conclusion of this study An Evaluation on Dental Students’ Performance during the First Year of Covid-19 Pandemic 85 was that, f2f learning method has no significant difference with online learning in terms of final examination performance.


Introduction
In 2015, Bill Gates gave a TED talk entitled 'The next outbreak? We're not ready' about the trend of past pandemics and how the world should be prepared to face the next one that can kill millions of people worldwide (TED, 2015). True enough, Covid-19 pandemic hit the world, necessitating a lockdown which put everything to a sudden halt. Billions of people are affected, the economy suffers, jobs are lost, and healthcare and frontline workers are overworked while educational institutions scramble to find an alternative teaching and learning methods. We are not prepared, and Covid-19 outbreak is changing the landscape of education.
Dental professionals are exposed to numerous occupational hazards with the risk of blood-borne and air-borne infections such as human immunodeficiency virus (HIV), hepatitis and MERS-related coronavirus, to name a few, and we brave the storm adapting our practices to the outbreak. In trying to fulfil clinical requirements, students are also exposed to the aerosol-generating dental procedures. Dental treatment is an aerosol-generating procedure in which the high-speed handpiece used during dental procedure is rotating up to 200,000 revolutions per minute (rpm) while spewing droplets of saliva and blood at high speed from patients' airways. These small droplets, that remain suspended in air over the treatment area for a few hours before settling on surfaces, are hazardous for cross infection (Luzzi, 2021). This hazard is amplified due to the proximity of operators to patients with high aerosol exposure during treatment procedure (Ayatollahi et al., 2012).

Teaching and Learning Pre-Pandemic
Learning is a process of externalization and internalization where a person's knowledge was integrated with a new one to create deeper understanding (Cress & Kimmerle, 2008). In dentistry, this process is accomplished through theoretical classes, practical and clinical procedures. Dentistry is a professional programme and accredited by the Malaysian Qualifications Agency (MQA) and Malaysian Dental Council (MDC), a professional body to ensure the quality of Malaysian dental graduates is at par internationally.
Dental schools have been using the same methods of teaching and learning for years, and with the advancement of technology, blended learning has been incorporated recently. Blended learning is a combination of conventional classroom teaching and digital content in an online platform. Dental students are expected to master the art of combining theoretical knowledge with interpersonal skills, soft skills and hands-on technical skills through clinical and laboratory sessions (Gonzalez et al., 2013). In the Faculty of Dentistry, Universiti Sains Islam Malaysia (USIM), students enrolled in prosthodontics course from 2018 and 2019 were subjected to blended learning consisting of faceto-face (f2f) classes with support from library and online resources. Prosthodontics is a highly challenging subject in dentistry due to the high level of skills required to perform treatment gained through complex laboratory works, treatment planning and patient management (Carlsson & Omar, 2006). The course centres on fixed and removable prosthodontics and the lessons include theory and skill training which are taught in physical classrooms and hands-on practical sessions.
Traditionally, the classes consisted of lectures, tutorials, seminars, problem-based-learning and small group discussions. Clinical skills were developed during a pre-clinical hands-on practical using mannequin before a three-year clinical training under supervision. In the clinic, students were trained to be a competent dental professional by conducting clinical procedures and patient management. Students were assessed in the form of summative and formative assessments. Excellence in clinical skills and effective communication were of equal importance in the measure of a competent clinician. The clinical competency was assessed through a series of competency tests and clinical logbooks to measure psychomotor and affective aspects with the addition of theoretical examinations including written exams, Objective Structured Clinical Examination (OSCE) and written assignments.

Teaching and Learning during the Present Covid-19 Pandemic
In the wake of the pandemic, major changes were made to the conventional classroom teaching and learning although the same topics, presentations and demonstrations are used. All theoretical classes and discussions have been shifted online while clinics, hands-on practical sessions and patientrelated procedures were done physically with strict infection control measures. Prosthodontics f2f classes were fully converted into online learning commonly using a synchronized Microsoft Teams or Zoom meeting allowing two-way communication between lecturers and students. Seminars and discussion groups are now being conducted using Web 2.0 and among the applications used are Microsoft Teams, Kahoot, Padlet and USIM GOALS online learning system. The students for each cohort are given the same resources plus updated information according to the current evidencebased knowledge. The difference in resources is that the 2020 students' cohort are provided with recorded lectures that can be accessed at any moment to cater to the students' own learning time, at their own pace. All references and lecture notes are readily available for students via USIM GOALS for 2018 and 2019 with the addition of recorded lectures available on Microsoft Teams for 2020.
Clinical session hours are reduced to minimized contacts and cross-infection, and treatment is carried out by following a strict standard precaution according to the Ministry of Health's guidelines. The aim is to attain the minimum and expected clinical experience mainly for the final year students who will be graduating soon. An online OSCE examination is conducted using a combination of Zoom application for questions projected by lecturers. Students are provided with QR codes that bring them directly to Microsoft Form and Flipgrid for the uploading of written and video recordings of their answers respectively.
The assessment that was done for each clinical year remains the same as pre-pandemic. Final examination score is the sum of final written examination score and continuous summative assessments score as shown in Table 1.

Type of Assessments
Face-to-face (2018) Face-to-face (2019) Online Learning (2020) Year 3 Changes are made to the course outline to ensure an optimum balance of fairness of the situation with a minimum expected clinical requirement that ensures competency for the student to move to the next year and graduating. Standard operating procedures (SOP) are implemented and strict personal protective equipment (PPE) becomes the protocol for the protection of clinicians. Questions arise when SOP requires physical distancing in the practical sessions and all f2f classes were shifted to an online learning: what are the effects of this shift to the students? The objective of this study is to assess the effect of pre-pandemic vs present learning methods on the examination performance among undergraduate dental students for three different cohorts.

Methodology
This study is a retrospective cohort study that study the clinical year student examination performance in the Faculty of Dentistry, Universiti Sains Islam Malaysia (USIM).
The Approval for data usage was obtained from the faculty and the removal of identifiers was done to maintain anonymity of the subjects.
All data was analysed using Statistical Package for Social Sciences (SPSS 21). A one-way analysis of variance (ANOVA) was performed to assess the impact of learning methods on academic performance through assessment of final examination score. This was done by comparing the mean scores of the final examination score from the year 2018 to 2020 students' cohort. The independent variable that was being tested is the learning method against a dependent variable that was the final examination score. The ANOVA was followed by a post-hoc Tukey test to determine where the differences in means existed.

Overall Performance
Overall performance for each cohort is presented in

One-Way ANOVA and Post-Hoc Analysis
Descriptive statistics that summarize the impact of learning methods on the final examination test score are presented in Table 3. The variances were assumed to be equal based on Levene's test. The highest mean recorded was from  Table 3 shows that learning methods have a statistically significant impact on exam performance at the level of p < 0.05 for Year 3, F(2, 102) = 11.68, p = 0.000 and Year 5 F(2, 95) = 22.32 p = 0.000 at the level of p < 0.05. There is, however, no statistically significant effect across Year 4 students. The post-hoc Tukey tests (Table 4) were done to identify the differences between each statistically significant group. The comparison indicated that Year 3 2019 cohort is statistically different from 2018 (MD = 7.59; CI = 3.76, 11.42; p-value = 0.000) and 2020 (MD = 5.31; CI = 1.47, 9.14; pvalue = 0.004) in the students population at the 0.05 level. For Year 5 student population, the test indicated that Year 5 2019 cohort is significantly different from 2018 (MD = 5.01; CI = 2.47, 7.55; p-value = 0.000) and 2020 (MD = 6.54; CI = 4.11, 8.96; pvalue = 0.000) in the students population at the 0.05 level. Even though the result posed a significant value for Year 3 and Year 5, the actual difference in score was quite minimal such as when Cohen's d effect size was calculated, the value was small at 0.2 for Year 3 and 0.3 for Year 5.

Discussion
This study sets out to assess the effect of learning methods on the final examination performance. The analysis showed a similar trend for Year 3 and Year 5 cohorts where there was an increase for the final examination mean score from 2018 to 2019, followed by a reduction in overall assessment in 2020. The trend, however, was in contrast to the Year 4 cohort which started as a decrease in examination mean score from 2018 to 2019 and followed by an increased mean score in 2020.
The cause of this trend, looking specifically at the increase in final examination mean score i.e. the major change that occurred during the start of the year for Year 3 and Year 5 students, was the shuffling of course coordinators which caused a new output on the programme although the curriculum remained the same. This increase was then followed by a decrease in mean examination score where the major contributor was the change in the course structure during the beginning of the pandemic in 2020. On the structure of the course itself, a huge percentage for Year 3 and Year 5 final examination scores was based on clinical logbook and final examination. These cohorts were affected the most during the pandemic due to the major reduction of clinical sessions which affect the overall continuous assessment. In comparison, even though the clinical session for Year 4 students were reduced, a huge percentage of the clinical portion of the clinical assessment was the practical simulation on mannequin which still went on despite the pandemic restriction in the clinic. This put the Year 4 students as the least impacted cohort during the pandemic. Based on the result that follows each batch of students, it can also be suggested that there is the batch effect where the same cohort of students produced similar examination scores, case in point are the batch of Year 3 2018, Year 4 2019 and Year 5 2020 (Table 3). However, further study is needed to understand the effect of students' cohort-examination performance. Dentistry is a hands-on course where teaching and learning happen not only in theory but also in practical sessions, and the theoretical examination was based on both. Therefore, the reduction of clinical sessions and changes to the physical classes will have an impact on final examination score.
The results for the post-hoc Tukey tests revealed that there is a statistically significant difference in the observed mean examination score for 2018 -2019 and 2019 -2020 for Year 3 and Year 5 cohorts. This analysis shows that both learning methods are equal and did not significantly affect students' performance since the same face-to-face learning method was employed for the 2018 -2019 cohort. This finding supports a previous study done by Nguyen (2015) on the effectiveness of online learning that stated that there is, in general, a similar outcome in the effectiveness of online learning in comparison to a face-to-face classroom. However, because of the sudden change from face-to-face to online learning, although online courses are not inferior to conventional class, in a particularly low socio-economic student population it may cause them to fall back behind their peers (Jaggars & Bailey, 2010). This is important due to a number of the students being from the low household income or the bottom 40% (B40) background (USIM, 2018).
The Covid-19 pandemic will continue its course for a long time before herd immunity is achieved and experts are projecting that social distancing and safety measures will have to continue into 2022 (Kissler et al., 2020). Issues that could affect the effectiveness of distant online learning as stated by the students themselves are the availability and quality of device, the state of internet connection and distraction at home. Both lecturers and students require an adjustment period to prepare for the shift to online learning, however, we do not have the luxury of time. The building of effective online learning will take up to a year and this process includes the development of infrastructure and student-staff preparedness. A carefully curated online learning course is different from online learning that started as a temporary solution to a crisis. It has been suggested that the effectiveness of online learning could be increased after two or three cycles with further situational adaptations of students, staff and infrastructure (Hodges et al., 2020).
The biggest dilemma faced by the dental school is to ensure the future graduating clinicians are competent in all areas especially with the reduction and restriction in the clinical portion of the curriculum. With the support of the faculty management and dental community that has actively provided solutions to critical situations, we are positive that this dilemma can be solved like a wellirrigated root canal.
Limitation of this study was the analysis itself which was based on the objective to look at students' performance based on the learning methods only. Future study should investigate more aspects such as performance differences based on gender, study time, possible distractions, batches effect and also the curriculum in preclinical and clinical years.

Conclusion
In conclusion, there was no significant difference in mean examination score when learning methods were compared. In dental education where a high percentage of the assessments is based on the overall performance throughout the year, the students' performance is influenced by many factors, and these factors should be included in future studies.