Application of the “Plan-Do-Check-Action” plan in improving the pass rate of the “National Medical Licensing Examination” | BMC Medical Education

Information Collection
All the examinees who took the NMLE in our hospital from 2021 to 2022 were taken as the research subjects, and they were divided into an observation group and a control group according to whether they accepted the PDCA plan. The participants in the observation group accepted the PDCA plan and took the NMLE in 2022, while the participants in the control group did not accept the PDCA plan and took the NMLE in 2021. Between the two groups, there was no statistically significant variation in the age or gender distribution. Detailed information about the performance of students, and the attendance, assessment, evaluation and suggestions from our hospital were provided by the relevant departments, and the pass rate-associated data was collected online.
The PDCA Plan
We implemented the PDCA strategy, which had four stages, including Plan, Do, Check, and Action (Fig. 1), to improve the pass rate of NMLE in our hospital in 2022. The detailed operating procedures are exhibited as below.

Plan
A quality improvement team (QIT) composed of the medical education and hospital departments was set up. 18 participants were joined in the QIT with the age of 29–55 years old, and half of them came from the department of medical education. The work of the QIT in the “Plan” stage included: (1) Established the theme: improving the pass rate of NMLE in our hospital. (2) Analyzed the current situation. According to the different origins of personnel, the examinees in NMLE were divided into five categories, including the off-campus resident physicians who graduated from other medical schools, the clinical masters who would graduate with four certificates including the certificate of standardized resident training, the “5 + 3” students who just finished the 5 years of bachelor of science degree training in clinical medicine and would continuously receive 3 years of standardized resident training in our hospital, the doctors at sixth-year of an “eight-year doctorate in clinical medicine” or the PhD newly entering our hospital to work and the others. To dig out the factors that contributed to the poor performance in 2021, we organized conferences for faculty and students and questionnaire research to gain a comprehensive and in-depth understanding of the problems of various categories of personnel in the NMLE, and we also analyzed the data through “Plato analysis” (Fig. 2) and “Brainstorming method” (Fig. 3). (3) Identified the objectives: focusing on talent development, clinical competency, and improving the pass rate of the NMLE. (4) Formulated countermeasures and tailored training plans. (5) Developed the schedule of activities and arrange theoretical studies and training assessments by using the method of “Gantt chart” (Fig. 4). Subsequently, the QIT analyzed the results of the examinees with an emphasis on the pass rate of the NMLE before making further plans. Constant adjustments in this step were taken according the monthly QIT meeting until a complete and practical solution was determined.

The “Plato analysis” of the poor performance in 2021. The Plato analysis showed that the off-campus residents, the clinical masters (excluding “5 + 3” students), and the doctors occupied a major proportion

The “fishbone diagram” of the poor performance in 2021. The brainstorming method was utilized to analyze the factors that contributing to the poor performance in 2021 from the perspective of personnel categories

The PDCA schedule developed by the “Gantt chart”
Do
When the plan has been made, the next stage was to implement (Do). We constantly insist on talent training as the first step, clinical competence as the primary benchmark, and increasing NMLE pass rates as the end goal; as a result, all phases, from the study of fundamental medical knowledge to the development of clinical reasoning to exam preparation, should be strongly emphasized and supported. The undergraduate stage is the foundation stage for medical students to grasp medical professional knowledge, even if the application period for the examination is after undergraduate stage and the examinees are already in the standardized residency training or postgraduate stage. Therefore, we adopt a two-pronged approach with theoretical teaching and practical teaching, and carried out the study of the content of the NMLE throughout the whole process from “teaching” to “examination”, so as to consolidate the basic knowledge of medicine for the medical students and make a scientific plan for the path of growth. The enhancement program incorporated two stages, including the “undergraduate stage” and the “pre-examination stage”. Under the “undergraduate stage” of the enhancement program, the content of the theoretical, practical teaching and the evaluation of learning effects were designed according to requirements of the NMLE. Under the “pre-examination stage” of the enhancement program, the QIT held monthly working meetings to promote multiple rounds of simulation examinations for medical practitioners, strengthen the training of weak links, supervise students to brush up on the questions and investigate the situation of the students.
When determining the main reasons of the poor performance in 2021, the QIT strategically attempted to address these issues by taking a number of steps, such as increasing examinees’ attention, providing training venues and individualized guidance, supervising students to review exam questions and organizing simulation examinations; and creating “Gantt charts” to help implementing these measures. In order to increase the attention of examinees and supervisors, the QIT issued rules and regulations for both examinees and supervisors. For example, the pass rate of examinees was factored into the supervisors’ promotions, award applications, and annual evaluations by the department director, which helped to stimulate the enthusiasm of both the trainees and the supervisors. To improve the skill ability of examinees, the Training Center provided more specialized training venues, such as “standardized patient platforms” with a wide variety of clinical cases for “Medical History Collection,” dummies for “Physical Examination” and “Specialized Skills”, and extended training time to increase the number of practice opportunities for examinees who were engaged in clinical or research work. In addition, the skill courses in relation to electrocardiogram and X-ray interpretation and others were developed by the senior doctors from the corresponding department, and the students were required to participate with attendance records. Moreover, examinees who had problems in skill operation could ask for individual guidance from supervisors.
The theoretical subjects of the NMLE are divided into four parts, including basic classification, humanities classification, preventive medicine classification and clinical classification, which comprehensively evaluate the overall mastery of medical knowledge and their ability to internalize the basic knowledge of medicine into the diagnosis and treatment of diseases. To comprehensively acquire the theoretical knowledge, the QIT supervised students to brush up on exam questions from February 2022, and an attendance record system was introduced to make comprehensive records to track the training process of trainees. The results were required to be feedback to the mentorship. Furthermore, the QIT organized 5 cycles of simulation examinations to test all examinees’ abilities monthly prior to the final examination. After obtaining the results of the simulation tests, the QIT conducted a careful analysis of the examinees’ performances, and those who did not pass the exam would receive feedback from supervisors before being re-evaluated. Moreover, the QIT held a symposium for them through face-to-face talks to figure out the difficulties, and informed the department to pay close attention to them.
Check
In this step, the QIT calculated the pass rate for the next year to demonstrate the efficacy of the PDCA plan according to the comparisons with the results of previous years. Additionally, face-to-face symposiums and questionnaire studies were conducted to identify any issues and offer solutions. For example, in the questionnaire interview after the 2022 NMLE, 91% of the examinees believed that the PDCA management was helpful in their preparation for the NMLE while the others thought it ineffective. 14% of the examinees expressed that the amount of time given to prepare for the NMLE was not enough. They found it challenging to balance their time and energy due to the heavy clinical workload and demanding academic studies. Therefore, it is necessary to create effective time management strategies and delegate tasks based on the specific needs. Attendance, assessment, evaluation, and suggestion data were compiled to make necessary changes to the curriculum design and teaching strategies while also improving the quality of teaching.
Action
The last step was “Action,” which primarily included review and improvements for standardization workflow and continuous monitoring. In this stage, it was proposed to review the whole process of the PDCA plan and formulate the standardized preparation program for the NMLE according to the national requirements and standards. Continuous monitoring was associated with prolonging the PDCA plan’s duration and carrying on with data collection for the following cycle.
Statistical analysis
Statistical analyses were performed using Pearson’s chi-squared test, Yates-corrected chi-square test, Welch’s t test (Graphpad Prism 10) or Mann-Whitney U test (R software, version 4.2.1, Vienna, Austria). A P-value level of < 0.05 was considered significant.
link