닫기

Search
Search
Advanced Search +
닫기
Quick links
Most Cited Most View Most Download Ahead of Print Archives Go to e-Submission

Original Article

Split Viewer

Journal of Acupuncture Research 2022; 39(2): 105-114

Published online May 20, 2022

https://doi.org/10.13045/jar.2022.00038

© Korean Acupuncture & Moxibustion Medicine Society

Development of an Objective Structured Clinical Examination Checklist and a Post-Education Questionnaire for Musculoskeletal Ultrasound Training Focusing on Volar Wrist and Carpal Tunnel Syndrome

Eunbyul Cho1, Young-Min Han2, Yeonseok Kang1,3, Jae-Hyo Kim1,4, Min-Seop Shin5, Myungjin Oh6, Hyun-Jong Jung1, Hyesoo Jeon1, Nam Geun Cho1, Jungtae Leem1,4*

Author Info. +

1College of Korean Medicine, Wonkwang University, Iksan, Korea;2Cheoncheon Public Health Center Branch, Jangsu, Korea;3Center for International Cooperation and Education on Korean Medicine, Wonkwang University, Iksan, Korea;4Research Center of Traditional Korean Medicine, Wonkwang University, Iksan, Republic of Korea;5Shin Min Seop Korean Medicine Clinic, Jeonju, Korea;6Keumkang Korean Medical Clinic, Cheongju, Korea

Received: February 24, 2022; Revised: March 22, 2022; Accepted: April 8, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Go to

Background: The objective structured clinical examination (OSCE) is used in the colleges of Korean Medicine, but few studies have validated the OSCE evaluation criteria or post-education questionnaires. Diagnostic ultrasound is used in Korean medicine treatment including acupuncture, acupotomy, and pharmacopuncture to increase the safety and efficacy of treatment. We aimed to develop and validate a OSCE checklist and questionnaire for diagnostic musculoskeletal ultrasound training. Methods: A OSCE checklist and rubric for diagnostic ultrasound training, and questionnaire was developed using literature research. Eight expert panelists verified each draft item in a single-round survey. Items with a content validity ratio (CVR) < 0.75 were excluded or modified to reflect the experts’ opinions. Results: The OSCE checklist and rubric for diagnostic ultrasound training focusing on volar wrist and carpal tunnel syndrome included: 15 items revised according to CVR and expert opinions, the pre-examination procedure, structures to be identified by ultrasound, scans with 2 diagnostic criteria for carpal tunnel syndrome, an explanation of the exam results, and the post exam procedure. The questionnaire consisted of 15 items, including the overall evaluation of training, the effect of the OSCE, and the perception of the ultrasound. All 6 self-evaluation items were not revised, as they had a CVR of ≥ 0.75. Conclusion: An ultrasound OSCE for scanning the volar wrist and diagnosing carpal tunnel syndrome was developed using 15 validated tasks, 15 survey questions about ultrasound training, and 6 questions for selfevaluation. These results may be used in the future for education in diagnostic ultrasound.

Keywords carpal tunnel syndrome, checklist, self-evaluation, ultrasound

Introduction

Go to

The objective structured clinical examination (OSCE) was developed in 1975 in the UK, and is widely used in the assessment of healthcare students to evaluate clinical performance using a valid, reliable approach [1,2]. Previously, the students’ competence in taking the medical history, performing a physical examination, diagnosis, and providing a management plan was evaluated mainly at the bedside using a single, lengthy case. However, concerns were raised that the number of items was too small, not sufficiently objective, and in terms of reliability and validity it was deemed low. To resolve these problems, Harden et al developed the OSCE to control the characteristics for the patient, and examiner, and variables of the clinical examination. Students examined using the OSCE are evaluated over the diagnosis of various conditions and several patients, allowing many skills to be evaluated, and the evaluator bias is minimized due to the checklist items or rating scale assigned in advance (practical and flexible approaches are also provided for the use in other situations) [1,2]. As the OSCE became widely adopted around the world, versions of the OSCEs were developed and implemented in various areas of healthcare education including medicine nursing, and dentistry [1]. In Korea, since 2009, with the launch of the national practical examination of doctors, the OSCE has been implemented in all medical schools [3]. The implementation of the OSCE in Korean medicine education is increasing, however, there have been no discussions to increase its reliability and validity. The Institute of Korean Medicine Education and Evaluation require that the Clinical Performance Examination or the OSCE is conducted at least once a year in Korean medicine clinical education as an eligibility criteria of the 2nd cycle of evaluation and accreditation of Korean medicine [4]. It is likely that most colleges of Korean medicine are implementing the OSCE, as all colleges of Korean medicine (school of Korean medicine) have completed the 2nd cycle of evaluation and accreditation of Korean medicine [5]. Furthermore, each college of Korean medicine should carry out at least 10 OSCE (including at least 1 OSCE for each clinical subject), for the next evaluation and accreditation of Korean medicine [6]. However, research on the development and validation of a checklist for OSCE evaluation in Korean medicine education is still in an early stage, and there are few studies that have developed an OSCE checklist for Korean medicine. In a previous study on the experience and students’ perception of acupuncture and moxibustion medicine OSCE, the existing OSCE checklist was partially modified to suit the situation without a validation process [7]. OSCE checklists or rating scales are going to be required for introducing the practical examination in the 2030 national examination for Korean medicine, as discussed by the Deans Association of colleges of Korean medicine in 2019, which will require sufficient discussion by experts [8]. Referring to the introduction in 2009 of the medical license examination practical exam in Korea, after selecting the evaluation items during the preparation for the practical questions, the experts in each item were asked for a draft of a rating scale of detailed items that the student needs to perform, which the Questions Review Committee completed after deliberation [9]. Pilot studies related to the production of such checklists and rubrics are needed in Korean medicine, so, although the list of critical skills that students majoring in Korean medicine must be able to perform by graduation has not yet been established, the need for education on ultrasound diagnostic devices is increasing as reported in a 2013 survey of Korean medicine graduate school students [10]. In 2017 a survey of full-time faculty members at the colleges of Korean medicine revealed that competency was important to Korean medicine doctors (just after obtaining their license and in the next 10 years), specifically in the use of diagnostic devices (e.g., ultrasound, pulsimeter) [11]. In addition, studies using diagnostic ultrasound devices have been reported to increase the safety and efficacy of Korean medicine treatment [12–15]. Diagnostic ultrasound training is required for undergraduate courses because the safety of a procedure can be improved if the structure and depth of the treatment site are identifiable [16,17]. Evidence shows that ultrasound-guided interventions enable the delivery of more effective treatment for relieving pain and improving function compared with procedures where diagnostic ultrasound has not been used [18]. Instruction on scanning using ultrasound and scan analysis are included in the textbook for practice in diagnostics of Korean medicine [19], and ultrasound images of all acupoints are included in the textbook for the practice of acupuncture [20]. There has been a report of teaching how to perform a pelvic examination using ultrasound in clinical practice education in Gynecology of Korean medicine [21]; however, to our knowledge, there has been no report of implementing the OSCE or developing an OSCE checklist in the College of Korean Medicine. Therefore, we aim to report the process and results of developing an OSCE checklist for diagnostic musculoskeletal ultrasound and a questionnaire for student feedback using expert consensus. The goal of this study was to develop an evaluation sheet for ultrasound training in the future, thus improving the quality of training, and contributing to the competent use of ultrasound in the clinical practice of Korean medicine.

Materials and Methods

Go to

In this study, after drafting the OSCE checklist and questionnaires through a literature review, the validity of content was verified by an expert group to evaluate each item through a questionnaire survey.

Ethical consideration and informed consent

We acquired prior approval from the institutional review board of Wonkwang University for validation by survey (no.: WKIRB-202106-SB-039). The voluntary consent of the panelists was obtained before their participation in the survey.

 Objectives

The OSCE checklist of this study was developed and validated for diagnostic musculoskeletal ultrasound training for students of the College of Korean Medicine (Fig. 1).

 Setting learning objectives and contents

Education in ultrasound is provided to the senior students of the College of Korean Medicine during the clinical skills practice course. The contents of this course were drafted by a Korean Medicine doctor who was licensed to practice musculoskeletal sonography [22,23] and a doctor of Korean Medicine with experience in clinical research in ultrasound who had more than 2 years of implementing the OSCE. During the drafting of the learning objectives and training required in musculoskeletal ultrasound, advice was sought from a professor of Korean medicine diagnostics. The volar wrist was selected as the area to be scanned by ultrasound and the competency of the student to diagnose carpal tunnel syndrome was included as a learning objective. The anatomical structures in the volar wrist are located superficially, thus, it is fairly easy to perform an ultrasound scan in this location. In addition, scanning the wrist can be performed without concern about body exposure. Furthermore, testing the competency of the student to diagnose carpal tunnel syndrome is essential because the number of people diagnosed with, and receiving treatment for carpal tunnel syndrome at Korean medical institutions is increasing [24]. Table 1 presents the learning objectives of the clinical skills practice in ultrasound.

 Literature review

Two researchers drafted the OSCE checklist and questionnaire on ultrasound training. Firstly, the overall ultrasound diagnosis procedure was checked in the Practice in Diagnostics of Korean medicine [19], and Practice of Acupuncture; A Guidebook for College Students [20], which are the recommended textbooks for students at the College of Korean Medicine, and contain ultrasound-related content. In addition, structures to be scanned in the anterior part of the wrist and methods for diagnosing carpal tunnel syndrome were extracted and selected by referring to musculoskeletal ultrasound books [25–29] and a previous study [30]. The OSCE was designed with a checklist and rubric for each item, and post-education survey items by referring to previous studies [31–35] and books [36,37] on the ultrasound section in the OSCE. The draft OSCE checklist consisted of 11 tasks to be performed before, during, and after the ultrasound examination. The total potential score was 23 points. The questionnaire consisted of items about ultrasound training and the OSCE, and questions for self-evaluation after training. The draft was drawn through agreement by 2 researchers and was passed on to the next round after review by a specialist in internal Korean medicine with more than 10 years of clinical research experience.

 Expert panels

A group of 8 panelists was formed for content validation of the OSCE checklist and draft questionnaire. Information on the experts is presented in Table 2.

 Content validation

The survey was distributed using e-mail or messenger with an online survey tool (SurveyMonkey, San Mateo, CA, USA) between June 23, 2021, and June 27, 2021. The learning objectives of the ultrasound training, structures that can be observed with ultrasound from the volar wrist, and methods for diagnosing carpal tunnel syndrome with ultrasound were presented to the experts’ before the survey.

The content validity ratio (CVR) that was used in this study is calculated using the following formula to judge content validity where Ne is the number of panels that answered “essential,” and N is the total number of panels [38,39].

CVR=Ne-N/2N/2

According to the minimum CVR values, 0.75 (“essential” from 7 out of 8 panelists) is required to satisfy a 0.05 significance level. Therefore, 8 experts were asked to rate the draft on a 4-point scale (not at all essential, slightly not essential, slightly essential, strongly essential), and 7 or more panelists responded that “slightly essential” or “strongly essential” were acceptable. The validity of all the OSCE checklist items was evaluated, and then the validity of the scoring criteria (rubric) of each item was evaluated. Panelists were asked to present specific opinions, including revising the score on each item, if necessary.

Results

Go to
The OSCE checklist and rubric

Content validation of the draft OSCE checklist and rubric are shown in Table 3. Item 6, with a CVR of less than 0.75 was excluded from the final version of the checklist. For Item 3, the expression “with hand sanitizers” was removed and modified to only evaluate whether hand hygiene was performed. Although the CVR of Item 5 was 1, selecting the probe as “linear,” the area to be scanned as “musculoskeletal” and “wrist,” and adjusting the depth to scan were added to the rubric because the expert opinion was that the capability to select or adjust the setting of the device for musculoskeletal ultrasound should be added. Item 6 was included to evaluate probe handling, but the CVR was 0.5. There was an opinion that the evaluation criteria for proficiency were vague, and it would be better to consider the pressure and angle of the probe handling, and holding the probe in such a way as to prevent slipping, in addition to adjusting the direction and position of the probe. Thus, the item was modified to evaluate both adjusting the direction and position of the probe, and whether the probe was held stably. Item 7 had a CVR of 1 but was modified to “apply sufficient gel to the lens surface of the probe or the patient’s body to be scanned” to reflect the opinion that gel can be applied to the patient’s body rather than only the probe. Item 8 was to scan and explain the structures that can be observed from the volar wrist, but there was an opinion that using a log scale instead of a linear scale would be better because if the student could not scan, there was no point using a linear scale. Therefore, 10 structures of volar wrist were listed, and the rubric was changed to identifying 9–10 structures for 3 points, 4–8 structures for 2 points, 1–3 structures for 1 point, and 0 points if the student failed to scan any structures. Hand hygiene after the ultrasound examination was added following patient contact. Cleaning the device was added following expert opinion. In addition, self-introduction to the patient was added to the 1st part of the checklist, according to expert opinion that it was necessary. The final OSCE checklist consisted of 15 tasks with a total score of 28 points (Supplementary A).

 Questionnaire on ultrasound training

The draft questionnaire for ultrasound training and the OSCE consisted of 19 items, including 17 multiple-choice, and 2 subjective questions. Panelists were informed that a 5-point scale would be used to answer multiple-choice questions before the survey. The results of the expert validation of the draft questionnaire were analyzed (Table 4).

Question E-1 was excluded because it was similar to Question 1, and ultrasound was taught in the previous curriculum, in practice education in acupoint and meridians. For Question 1, the opinion that ultrasound is not only a diagnostic device and is not only used in the medical field was accepted, so it was specified as “diagnostic” ultrasound, and the expression “before this training” was added. Questions E-2 and E-3 in the draft were excluded because they had a CVR of 0.5. Question E-4 was excluded because it was related to Question E-3. By reflecting on the opinions of the expert panel Question 8 which had a CVR of 0.75, was modified to “Did the ultrasound OSCE help motivate independent practical training?” to improve the understanding of the question. For Question 9, the expression “difficulty high” was modified to “difficult” reflecting the opinion that expression was not appropriate. Through this modification, the final version of the questionnaire for ultrasound training which consisted of 15 questions, was derived (Supplementary B).

 Questionnaire for self-evaluation

The draft questionnaire for self-evaluation was finalized without revision since all self-evaluation questions had a CVR of 0.75 or higher and there were no additional opinions offered by the expert panel (Table 5). The Korean version of the final OSCE checklist and questionnaire on training in performing ultrasound are available online at http://dx.doi.org/10.13140/RG.2.2.35088.89609.

Discussion

Go to

In this study, after setting the learning objectives that senior students of the College of Korean Medicine can use ultrasound devices, examine the structures of the body, and evaluate clinical implications, we developed and validated an OSCE checklist for scanning volar wrist with ultrasound and examined it according to the diagnostic criteria for carpal tunnel syndrome, a questionnaire concerning ultrasound training, and a questionnaire for self-evaluation. The OSCE checklist was modified from 11 items (23 points) in the draft to 15 items (28 points) in the final version. As for the questionnaire on ultrasound training, the draft consisted of 17 multiple-choice and 2 subjective questions, and the final draft, derived after review by 8 expert panels, included 14 multiple-choice and 1 subjective question. The draft questionnaire for self-evaluation was verified as valid and was decided as the final draft without modification. The results of this study will be used for musculoskeletal ultrasound training and OSCE for future doctors in Korean medicine.

Ultrasound is increasingly being used in education and clinical fields. Abdominal ultrasound OSCE was performed in postgraduate medical education in the 1990s, and the national ultrasound curriculum was developed in the United States in the 2010s [31]. Point-of-care ultrasound, taking ultrasound to the bedside, and examining the patient’s symptoms and signs in real time has been rapidly spreading in patient care and education [40]. Point-of-care ultrasound is being used in various fields, including anatomy, physiology, physical examination, pathology and clinical skills, and the importance of ultrasound training in medical schools [40, 41]. In Korea, ultrasound has been included in the training course of internal medicine residents since 2017, and a resident training staff certification system for ultrasound education was introduced [42].

The development and validation of the ultrasound OSCE checklist have been reported in several studies, but it has not yet actively been used in the educational field. One such study evaluated the importance of the rheumatoid OSCE assessment checklist through a single-round survey of experts in 2015 [43]. In that study, musculoskeletal ultrasound, especially “demonstrate/describe how to inject carpal tunnel syndrome” was included in the final rehabilitation station, but there were no items related to scanning the volar wrist or diagnosing carpal tunnel syndrome [43]. In addition, even though assessments for critical care ultrasound [44] and thoracic sonography [45] have been developed, validated checklists in previous studies mainly include procedures for using ultrasound and identification, rather than pre-examination procedures such as explaining the examination, hand hygiene, and post-examination procedures such as explaining the final findings. There have been several feasibility studies of training musculoskeletal ultrasound for undergraduate students [46], internal medicine residents [35], and rheumatology fellows [47], but specific tasks or checklists to be performed by the student have not been described. There are no previous studies on the development of an ultrasound OSCE checklist in traditional medicine.

According to a scoping review of ultrasound curricula in undergraduate medical education, musculoskeletal ultrasound is included in various topics of integrated point-of-care ultrasound curricula. The objectives/skills of musculoskeletal ultrasound are identifying the ultrasound appearance of muscles, tendons, bones, and nerves. The median nerve and carpal tunnel are presented as examples of the nerve [41]. Although ultrasound scanning of the median nerve and carpal tunnel has been included in medical education, cases of ultrasound education or specific OSCE evaluation checklist have not been reported so far. The OSCE checklist of this study could be applied to various fields in the future since proper OSCE rating scales are lacking, although the frequency of ultrasound use in clinical and educational fields is increasing.

In this study, most of the items had a CVR of 0.75 or higher, so content validity could be considered high, but we considered the average, median, mode, and opinions of expert panels to further adapt to the educational field. According to previous OSCE experience, there were cases in which evaluation items were not clear among checklists published as books, or it was difficult to grade because 2 or more tasks were included in 1 item [7]. Therefore, we aimed to include 1 task per item when developing the OSCE checklist. In addition, expert consensus could reduce the number of variables that can occur in OSCE as much as possible.

We aimed to increase validity through content validation from 8 expert panelists. To date, several studies have reported surveys of education. Some studies transculturally translated validated English questionnaires into Korean [48], but most existing studies published in journals of Korean medicine produced questionnaires themselves and did not mention how to validate the questions or conducted the survey after being reviewed by 4 or 5 experts [7,10,21,49–59]. Researchers planning future surveys for Korean medical students must consider how to determine the reliability and validity of the survey questions.

Eight panelists were requested for content validation, and professors from the College of Korean Medicine, where the OSCE checklist and survey questions on ultrasound training will be used, were targeted in particular. Considering the previous study [60] that suggested recruiting 5 to 20 panelists in the Delphi technique, this may be a relatively small number. It is desirable for key stakeholders to discuss the OSCE rating scale, but it is still difficult for several professors to discuss OSCE within a single college of Korean medicine. The panels in this study are not directly related to the OSCE. However, we intended to include at least 2 experts in each ultrasound, practice, and education. Panelists who majored in basic Korean medicine, clinical Korean medicine, and pedagogy were included. The composition of this expert group allowed a review of various viewpoints and backgrounds.

This study had several strengths. To our knowledge, this study is the first to develop an ultrasound OSCE checklist in the Korean medicine or integrative medicine field. We comprehensively reviewed textbooks and guidelines, as well as related previous research, to develop a practical checklist. In addition, we adopted an expert survey method for content validity. The validated questionnaire, not only for the ultrasound OSCE but also for overall training, could be used for further assessment and improvement of ultrasound training models.

However, this study has some limitations. One limitation is that the content validation procedure was conducted only once. A more detailed discussion would have been possible if a focus group interview was conducted after the survey. In addition, if the Delphi technique is used in the future, it would be possible to develop a more valid OSCE by collecting expert opinions several times and drawing consensus. A scoping review is needed to understand the current status of Korean medicine education, especially on the OSCE. Future ultrasound training might be supplemented by receiving feedback from students using the results of this study and post-education questionnaire. Moreover, a before-after study of self-efficacy on musculoskeletal ultrasound could be conducted by a self-evaluation survey before and after ultrasound training. A feasibility study on training ultrasound, implementing OSCE, and improving education by students’ satisfaction survey using the results of this study is needed.

Conclusion

Go to

As practical examination is scheduled to be introduced in the national examination for Korean medicine doctors and clinical skills examinations are beginning at the College of Korean Medicine, studies on educating various clinical skills and designing and validating OSCE are needed. This study suggested a validated musculoskeletal ultrasound OSCE checklist and rubric to train senior students of Korean medicine for their future use of musculoskeletal ultrasound after graduation. In addition, we described the process and results of developing and validating questionnaires about training and self-evaluation questions to investigate the opinions of students. The quality of education could be improved by using the results of this study for ultrasound training and OSCE for various diseases and anatomical structures in the future.

Funding

Go to
This work was supported by a National Research Foundation of Korea grant funded by the Korea government (no.: 2022R1C1C200873811). This research was also supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute, funded by the Ministry of Health and Welfare, Republic of Korea (grant no.: HF20C0173)

Figures

Go to
Fig. 1.

Students of the College of Korean medicine training ultrasound in clinical skills center, and a preceptor observing the students’ performances and providing feedback.


Tables

Go to

Learning Objectives of Musculoskeletal Ultrasound Training, Focusing on Scanning the Volar Wrist Area, and Diagnosis of Carpal Tunnel Syndrome.

Learning outcomes
1. I can use an ultrasonic device.
 1) I can guide the patient into the correct posture before conducting the ultrasound examination and explain thoroughly to them about the upcoming examinations.
 2) I can select the probe suitable for the anatomical structure during the examination and also use it appropriately.
 3) I understand the principles of ultrasonic devices and can use each button appropriately while conducting the examination.
 4) I can make quantitative evaluation and diagnosis of a patient’s condition using various functions of ultrasound.
2. I can examine structures of the body with ultrasound and evaluate their clinical implications.
 1) I can explain the anatomical structure of the wrist anterior aspect.
 2) I can examine the wrist anterior aspect through an ultrasonic device and describe each structure in the clip.
 3)I can understand the diagnosis of carpal tunnel syndrome using ultrasound and explain the test results to the patient.

Characteristics of the 8 Experts on the Panel.

Expert panelAgeExpertiseExperience with ultrasoundExperience with education
150’sKorean medicineUsed ultrasound devices in clinic > 25 yLectured on the musculoskeletal system and abdominal clinical ultrasound education for > 5 y
240’sAcupuncture and moxibustion medicinePublished ultrasound-related clinical research papersLectured on acupuncture and moxibustion medicine at the College of Korean medicine for > 15 y
340’sRehabilitation of Korean medicinePublished ultrasound-related clinical research papersLectured on rehabilitation of Korean medicine, Chuna manual medicine, and clinical practice at the College of Korean medicine for > 10 y
440’sKorean medicine diagnosticsPublished papers on ultrasoundLectured on diagnostic practice at the College of Korean medicine
550’sMeridian and acupointUsed ultrasound in teaching for > 5 yLectured on meridian and acupoint practice at the College of Koran medicine for > 20 y
640’sMeridian and acupointDeveloped an ultrasonic moxibustion device and illustrated an ultrasonic diagramLectured on meridian and acupoint at the College of Korean medicine for > 5 y
740’sEducation policyNoneInvestigated various educational policies (national examination, evaluation and accreditation of Korean medicine education, license system, education policy)
840’sPedagogyNoneInvestigated various studies on competencies and education of Korean medicine

Analysis of the Expert Opinions on the OSCE Draft Checklist and Rubric for Musculoskeletal Ultrasound Focused on Volar Wrist and Carpal Tunnel Syndrome.

No.OSCE checklist, rubric (score)AverageMedianModeCVR
1Was patient identification (name, date of birth) performed? (1)4441
1-R1 point: Performed0 points: Not performed3.625440.75
2Did the student explain the ultrasound examination to the patient? (1)4441
2-R1 point: The student explained the purpose, necessity, time required, and possible side effects of the ultrasound examination0 points: The student didn’t explain about the ultrasound examination4441
3Was hand hygiene performed with hand sanitizers before the examination? (2)3.875441
3-R2 points: Hand hygiene was performed according to the 6 steps of hand hygiene1 point: Hand hygiene was performed, but only some steps out of the 6 hand hygiene steps were followed0 points: Was not performed3.75441
4Was guidance into the correct posture for the examination provided to the patient? (2)4441
4-R2 points: The student guided the patient to supinate the forearm and extend the wrist at about 45 degrees1 point: The student encouraged the patient into the correct posture, but it was not suitable for the examination0 points: Not performed3.75441
5Did the student choose an appropriate probe for scanning the body part? (1)3.875441
5-R1 point: The student set an appropriate probe for the wrist examination0 points: Not performed3.53.541
6Was the student proficient in using the probe? (2)3.125340.5
6-R2 points: The student could adjust the direction and position of the probe to identify the required anatomical structures1 point: The student could adjust the direction and position of the probe, but was not proficient in using it0 points: The student failed to use the probe in a proper way3.53.541
7Was the examination conducted after the gel was sufficiently applied to the lens surface of the probe? (1)3.625441
7-R1 point: Enough gel was applied0 points: Gel was not applied3.375331
8Were the following structures identified and described? (5)3.75441
8-1Scaphoid, Pisiform (1)3.875441
8-2Median nerve (1)3.625440.75
8-3Ulnar artery (1)3.625440.75
8-4Flexor retinaculum (1)3.875441
8-5Flexor pollicis longus (1)3.875441
8-R1 point: Both identification and description were correct0 points: Identification or description was incorrect (2 structures under 1 item have to be performed both accurately for the point)3.5440.75
9Measuring the structure of the proximal carpal tunnel and explaining the result correctly (3)3.625440.75
9-1Did the student successfully connect the top of the scaphoid and the pisiform with a line? (1)3.875441
9-2Did the student measure the vertical distance between the line connecting the top of the scaphoid and the pisiform and the flexor retinaculum? (1)3.75441
9-3Did the student accurately explain the examination results? (1)3.5440.75
10Measuring the median nerve’s cross-sectional area proximal to the Flexor retinaculum and explaining the results (3)3.3753.540.75
10-1Did the student properly measure the cross-sectional area of the median nerve proximal to the flexor retinaculum? (2)3.625441
10-R2 points: The student measured the cross-sectional area of the median nerve proximal to the flexor retinaculum1 point: The student measured the cross-sectional area of the median nerve proximal to the flexor retinaculum, but the measurement was inaccurate1 point: The student measured the cross-sectional area of the median nerve, but the area to be scanned was incorrect0 points: Both the area to be scanned and the cross-sectional area of the median nerve were inaccurate3.5440.75
10-2Did the student accurately explain the examination results? (1)3.625440.75
11Did the student inform the patient about the examination’s completion and explain the final findings after the examination? (2)4441
11-R2 points: Both tasks were completed1 point: Only 1 task was completed0 points: Both were not performed3.625441

CVR, content validity ratio; OSCE, objective structured clinical examination; R, rubric.

Analysis of Expert Opinions on the Draft for the Questionnaire on Ultrasound Training.

No.Survey questionsAverageMedianModeCVR
E-1Have you been trained on ultrasound before?3.53.541
1Do you have first-hand experience of using diagnostic ultrasound before this training?3.75441
E-2Do you think that the topic of ultrasound education (identifying anterior wrist structures and diagnosis of carpal tunnel syndrome) was appropriate?3.5441
2Were the learning objectives of ultrasound training appropriate?3.625440.75
3Were the learning objectives and training contents well related?3.75440.75
4Was it easy to understand the contents of ultrasound training?3.625440.75
5Has ultrasound training improved your knowledge about the anatomy and physiology of carpal tunnels?3.875441
E-3Do you want more time for ultrasound practice than this one?3.375440.5
E-4(only for those who answered positively on E-3 to be surveyed) Please fill out the desired training time. (Subjective question)Example: ○ times per week, ○ weeks, or ○ semesters3.253.540.75
6Did the ultrasound practice help you develop ultrasound skills?3.625440.75
7Did the ultrasound OSCE checklist reflect the training?3.625440.75
8Was OSCE useful in evaluating your skills?3.625440.75
9Was the ultrasound OSCE’s difficulty high?3.5440.75
10Was the OSCE time limit (5 min) enough?3.5440.75
11Are you generally satisfied with this ultrasound training?4441
12Do you have any intentions of studying ultrasound more in the future?3.875441
13(only for those who answered positively on 12 to be surveyed) Please fill out the human body parts that you want to learn more about, through ultrasound education, in addition to wrist. (Subjective question)3.5440.75
14Are you willing to use ultrasound in clinical practice after graduation?3.375440.75
15Has your opinion about ultrasound usage changed positively through this education?3.625440.75

CVR, content validity ratio; E, excluded; OSCE, objective structured clinical examination.

Analysis of the Expert Opinions on the Draft Questionnaire for Self-Evaluation After Ultrasound Training.

No.Survey questionsAverageMedianModeCVR
1I actively participated in ultrasound training.3.875441
2I was considerate of the subject during the ultrasound examination.3.625440.75
3I communicated appropriately with the subject during the ultrasound examination.3.875441
4I can use basic functions of ultrasound such as ultrasonic probe selection, depth and focus control, and freeze.3.75441
5I can identify the anterior structure of the wrist through ultrasound.3.75441
6I can do ultrasound scans according to the criteria for diagnosing carpal tunnel syndrome.3.375440.75

The draft version and the final version are the same.

CVR, content validity ratio.

REFERENCES

Go to
  1. Harden, RM, Lilley, P, and Patricio, M. The definitive guide to the OSCE: The Objective Structured Clinical Examination as a performance assessment. Edinburgh (UK): Elsevier; 2016. p. 13-16, 130-134, 213-322
  2. Harden, RM, Stevenson, M, Downie, WW, and Wilson, GM. Assessment of clinical competence using objective structured examination. BMJ 1975;1:447-451.
    Pubmed KoreaMed CrossRef
  3. Park, H-K. The impact of introducing the Korean Medical Licensing Examination clinical skills assessment on medical education. J Korean Med Assoc 2012;55:116-123. [in Korean]
    CrossRef
  4. Institute of Korean Medicine Educationand Evaluation. The manual of 2nd cycle Korean medicine education evaluation and certification for College of Korean Medicine and Graduate School of Korean Medicine. Seoul (Korea): Karammunhwasa; 2017. 9 p.
  5. Institute of Korean Medicine Education and Evaluation [Internet]. Status of evaluation and certification by year [cited 2022 Jan 31].
    Available from: https://www.ikmee.or.kr/html_2016/02/03.php. [in Korean]
  6. Institute of Korean Medicine Education and Evaluation. KAS2021 Amendment Announcement Public Hearing. Seoul (Korea): Institute of Korean Medicine Education Clinical Performance Examination Evaluation; 2021. p. 14-15.
  7. Cho, E, Lee, J-H, Kwon, OS, Hong, J, and Cho, NG. Assessment of Acupuncture and Moxibustion Medicine Clinical Practice Using the Objective Structured Clinical Examination. J Acupunct Res 2021;38:219-226.
    CrossRef
  8. Akom news [Internet]. Preparing a ‘place of communication’ on current issues of Korean medicine education such as unification and reorganization of licensing examination. 2019 [cited 2021 Dec 6].
    Available from: http://www.akomnews.com/bbs/board.php?bo_table=news&wr_id=36073.[in Korean]
  9. Kim, JH. The effects and challenges of clinical skills assessment in the Korean medical license examination. Korean Med Educ Rev 2013;15:136-143. [in Korean]
    CrossRef
  10. Sim, SB, Kweon, JH, Kim, HW, Hong, JW, and Shin, SW. Student Satisfaction Study of Clinical Skills Training in Korean Medical Education. J Korean Med 2013;34:37-53. [in Korean]
    CrossRef
  11. Kang, YS, Chae, YB, Go, HY, Kim, HH, Kim, HS, and Hong, JS. A study for improvement of the national examination system for Korean medicine doctors. Seoul (Korea): Korea Health Personnel Licensing Examination Institute; 2017. p. 80-81. Report No.: RE02-1802-1803 [in Korean]
  12. Go, HI, Cho, E, Jung, H-J, Lee, S, Kim, J-H, and Kwon, OS. What is the source of pressure pain during abdominal examination in Korean medicine?. Integr Med Res 2020;9:100406.
    Pubmed KoreaMed CrossRef
  13. Chu, H, Kim, J, Ha, W, Cho, E, Kang, G, and Park, S. Change of Safe Needling Depth at Acupoint GB21 according to Posture and Breathing. Evid Based Complement Alternat Med 2018;2018:2308102.
    Pubmed KoreaMed CrossRef
  14. Chu, H, Park, K, Jang, J, Shin, H-R, Yang, M, and Kim, DR. Comparison of Cervical Hyeopcheok Acupoint’s Depth Change with Cervical Rotation in a Prone Position: Protocol and Pilot Study Result for Accurate Observation of Cervical Hyeopcheok Acupoint by Ultrasonography. J Korean Med Rehabil 2019;29:81-88. [in Korean]
    CrossRef
  15. Ahn, T-S, Moon, J-H, Park, C-Y, Oh, M-J, and Choi, Y-M. The Effectiveness of Ultrasound-Guided Essential Bee Venom Pharmacopuncture Combined with Integrative Korean Medical Treatment for Rib Fracture: A Case Study. J Korean Med Rehabil 2019;29:157-163. [in Korean]
    CrossRef
  16. Kim, S, Lee, S, Ha, W-B, Lee, J-H, Jung, H-J, and Chu, H-M. Development of an ultrasound-imaging procedure and acquisition of ultrasound images of acupuncture points for safety and accuracy of needle insertion. Integr Med Res 2017;6:427-433.
    Pubmed KoreaMed CrossRef
  17. Yoon, S-H, Kim, S-A, Lee, G-Y, Kim, H, Lee, J-H, and Leem, J. Using magnetic resonance imaging to measure the depth of acupotomy points in the lumbar spine: A retrospective study. Integr Med Res 2021;10:100679.
    Pubmed KoreaMed CrossRef
  18. Diep, D, Chen, KJQ, and Kumbhare, D. Ultrasound-guided interventional procedures for myofascial trigger points: a systematic review. Reg Anesth Pain Med 2021;46:73-80.
    Pubmed CrossRef
  19. Korean Medicine Diagnostic Practice Compilation Committee. Practice in diagnostics of Korean medicine, 2nd ed. Seoul (Korea): Koonja Publish Co; 2017. p. 51-54. [in Korean]
  20. Editing Textbook of the Acupuncture. Practice of Acupuncture; A Guidebook for College Students, 7th ed. Daejeon (Korea): Jongryeonamoo; 2015. p. 5-610. [in Korean]
  21. Yang, S-J, Park, K-M, and Cho, S-H. The analysis of satisfaction with clinical training and the related factors in the Korean medicine obstetrics and gynecology. J Korean Obstet Gynecol 2020;33:116-127. [in Korean]
  22. Musculoskeletal sonography examination - MSKS / ARDMS [Internet]. 2021 [cited 2021 Dec 23].
    Available from: https://www.ardms.org/get-certified/rmsks/musculoskeletal-sonographer-msks/.
  23. Chung, SH, Lee, HJ, Kim, HS, and Oh, J-Y. Health Insurance Benefit Criteria and Quality Assurance Policies of Diagnostic Ultrasound Services in Other Countries. Health Policy Manag 2014;24:109-119. [in Korean]
    CrossRef
  24. Lee, S-H, Park, H-J, Kim, H-T, Park, S-Y, Heo, I, and Hwang, M-S. Clinical Practice Patterns for Carpal Tunnel Syndrome in Korean Medicine: An Online Survey. Korean Soc Chuna Man Med Spine Nerves 2021;16:73-89. [in Korean]
    CrossRef
  25. Bianchi, S, and Martinoli, C. Ultrasound of the musculoskeletal system. Seoul (Korea): Epublic; 2009. p. 21-27, 46, 48, 55, 98, 100, 448 [in Korean]
  26. Association of Pain and Function Analysis Research group of ultrasound. Muscular ultrasound for pain physician. Seoul (Korea): Medianbook; 2019. p. 6-9. [in Korean]
  27. Han, JH, Choi, SY, Jung, SY, Jang, YI, Lim, CS, and Lee, HS. Ultrasonography Imaging, 2nd ed. Seoul (Korea): MDWorld; 2013. p. 11, 15 [in Korean]
  28. Korean Society of Musculoskeletal Radiology. Musculoskeletal ultrasound, 2nd ed. Paju (Korea): Koonja publish Co; 2018. p. 5-10, 14-15, 494, 591 [in Korean]
  29. Korean Orthopedics Ultrasound Society. Orthopaedic ultrasound. Paju (Korea): Koonja Publish Co; 2019. 78 p. [in Korean]
  30. Wong, SM, Griffith, JF, Hui, ACF, Lo, SK, Fu, M, and Wong, KS. Carpal Tunnel Syndrome: Diagnostic Usefulness of Sonography. Radiology 2004;232:93-99.
    Pubmed CrossRef
  31. Baltarowich, OH, Di Salvo, DN, Scoutt, LM, Brown, DL, Cox, CW, and DiPietro, MA. National Ultrasound Curriculum for Medical Students. Ultrasound Q 2014;30:13-19.
    Pubmed CrossRef
  32. Knobe, M, Münker, R, Sellei, RM, Holschen, M, Mooij, SC, and Schmidt-Rohlfing, B. Peer teaching: a randomised controlled trial using student-teachers to teach musculoskeletal ultrasound. Med Educ 2010;44:148-155.
    Pubmed CrossRef
  33. Kessler, C, and Bhandarkar, S. Ultrasound training for medical students and internal medicine residents-A needs assessment. J Clin Ultrasound 2010;38:401-408.
    Pubmed CrossRef
  34. Ma, I, Wishart, I, Kaminska, M, McLaughlin, K, Weeks, S, and Lautner, D. Medical educators’ perspectives of teaching physical examinations using ultrasonography at the undergraduate level. Can Med Educ J 2013;4:e59-68.
    CrossRef
  35. Gulati, G, Alweis, R, and George, D. Musculoskeletal ultrasound in internal medicine residency - a feasibility study. J Community Hosp Intern Med Perspect 2015;5:27481.
    Pubmed KoreaMed CrossRef
  36. Kyunghee University College of Korean Medicine Clinical Skills Compilation Committee. Korean medicine clinical skills OSCE. Seoul (Korea): Korea Medical Book Publishing Company; 2019. p. 12-20. [in Korean]
  37. Pacific Academy of Sciences 5 minutes OSCE, 5th ed. Seoul (Korea): Pacific Books; 2018. p. 303-309. [in Korean]
  38. Lawshe, CH. A quantitative approach to content validity. Pers Psychol 1975;28:563-575.
    CrossRef
  39. Lee, E-H. Psychometric property of an instrument 1: content validity. Korean J Women Health Nurs 2021;27:10-13. [in Korean]
    Pubmed KoreaMed CrossRef
  40. Moore, CL, and Copel, JA. Point-of-Care Ultrasonography. N Engl J Med 2011;364:749-757.
    Pubmed CrossRef
  41. Tarique, U, Tang, B, Singh, M, Kulasegaram, KM, and Ailon, J. Ultrasound Curricula in Undergraduate Medical Education: A Scoping Review: Ultrasound Curricula in Undergraduate Medical Education. J Ultrasound Med 2018;37:69-82.
    Pubmed CrossRef
  42. Kim, BK, Cha, JM, and Song, DS. How to Revitalize the Abdominal Ultrasonography Education Program. Korean J Gastroenterol 2019;73:66-69. [in Korean]
    CrossRef
  43. Criscione-Schreiber, LG, Sloane, RJ, Hawley, J, Jonas, BL, O’Rourke, KS, and Bolster, MB. Expert Panel Consensus on Assessment Checklists for a Rheumatology Objective Structured Clinical Examination: Consensus on Rheumatology OSCE Assessment Checklists. Arthritis Care Res 2015;67:898-904.
    Pubmed CrossRef
  44. Patrawalla, P, Eisen, LA, Shiloh, A, Shah, BJ, Savenkov, O, and Wise, W. Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound. J Grad Med Educ 2015;7:567-573.
    Pubmed KoreaMed CrossRef
  45. Millington, SJ, Arntfield, RT, Guo, RJ, Koenig, S, Kory, P, and Noble, V. The Assessment of Competency in Thoracic Sonography (ACTS) scale: validation of a tool for point-of-care ultrasound. Crit Ultrasound J 2017;9:25.
    Pubmed KoreaMed CrossRef
  46. Wright, SA, and Bell, AL. Enhancement of undergraduate rheumatology teaching through the use of musculoskeletal ultrasound. Rheumatology 2008;47:1564-1566.
    Pubmed CrossRef
  47. Kissin, EY, Niu, J, Balint, P, Bong, D, Evangelisto, A, and Goyal, J. Musculoskeletal Ultrasound Training and Competency Assessment Program for Rheumatology Fellows. J Ultrasound Med 2013;32:1735-1743.
    Pubmed CrossRef
  48. Kim, H, Jeon, P, Kim, S, Hong, J, and Kang, Y. Evid Based Complement Alternat Med; 2021. 5591911 p.
    Pubmed KoreaMed CrossRef
  49. Joo, S, Eom, D, and Song, J. A research for recognitions of Korean medical student toward traditional Korean medicine and education. J Soc Prev Korean Med 2016;20:1-8. [in Korean]
  50. Park, S, Choi, E, and Kim, S-Y. A Study on the Korean Medical Students’ Perception of Medical Ethics Education Using Flipped Learning. J Soc Prev Korean Med 2018;22:1-12. [in Korean]
  51. Kwon, SW, Shin, SW, and Lim, B. A survey of students’ satisfaction with education in traditional Korean medicine. J Korean Orient Med 2012;33:1-11. [in Korean]
  52. Jo, H-J, Park, J-S, Roh, JD, and Sung, HK. A Survey on Students’ Perception of Clinical Performance Examination (CPX) in College of Korean Medicine Using Student Standardized Patients. J Soc Prev Korean Med 2020;24:1-13. [in Korean]
  53. Kweon, JH, Sim, SB, Kim, EJ, Hong, JW, and Shin, SW. Evaluation of the Implementation of Clinical Performance Examination in Korean Medicine Education. J Physiol Pathol Korean Med 2018;32:51-61. [in Korean]
    CrossRef
  54. Cha, HY, Kim, NH, Hong, JW, and Shin, SW. Evaluation of the implementation of problem-based learning in Korean medical education. Korean J Orient Physiol Pathol 2012;26:351-359. [in Korean]
    Pubmed KoreaMed CrossRef
  55. Cho, CS. Student Satisfaction Study and Interrater Comparative Study on Patient-Physician Interaction Score of Clinical Performance Examination in Korean Medical Education. Korean J Orient Physiol Pathol 2015;29:152. [in Korean]
    CrossRef
  56. Yang, S-B. Survey research for evaluation of real-time non-face-to-face medical education in clinical Korean medicine. J Intern Korean Med 2020;41:984-992. [in Korean]
    CrossRef
  57. Cho, HW, Hwang, EH, Shin, BC, Sul, JU, Hong, JW, and Shin, SW. The analysis of satisfaction with clinical training and the related factors-especially in oriental rehabilitation medicine-. Korea J Orient Prev Med Soc 2012;16:1-15. [in Korean]
  58. Chang, Y-J, and Hong, S-U. The Efficiency of Objective Structured Clinical Examination (OSCE) as a Method of Clinical Clerkship in the Korean Ophthalmology and Otolaryngology Department. J Korean Med Ophthalmol Otolaryngol Dermatol 2014;27:121-130.
    CrossRef
  59. Lim, KS, Lee, JC, and Park, YG. The result of oriental medical ophthalmology and otolaryngology class applying student-oriented teaching and learning method. J Korean Med Ophthalmol Otolaryngol Dermatol 2010;23:163-173. [in Korean]
  60. Rowe, G, and Wright, G. [Internet]. Expert Opinions in Forecasting: The Role of the Delphi Technique. 2001 [cited 2021 Dec 16].
    Available from: http://link.springer.com/10.1007/978-0-306-47630-3_7.
    CrossRef

Article

Original Article

Journal of Acupuncture Research 2022; 39(2): 105-114

Published online May 20, 2022 https://doi.org/10.13045/jar.2022.00038

Copyright © Korean Acupuncture & Moxibustion Medicine Society.

Development of an Objective Structured Clinical Examination Checklist and a Post-Education Questionnaire for Musculoskeletal Ultrasound Training Focusing on Volar Wrist and Carpal Tunnel Syndrome

Eunbyul Cho1, Young-Min Han2, Yeonseok Kang1,3, Jae-Hyo Kim1,4, Min-Seop Shin5, Myungjin Oh6, Hyun-Jong Jung1, Hyesoo Jeon1, Nam Geun Cho1, Jungtae Leem1,4*

1College of Korean Medicine, Wonkwang University, Iksan, Korea;2Cheoncheon Public Health Center Branch, Jangsu, Korea;3Center for International Cooperation and Education on Korean Medicine, Wonkwang University, Iksan, Korea;4Research Center of Traditional Korean Medicine, Wonkwang University, Iksan, Republic of Korea;5Shin Min Seop Korean Medicine Clinic, Jeonju, Korea;6Keumkang Korean Medical Clinic, Cheongju, Korea

Received: February 24, 2022; Revised: March 22, 2022; Accepted: April 8, 2022

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: The objective structured clinical examination (OSCE) is used in the colleges of Korean Medicine, but few studies have validated the OSCE evaluation criteria or post-education questionnaires. Diagnostic ultrasound is used in Korean medicine treatment including acupuncture, acupotomy, and pharmacopuncture to increase the safety and efficacy of treatment. We aimed to develop and validate a OSCE checklist and questionnaire for diagnostic musculoskeletal ultrasound training. Methods: A OSCE checklist and rubric for diagnostic ultrasound training, and questionnaire was developed using literature research. Eight expert panelists verified each draft item in a single-round survey. Items with a content validity ratio (CVR) < 0.75 were excluded or modified to reflect the experts’ opinions. Results: The OSCE checklist and rubric for diagnostic ultrasound training focusing on volar wrist and carpal tunnel syndrome included: 15 items revised according to CVR and expert opinions, the pre-examination procedure, structures to be identified by ultrasound, scans with 2 diagnostic criteria for carpal tunnel syndrome, an explanation of the exam results, and the post exam procedure. The questionnaire consisted of 15 items, including the overall evaluation of training, the effect of the OSCE, and the perception of the ultrasound. All 6 self-evaluation items were not revised, as they had a CVR of ≥ 0.75. Conclusion: An ultrasound OSCE for scanning the volar wrist and diagnosing carpal tunnel syndrome was developed using 15 validated tasks, 15 survey questions about ultrasound training, and 6 questions for selfevaluation. These results may be used in the future for education in diagnostic ultrasound.

Keywords: carpal tunnel syndrome, checklist, self-evaluation, ultrasound

Introduction

The objective structured clinical examination (OSCE) was developed in 1975 in the UK, and is widely used in the assessment of healthcare students to evaluate clinical performance using a valid, reliable approach [1,2]. Previously, the students’ competence in taking the medical history, performing a physical examination, diagnosis, and providing a management plan was evaluated mainly at the bedside using a single, lengthy case. However, concerns were raised that the number of items was too small, not sufficiently objective, and in terms of reliability and validity it was deemed low. To resolve these problems, Harden et al developed the OSCE to control the characteristics for the patient, and examiner, and variables of the clinical examination. Students examined using the OSCE are evaluated over the diagnosis of various conditions and several patients, allowing many skills to be evaluated, and the evaluator bias is minimized due to the checklist items or rating scale assigned in advance (practical and flexible approaches are also provided for the use in other situations) [1,2]. As the OSCE became widely adopted around the world, versions of the OSCEs were developed and implemented in various areas of healthcare education including medicine nursing, and dentistry [1]. In Korea, since 2009, with the launch of the national practical examination of doctors, the OSCE has been implemented in all medical schools [3]. The implementation of the OSCE in Korean medicine education is increasing, however, there have been no discussions to increase its reliability and validity. The Institute of Korean Medicine Education and Evaluation require that the Clinical Performance Examination or the OSCE is conducted at least once a year in Korean medicine clinical education as an eligibility criteria of the 2nd cycle of evaluation and accreditation of Korean medicine [4]. It is likely that most colleges of Korean medicine are implementing the OSCE, as all colleges of Korean medicine (school of Korean medicine) have completed the 2nd cycle of evaluation and accreditation of Korean medicine [5]. Furthermore, each college of Korean medicine should carry out at least 10 OSCE (including at least 1 OSCE for each clinical subject), for the next evaluation and accreditation of Korean medicine [6]. However, research on the development and validation of a checklist for OSCE evaluation in Korean medicine education is still in an early stage, and there are few studies that have developed an OSCE checklist for Korean medicine. In a previous study on the experience and students’ perception of acupuncture and moxibustion medicine OSCE, the existing OSCE checklist was partially modified to suit the situation without a validation process [7]. OSCE checklists or rating scales are going to be required for introducing the practical examination in the 2030 national examination for Korean medicine, as discussed by the Deans Association of colleges of Korean medicine in 2019, which will require sufficient discussion by experts [8]. Referring to the introduction in 2009 of the medical license examination practical exam in Korea, after selecting the evaluation items during the preparation for the practical questions, the experts in each item were asked for a draft of a rating scale of detailed items that the student needs to perform, which the Questions Review Committee completed after deliberation [9]. Pilot studies related to the production of such checklists and rubrics are needed in Korean medicine, so, although the list of critical skills that students majoring in Korean medicine must be able to perform by graduation has not yet been established, the need for education on ultrasound diagnostic devices is increasing as reported in a 2013 survey of Korean medicine graduate school students [10]. In 2017 a survey of full-time faculty members at the colleges of Korean medicine revealed that competency was important to Korean medicine doctors (just after obtaining their license and in the next 10 years), specifically in the use of diagnostic devices (e.g., ultrasound, pulsimeter) [11]. In addition, studies using diagnostic ultrasound devices have been reported to increase the safety and efficacy of Korean medicine treatment [12–15]. Diagnostic ultrasound training is required for undergraduate courses because the safety of a procedure can be improved if the structure and depth of the treatment site are identifiable [16,17]. Evidence shows that ultrasound-guided interventions enable the delivery of more effective treatment for relieving pain and improving function compared with procedures where diagnostic ultrasound has not been used [18]. Instruction on scanning using ultrasound and scan analysis are included in the textbook for practice in diagnostics of Korean medicine [19], and ultrasound images of all acupoints are included in the textbook for the practice of acupuncture [20]. There has been a report of teaching how to perform a pelvic examination using ultrasound in clinical practice education in Gynecology of Korean medicine [21]; however, to our knowledge, there has been no report of implementing the OSCE or developing an OSCE checklist in the College of Korean Medicine. Therefore, we aim to report the process and results of developing an OSCE checklist for diagnostic musculoskeletal ultrasound and a questionnaire for student feedback using expert consensus. The goal of this study was to develop an evaluation sheet for ultrasound training in the future, thus improving the quality of training, and contributing to the competent use of ultrasound in the clinical practice of Korean medicine.

Materials and Methods

In this study, after drafting the OSCE checklist and questionnaires through a literature review, the validity of content was verified by an expert group to evaluate each item through a questionnaire survey.

Ethical consideration and informed consent

We acquired prior approval from the institutional review board of Wonkwang University for validation by survey (no.: WKIRB-202106-SB-039). The voluntary consent of the panelists was obtained before their participation in the survey.

 Objectives

The OSCE checklist of this study was developed and validated for diagnostic musculoskeletal ultrasound training for students of the College of Korean Medicine (Fig. 1).

 Setting learning objectives and contents

Education in ultrasound is provided to the senior students of the College of Korean Medicine during the clinical skills practice course. The contents of this course were drafted by a Korean Medicine doctor who was licensed to practice musculoskeletal sonography [22,23] and a doctor of Korean Medicine with experience in clinical research in ultrasound who had more than 2 years of implementing the OSCE. During the drafting of the learning objectives and training required in musculoskeletal ultrasound, advice was sought from a professor of Korean medicine diagnostics. The volar wrist was selected as the area to be scanned by ultrasound and the competency of the student to diagnose carpal tunnel syndrome was included as a learning objective. The anatomical structures in the volar wrist are located superficially, thus, it is fairly easy to perform an ultrasound scan in this location. In addition, scanning the wrist can be performed without concern about body exposure. Furthermore, testing the competency of the student to diagnose carpal tunnel syndrome is essential because the number of people diagnosed with, and receiving treatment for carpal tunnel syndrome at Korean medical institutions is increasing [24]. Table 1 presents the learning objectives of the clinical skills practice in ultrasound.

 Literature review

Two researchers drafted the OSCE checklist and questionnaire on ultrasound training. Firstly, the overall ultrasound diagnosis procedure was checked in the Practice in Diagnostics of Korean medicine [19], and Practice of Acupuncture; A Guidebook for College Students [20], which are the recommended textbooks for students at the College of Korean Medicine, and contain ultrasound-related content. In addition, structures to be scanned in the anterior part of the wrist and methods for diagnosing carpal tunnel syndrome were extracted and selected by referring to musculoskeletal ultrasound books [25–29] and a previous study [30]. The OSCE was designed with a checklist and rubric for each item, and post-education survey items by referring to previous studies [31–35] and books [36,37] on the ultrasound section in the OSCE. The draft OSCE checklist consisted of 11 tasks to be performed before, during, and after the ultrasound examination. The total potential score was 23 points. The questionnaire consisted of items about ultrasound training and the OSCE, and questions for self-evaluation after training. The draft was drawn through agreement by 2 researchers and was passed on to the next round after review by a specialist in internal Korean medicine with more than 10 years of clinical research experience.

 Expert panels

A group of 8 panelists was formed for content validation of the OSCE checklist and draft questionnaire. Information on the experts is presented in Table 2.

 Content validation

The survey was distributed using e-mail or messenger with an online survey tool (SurveyMonkey, San Mateo, CA, USA) between June 23, 2021, and June 27, 2021. The learning objectives of the ultrasound training, structures that can be observed with ultrasound from the volar wrist, and methods for diagnosing carpal tunnel syndrome with ultrasound were presented to the experts’ before the survey.

The content validity ratio (CVR) that was used in this study is calculated using the following formula to judge content validity where Ne is the number of panels that answered “essential,” and N is the total number of panels [38,39].

CVR=Ne-N/2N/2

According to the minimum CVR values, 0.75 (“essential” from 7 out of 8 panelists) is required to satisfy a 0.05 significance level. Therefore, 8 experts were asked to rate the draft on a 4-point scale (not at all essential, slightly not essential, slightly essential, strongly essential), and 7 or more panelists responded that “slightly essential” or “strongly essential” were acceptable. The validity of all the OSCE checklist items was evaluated, and then the validity of the scoring criteria (rubric) of each item was evaluated. Panelists were asked to present specific opinions, including revising the score on each item, if necessary.

Results

The OSCE checklist and rubric

Content validation of the draft OSCE checklist and rubric are shown in Table 3. Item 6, with a CVR of less than 0.75 was excluded from the final version of the checklist. For Item 3, the expression “with hand sanitizers” was removed and modified to only evaluate whether hand hygiene was performed. Although the CVR of Item 5 was 1, selecting the probe as “linear,” the area to be scanned as “musculoskeletal” and “wrist,” and adjusting the depth to scan were added to the rubric because the expert opinion was that the capability to select or adjust the setting of the device for musculoskeletal ultrasound should be added. Item 6 was included to evaluate probe handling, but the CVR was 0.5. There was an opinion that the evaluation criteria for proficiency were vague, and it would be better to consider the pressure and angle of the probe handling, and holding the probe in such a way as to prevent slipping, in addition to adjusting the direction and position of the probe. Thus, the item was modified to evaluate both adjusting the direction and position of the probe, and whether the probe was held stably. Item 7 had a CVR of 1 but was modified to “apply sufficient gel to the lens surface of the probe or the patient’s body to be scanned” to reflect the opinion that gel can be applied to the patient’s body rather than only the probe. Item 8 was to scan and explain the structures that can be observed from the volar wrist, but there was an opinion that using a log scale instead of a linear scale would be better because if the student could not scan, there was no point using a linear scale. Therefore, 10 structures of volar wrist were listed, and the rubric was changed to identifying 9–10 structures for 3 points, 4–8 structures for 2 points, 1–3 structures for 1 point, and 0 points if the student failed to scan any structures. Hand hygiene after the ultrasound examination was added following patient contact. Cleaning the device was added following expert opinion. In addition, self-introduction to the patient was added to the 1st part of the checklist, according to expert opinion that it was necessary. The final OSCE checklist consisted of 15 tasks with a total score of 28 points (Supplementary A).

 Questionnaire on ultrasound training

The draft questionnaire for ultrasound training and the OSCE consisted of 19 items, including 17 multiple-choice, and 2 subjective questions. Panelists were informed that a 5-point scale would be used to answer multiple-choice questions before the survey. The results of the expert validation of the draft questionnaire were analyzed (Table 4).

Question E-1 was excluded because it was similar to Question 1, and ultrasound was taught in the previous curriculum, in practice education in acupoint and meridians. For Question 1, the opinion that ultrasound is not only a diagnostic device and is not only used in the medical field was accepted, so it was specified as “diagnostic” ultrasound, and the expression “before this training” was added. Questions E-2 and E-3 in the draft were excluded because they had a CVR of 0.5. Question E-4 was excluded because it was related to Question E-3. By reflecting on the opinions of the expert panel Question 8 which had a CVR of 0.75, was modified to “Did the ultrasound OSCE help motivate independent practical training?” to improve the understanding of the question. For Question 9, the expression “difficulty high” was modified to “difficult” reflecting the opinion that expression was not appropriate. Through this modification, the final version of the questionnaire for ultrasound training which consisted of 15 questions, was derived (Supplementary B).

 Questionnaire for self-evaluation

The draft questionnaire for self-evaluation was finalized without revision since all self-evaluation questions had a CVR of 0.75 or higher and there were no additional opinions offered by the expert panel (Table 5). The Korean version of the final OSCE checklist and questionnaire on training in performing ultrasound are available online at http://dx.doi.org/10.13140/RG.2.2.35088.89609.

Discussion

In this study, after setting the learning objectives that senior students of the College of Korean Medicine can use ultrasound devices, examine the structures of the body, and evaluate clinical implications, we developed and validated an OSCE checklist for scanning volar wrist with ultrasound and examined it according to the diagnostic criteria for carpal tunnel syndrome, a questionnaire concerning ultrasound training, and a questionnaire for self-evaluation. The OSCE checklist was modified from 11 items (23 points) in the draft to 15 items (28 points) in the final version. As for the questionnaire on ultrasound training, the draft consisted of 17 multiple-choice and 2 subjective questions, and the final draft, derived after review by 8 expert panels, included 14 multiple-choice and 1 subjective question. The draft questionnaire for self-evaluation was verified as valid and was decided as the final draft without modification. The results of this study will be used for musculoskeletal ultrasound training and OSCE for future doctors in Korean medicine.

Ultrasound is increasingly being used in education and clinical fields. Abdominal ultrasound OSCE was performed in postgraduate medical education in the 1990s, and the national ultrasound curriculum was developed in the United States in the 2010s [31]. Point-of-care ultrasound, taking ultrasound to the bedside, and examining the patient’s symptoms and signs in real time has been rapidly spreading in patient care and education [40]. Point-of-care ultrasound is being used in various fields, including anatomy, physiology, physical examination, pathology and clinical skills, and the importance of ultrasound training in medical schools [40, 41]. In Korea, ultrasound has been included in the training course of internal medicine residents since 2017, and a resident training staff certification system for ultrasound education was introduced [42].

The development and validation of the ultrasound OSCE checklist have been reported in several studies, but it has not yet actively been used in the educational field. One such study evaluated the importance of the rheumatoid OSCE assessment checklist through a single-round survey of experts in 2015 [43]. In that study, musculoskeletal ultrasound, especially “demonstrate/describe how to inject carpal tunnel syndrome” was included in the final rehabilitation station, but there were no items related to scanning the volar wrist or diagnosing carpal tunnel syndrome [43]. In addition, even though assessments for critical care ultrasound [44] and thoracic sonography [45] have been developed, validated checklists in previous studies mainly include procedures for using ultrasound and identification, rather than pre-examination procedures such as explaining the examination, hand hygiene, and post-examination procedures such as explaining the final findings. There have been several feasibility studies of training musculoskeletal ultrasound for undergraduate students [46], internal medicine residents [35], and rheumatology fellows [47], but specific tasks or checklists to be performed by the student have not been described. There are no previous studies on the development of an ultrasound OSCE checklist in traditional medicine.

According to a scoping review of ultrasound curricula in undergraduate medical education, musculoskeletal ultrasound is included in various topics of integrated point-of-care ultrasound curricula. The objectives/skills of musculoskeletal ultrasound are identifying the ultrasound appearance of muscles, tendons, bones, and nerves. The median nerve and carpal tunnel are presented as examples of the nerve [41]. Although ultrasound scanning of the median nerve and carpal tunnel has been included in medical education, cases of ultrasound education or specific OSCE evaluation checklist have not been reported so far. The OSCE checklist of this study could be applied to various fields in the future since proper OSCE rating scales are lacking, although the frequency of ultrasound use in clinical and educational fields is increasing.

In this study, most of the items had a CVR of 0.75 or higher, so content validity could be considered high, but we considered the average, median, mode, and opinions of expert panels to further adapt to the educational field. According to previous OSCE experience, there were cases in which evaluation items were not clear among checklists published as books, or it was difficult to grade because 2 or more tasks were included in 1 item [7]. Therefore, we aimed to include 1 task per item when developing the OSCE checklist. In addition, expert consensus could reduce the number of variables that can occur in OSCE as much as possible.

We aimed to increase validity through content validation from 8 expert panelists. To date, several studies have reported surveys of education. Some studies transculturally translated validated English questionnaires into Korean [48], but most existing studies published in journals of Korean medicine produced questionnaires themselves and did not mention how to validate the questions or conducted the survey after being reviewed by 4 or 5 experts [7,10,21,49–59]. Researchers planning future surveys for Korean medical students must consider how to determine the reliability and validity of the survey questions.

Eight panelists were requested for content validation, and professors from the College of Korean Medicine, where the OSCE checklist and survey questions on ultrasound training will be used, were targeted in particular. Considering the previous study [60] that suggested recruiting 5 to 20 panelists in the Delphi technique, this may be a relatively small number. It is desirable for key stakeholders to discuss the OSCE rating scale, but it is still difficult for several professors to discuss OSCE within a single college of Korean medicine. The panels in this study are not directly related to the OSCE. However, we intended to include at least 2 experts in each ultrasound, practice, and education. Panelists who majored in basic Korean medicine, clinical Korean medicine, and pedagogy were included. The composition of this expert group allowed a review of various viewpoints and backgrounds.

This study had several strengths. To our knowledge, this study is the first to develop an ultrasound OSCE checklist in the Korean medicine or integrative medicine field. We comprehensively reviewed textbooks and guidelines, as well as related previous research, to develop a practical checklist. In addition, we adopted an expert survey method for content validity. The validated questionnaire, not only for the ultrasound OSCE but also for overall training, could be used for further assessment and improvement of ultrasound training models.

However, this study has some limitations. One limitation is that the content validation procedure was conducted only once. A more detailed discussion would have been possible if a focus group interview was conducted after the survey. In addition, if the Delphi technique is used in the future, it would be possible to develop a more valid OSCE by collecting expert opinions several times and drawing consensus. A scoping review is needed to understand the current status of Korean medicine education, especially on the OSCE. Future ultrasound training might be supplemented by receiving feedback from students using the results of this study and post-education questionnaire. Moreover, a before-after study of self-efficacy on musculoskeletal ultrasound could be conducted by a self-evaluation survey before and after ultrasound training. A feasibility study on training ultrasound, implementing OSCE, and improving education by students’ satisfaction survey using the results of this study is needed.

Conclusion

As practical examination is scheduled to be introduced in the national examination for Korean medicine doctors and clinical skills examinations are beginning at the College of Korean Medicine, studies on educating various clinical skills and designing and validating OSCE are needed. This study suggested a validated musculoskeletal ultrasound OSCE checklist and rubric to train senior students of Korean medicine for their future use of musculoskeletal ultrasound after graduation. In addition, we described the process and results of developing and validating questionnaires about training and self-evaluation questions to investigate the opinions of students. The quality of education could be improved by using the results of this study for ultrasound training and OSCE for various diseases and anatomical structures in the future.

Supplementary Materials

Conflicts of Interest

The authors declare that they have no competing interests.

Acknowledgments

EC thanks Jung-Han Lee, O Sang Kwon, Jiseong Hong for advice on OSCE checklist and questionnaire.

Funding

This work was supported by a National Research Foundation of Korea grant funded by the Korea government (no.: 2022R1C1C200873811). This research was also supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute, funded by the Ministry of Health and Welfare, Republic of Korea (grant no.: HF20C0173)

Ethical Statement

This research did not involve any human or animal experiments.

Data Availability

All relevant data are included in this manuscript.

Fig 1.

Download
Figure 1.

Students of the College of Korean medicine training ultrasound in clinical skills center, and a preceptor observing the students’ performances and providing feedback.

Journal of Acupuncture Research 2022; 39: 105-114https://doi.org/10.13045/jar.2022.00038

Table 1 . Learning Objectives of Musculoskeletal Ultrasound Training, Focusing on Scanning the Volar Wrist Area, and Diagnosis of Carpal Tunnel Syndrome..

Learning outcomes
1. I can use an ultrasonic device.
 1) I can guide the patient into the correct posture before conducting the ultrasound examination and explain thoroughly to them about the upcoming examinations.
 2) I can select the probe suitable for the anatomical structure during the examination and also use it appropriately.
 3) I understand the principles of ultrasonic devices and can use each button appropriately while conducting the examination.
 4) I can make quantitative evaluation and diagnosis of a patient’s condition using various functions of ultrasound.
2. I can examine structures of the body with ultrasound and evaluate their clinical implications.
 1) I can explain the anatomical structure of the wrist anterior aspect.
 2) I can examine the wrist anterior aspect through an ultrasonic device and describe each structure in the clip.
 3)I can understand the diagnosis of carpal tunnel syndrome using ultrasound and explain the test results to the patient.

Table 2 . Characteristics of the 8 Experts on the Panel..

Expert panelAgeExpertiseExperience with ultrasoundExperience with education
150’sKorean medicineUsed ultrasound devices in clinic > 25 yLectured on the musculoskeletal system and abdominal clinical ultrasound education for > 5 y
240’sAcupuncture and moxibustion medicinePublished ultrasound-related clinical research papersLectured on acupuncture and moxibustion medicine at the College of Korean medicine for > 15 y
340’sRehabilitation of Korean medicinePublished ultrasound-related clinical research papersLectured on rehabilitation of Korean medicine, Chuna manual medicine, and clinical practice at the College of Korean medicine for > 10 y
440’sKorean medicine diagnosticsPublished papers on ultrasoundLectured on diagnostic practice at the College of Korean medicine
550’sMeridian and acupointUsed ultrasound in teaching for > 5 yLectured on meridian and acupoint practice at the College of Koran medicine for > 20 y
640’sMeridian and acupointDeveloped an ultrasonic moxibustion device and illustrated an ultrasonic diagramLectured on meridian and acupoint at the College of Korean medicine for > 5 y
740’sEducation policyNoneInvestigated various educational policies (national examination, evaluation and accreditation of Korean medicine education, license system, education policy)
840’sPedagogyNoneInvestigated various studies on competencies and education of Korean medicine

Table 3 . Analysis of the Expert Opinions on the OSCE Draft Checklist and Rubric for Musculoskeletal Ultrasound Focused on Volar Wrist and Carpal Tunnel Syndrome..

No.OSCE checklist, rubric (score)AverageMedianModeCVR
1Was patient identification (name, date of birth) performed? (1)4441
1-R1 point: Performed0 points: Not performed3.625440.75
2Did the student explain the ultrasound examination to the patient? (1)4441
2-R1 point: The student explained the purpose, necessity, time required, and possible side effects of the ultrasound examination0 points: The student didn’t explain about the ultrasound examination4441
3Was hand hygiene performed with hand sanitizers before the examination? (2)3.875441
3-R2 points: Hand hygiene was performed according to the 6 steps of hand hygiene1 point: Hand hygiene was performed, but only some steps out of the 6 hand hygiene steps were followed0 points: Was not performed3.75441
4Was guidance into the correct posture for the examination provided to the patient? (2)4441
4-R2 points: The student guided the patient to supinate the forearm and extend the wrist at about 45 degrees1 point: The student encouraged the patient into the correct posture, but it was not suitable for the examination0 points: Not performed3.75441
5Did the student choose an appropriate probe for scanning the body part? (1)3.875441
5-R1 point: The student set an appropriate probe for the wrist examination0 points: Not performed3.53.541
6Was the student proficient in using the probe? (2)3.125340.5
6-R2 points: The student could adjust the direction and position of the probe to identify the required anatomical structures1 point: The student could adjust the direction and position of the probe, but was not proficient in using it0 points: The student failed to use the probe in a proper way3.53.541
7Was the examination conducted after the gel was sufficiently applied to the lens surface of the probe? (1)3.625441
7-R1 point: Enough gel was applied0 points: Gel was not applied3.375331
8Were the following structures identified and described? (5)3.75441
8-1Scaphoid, Pisiform (1)3.875441
8-2Median nerve (1)3.625440.75
8-3Ulnar artery (1)3.625440.75
8-4Flexor retinaculum (1)3.875441
8-5Flexor pollicis longus (1)3.875441
8-R1 point: Both identification and description were correct0 points: Identification or description was incorrect (2 structures under 1 item have to be performed both accurately for the point)3.5440.75
9Measuring the structure of the proximal carpal tunnel and explaining the result correctly (3)3.625440.75
9-1Did the student successfully connect the top of the scaphoid and the pisiform with a line? (1)3.875441
9-2Did the student measure the vertical distance between the line connecting the top of the scaphoid and the pisiform and the flexor retinaculum? (1)3.75441
9-3Did the student accurately explain the examination results? (1)3.5440.75
10Measuring the median nerve’s cross-sectional area proximal to the Flexor retinaculum and explaining the results (3)3.3753.540.75
10-1Did the student properly measure the cross-sectional area of the median nerve proximal to the flexor retinaculum? (2)3.625441
10-R2 points: The student measured the cross-sectional area of the median nerve proximal to the flexor retinaculum1 point: The student measured the cross-sectional area of the median nerve proximal to the flexor retinaculum, but the measurement was inaccurate1 point: The student measured the cross-sectional area of the median nerve, but the area to be scanned was incorrect0 points: Both the area to be scanned and the cross-sectional area of the median nerve were inaccurate3.5440.75
10-2Did the student accurately explain the examination results? (1)3.625440.75
11Did the student inform the patient about the examination’s completion and explain the final findings after the examination? (2)4441
11-R2 points: Both tasks were completed1 point: Only 1 task was completed0 points: Both were not performed3.625441

CVR, content validity ratio; OSCE, objective structured clinical examination; R, rubric..

Table 4 . Analysis of Expert Opinions on the Draft for the Questionnaire on Ultrasound Training..

No.Survey questionsAverageMedianModeCVR
E-1Have you been trained on ultrasound before?3.53.541
1Do you have first-hand experience of using diagnostic ultrasound before this training?3.75441
E-2Do you think that the topic of ultrasound education (identifying anterior wrist structures and diagnosis of carpal tunnel syndrome) was appropriate?3.5441
2Were the learning objectives of ultrasound training appropriate?3.625440.75
3Were the learning objectives and training contents well related?3.75440.75
4Was it easy to understand the contents of ultrasound training?3.625440.75
5Has ultrasound training improved your knowledge about the anatomy and physiology of carpal tunnels?3.875441
E-3Do you want more time for ultrasound practice than this one?3.375440.5
E-4(only for those who answered positively on E-3 to be surveyed) Please fill out the desired training time. (Subjective question)Example: ○ times per week, ○ weeks, or ○ semesters3.253.540.75
6Did the ultrasound practice help you develop ultrasound skills?3.625440.75
7Did the ultrasound OSCE checklist reflect the training?3.625440.75
8Was OSCE useful in evaluating your skills?3.625440.75
9Was the ultrasound OSCE’s difficulty high?3.5440.75
10Was the OSCE time limit (5 min) enough?3.5440.75
11Are you generally satisfied with this ultrasound training?4441
12Do you have any intentions of studying ultrasound more in the future?3.875441
13(only for those who answered positively on 12 to be surveyed) Please fill out the human body parts that you want to learn more about, through ultrasound education, in addition to wrist. (Subjective question)3.5440.75
14Are you willing to use ultrasound in clinical practice after graduation?3.375440.75
15Has your opinion about ultrasound usage changed positively through this education?3.625440.75

CVR, content validity ratio; E, excluded; OSCE, objective structured clinical examination..

Table 5 . Analysis of the Expert Opinions on the Draft Questionnaire for Self-Evaluation After Ultrasound Training..

No.Survey questionsAverageMedianModeCVR
1I actively participated in ultrasound training.3.875441
2I was considerate of the subject during the ultrasound examination.3.625440.75
3I communicated appropriately with the subject during the ultrasound examination.3.875441
4I can use basic functions of ultrasound such as ultrasonic probe selection, depth and focus control, and freeze.3.75441
5I can identify the anterior structure of the wrist through ultrasound.3.75441
6I can do ultrasound scans according to the criteria for diagnosing carpal tunnel syndrome.3.375440.75

The draft version and the final version are the same..

CVR, content validity ratio..

References

  1. Harden, RM, Lilley, P, and Patricio, M. The definitive guide to the OSCE: The Objective Structured Clinical Examination as a performance assessment. Edinburgh (UK): Elsevier; 2016. p. 13-16, 130-134, 213-322
  2. Harden, RM, Stevenson, M, Downie, WW, and Wilson, GM. Assessment of clinical competence using objective structured examination. BMJ 1975;1:447-451.
    Pubmed KoreaMed CrossRef
  3. Park, H-K. The impact of introducing the Korean Medical Licensing Examination clinical skills assessment on medical education. J Korean Med Assoc 2012;55:116-123. [in Korean]
    CrossRef
  4. Institute of Korean Medicine Educationand Evaluation. The manual of 2nd cycle Korean medicine education evaluation and certification for College of Korean Medicine and Graduate School of Korean Medicine. Seoul (Korea): Karammunhwasa; 2017. 9 p.
  5. Institute of Korean Medicine Education and Evaluation [Internet]. Status of evaluation and certification by year [cited 2022 Jan 31]. Available from: https://www.ikmee.or.kr/html_2016/02/03.php. [in Korean]
  6. Institute of Korean Medicine Education and Evaluation. KAS2021 Amendment Announcement Public Hearing. Seoul (Korea): Institute of Korean Medicine Education Clinical Performance Examination Evaluation; 2021. p. 14-15.
  7. Cho, E, Lee, J-H, Kwon, OS, Hong, J, and Cho, NG. Assessment of Acupuncture and Moxibustion Medicine Clinical Practice Using the Objective Structured Clinical Examination. J Acupunct Res 2021;38:219-226.
    CrossRef
  8. Akom news [Internet]. Preparing a ‘place of communication’ on current issues of Korean medicine education such as unification and reorganization of licensing examination. 2019 [cited 2021 Dec 6]. Available from: http://www.akomnews.com/bbs/board.php?bo_table=news&wr_id=36073.[in Korean]
  9. Kim, JH. The effects and challenges of clinical skills assessment in the Korean medical license examination. Korean Med Educ Rev 2013;15:136-143. [in Korean]
    CrossRef
  10. Sim, SB, Kweon, JH, Kim, HW, Hong, JW, and Shin, SW. Student Satisfaction Study of Clinical Skills Training in Korean Medical Education. J Korean Med 2013;34:37-53. [in Korean]
    CrossRef
  11. Kang, YS, Chae, YB, Go, HY, Kim, HH, Kim, HS, and Hong, JS. A study for improvement of the national examination system for Korean medicine doctors. Seoul (Korea): Korea Health Personnel Licensing Examination Institute; 2017. p. 80-81. Report No.: RE02-1802-1803 [in Korean]
  12. Go, HI, Cho, E, Jung, H-J, Lee, S, Kim, J-H, and Kwon, OS. What is the source of pressure pain during abdominal examination in Korean medicine?. Integr Med Res 2020;9:100406.
    Pubmed KoreaMed CrossRef
  13. Chu, H, Kim, J, Ha, W, Cho, E, Kang, G, and Park, S. Change of Safe Needling Depth at Acupoint GB21 according to Posture and Breathing. Evid Based Complement Alternat Med 2018;2018:2308102.
    Pubmed KoreaMed CrossRef
  14. Chu, H, Park, K, Jang, J, Shin, H-R, Yang, M, and Kim, DR. Comparison of Cervical Hyeopcheok Acupoint’s Depth Change with Cervical Rotation in a Prone Position: Protocol and Pilot Study Result for Accurate Observation of Cervical Hyeopcheok Acupoint by Ultrasonography. J Korean Med Rehabil 2019;29:81-88. [in Korean]
    CrossRef
  15. Ahn, T-S, Moon, J-H, Park, C-Y, Oh, M-J, and Choi, Y-M. The Effectiveness of Ultrasound-Guided Essential Bee Venom Pharmacopuncture Combined with Integrative Korean Medical Treatment for Rib Fracture: A Case Study. J Korean Med Rehabil 2019;29:157-163. [in Korean]
    CrossRef
  16. Kim, S, Lee, S, Ha, W-B, Lee, J-H, Jung, H-J, and Chu, H-M. Development of an ultrasound-imaging procedure and acquisition of ultrasound images of acupuncture points for safety and accuracy of needle insertion. Integr Med Res 2017;6:427-433.
    Pubmed KoreaMed CrossRef
  17. Yoon, S-H, Kim, S-A, Lee, G-Y, Kim, H, Lee, J-H, and Leem, J. Using magnetic resonance imaging to measure the depth of acupotomy points in the lumbar spine: A retrospective study. Integr Med Res 2021;10:100679.
    Pubmed KoreaMed CrossRef
  18. Diep, D, Chen, KJQ, and Kumbhare, D. Ultrasound-guided interventional procedures for myofascial trigger points: a systematic review. Reg Anesth Pain Med 2021;46:73-80.
    Pubmed CrossRef
  19. Korean Medicine Diagnostic Practice Compilation Committee. Practice in diagnostics of Korean medicine, 2nd ed. Seoul (Korea): Koonja Publish Co; 2017. p. 51-54. [in Korean]
  20. Editing Textbook of the Acupuncture. Practice of Acupuncture; A Guidebook for College Students, 7th ed. Daejeon (Korea): Jongryeonamoo; 2015. p. 5-610. [in Korean]
  21. Yang, S-J, Park, K-M, and Cho, S-H. The analysis of satisfaction with clinical training and the related factors in the Korean medicine obstetrics and gynecology. J Korean Obstet Gynecol 2020;33:116-127. [in Korean]
  22. Musculoskeletal sonography examination - MSKS / ARDMS [Internet]. 2021 [cited 2021 Dec 23]. Available from: https://www.ardms.org/get-certified/rmsks/musculoskeletal-sonographer-msks/.
  23. Chung, SH, Lee, HJ, Kim, HS, and Oh, J-Y. Health Insurance Benefit Criteria and Quality Assurance Policies of Diagnostic Ultrasound Services in Other Countries. Health Policy Manag 2014;24:109-119. [in Korean]
    CrossRef
  24. Lee, S-H, Park, H-J, Kim, H-T, Park, S-Y, Heo, I, and Hwang, M-S. Clinical Practice Patterns for Carpal Tunnel Syndrome in Korean Medicine: An Online Survey. Korean Soc Chuna Man Med Spine Nerves 2021;16:73-89. [in Korean]
    CrossRef
  25. Bianchi, S, and Martinoli, C. Ultrasound of the musculoskeletal system. Seoul (Korea): Epublic; 2009. p. 21-27, 46, 48, 55, 98, 100, 448 [in Korean]
  26. Association of Pain and Function Analysis Research group of ultrasound. Muscular ultrasound for pain physician. Seoul (Korea): Medianbook; 2019. p. 6-9. [in Korean]
  27. Han, JH, Choi, SY, Jung, SY, Jang, YI, Lim, CS, and Lee, HS. Ultrasonography Imaging, 2nd ed. Seoul (Korea): MDWorld; 2013. p. 11, 15 [in Korean]
  28. Korean Society of Musculoskeletal Radiology. Musculoskeletal ultrasound, 2nd ed. Paju (Korea): Koonja publish Co; 2018. p. 5-10, 14-15, 494, 591 [in Korean]
  29. Korean Orthopedics Ultrasound Society. Orthopaedic ultrasound. Paju (Korea): Koonja Publish Co; 2019. 78 p. [in Korean]
  30. Wong, SM, Griffith, JF, Hui, ACF, Lo, SK, Fu, M, and Wong, KS. Carpal Tunnel Syndrome: Diagnostic Usefulness of Sonography. Radiology 2004;232:93-99.
    Pubmed CrossRef
  31. Baltarowich, OH, Di Salvo, DN, Scoutt, LM, Brown, DL, Cox, CW, and DiPietro, MA. National Ultrasound Curriculum for Medical Students. Ultrasound Q 2014;30:13-19.
    Pubmed CrossRef
  32. Knobe, M, Münker, R, Sellei, RM, Holschen, M, Mooij, SC, and Schmidt-Rohlfing, B. Peer teaching: a randomised controlled trial using student-teachers to teach musculoskeletal ultrasound. Med Educ 2010;44:148-155.
    Pubmed CrossRef
  33. Kessler, C, and Bhandarkar, S. Ultrasound training for medical students and internal medicine residents-A needs assessment. J Clin Ultrasound 2010;38:401-408.
    Pubmed CrossRef
  34. Ma, I, Wishart, I, Kaminska, M, McLaughlin, K, Weeks, S, and Lautner, D. Medical educators’ perspectives of teaching physical examinations using ultrasonography at the undergraduate level. Can Med Educ J 2013;4:e59-68.
    CrossRef
  35. Gulati, G, Alweis, R, and George, D. Musculoskeletal ultrasound in internal medicine residency - a feasibility study. J Community Hosp Intern Med Perspect 2015;5:27481.
    Pubmed KoreaMed CrossRef
  36. Kyunghee University College of Korean Medicine Clinical Skills Compilation Committee. Korean medicine clinical skills OSCE. Seoul (Korea): Korea Medical Book Publishing Company; 2019. p. 12-20. [in Korean]
  37. Pacific Academy of Sciences 5 minutes OSCE, 5th ed. Seoul (Korea): Pacific Books; 2018. p. 303-309. [in Korean]
  38. Lawshe, CH. A quantitative approach to content validity. Pers Psychol 1975;28:563-575.
    CrossRef
  39. Lee, E-H. Psychometric property of an instrument 1: content validity. Korean J Women Health Nurs 2021;27:10-13. [in Korean]
    Pubmed KoreaMed CrossRef
  40. Moore, CL, and Copel, JA. Point-of-Care Ultrasonography. N Engl J Med 2011;364:749-757.
    Pubmed CrossRef
  41. Tarique, U, Tang, B, Singh, M, Kulasegaram, KM, and Ailon, J. Ultrasound Curricula in Undergraduate Medical Education: A Scoping Review: Ultrasound Curricula in Undergraduate Medical Education. J Ultrasound Med 2018;37:69-82.
    Pubmed CrossRef
  42. Kim, BK, Cha, JM, and Song, DS. How to Revitalize the Abdominal Ultrasonography Education Program. Korean J Gastroenterol 2019;73:66-69. [in Korean]
    CrossRef
  43. Criscione-Schreiber, LG, Sloane, RJ, Hawley, J, Jonas, BL, O’Rourke, KS, and Bolster, MB. Expert Panel Consensus on Assessment Checklists for a Rheumatology Objective Structured Clinical Examination: Consensus on Rheumatology OSCE Assessment Checklists. Arthritis Care Res 2015;67:898-904.
    Pubmed CrossRef
  44. Patrawalla, P, Eisen, LA, Shiloh, A, Shah, BJ, Savenkov, O, and Wise, W. Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound. J Grad Med Educ 2015;7:567-573.
    Pubmed KoreaMed CrossRef
  45. Millington, SJ, Arntfield, RT, Guo, RJ, Koenig, S, Kory, P, and Noble, V. The Assessment of Competency in Thoracic Sonography (ACTS) scale: validation of a tool for point-of-care ultrasound. Crit Ultrasound J 2017;9:25.
    Pubmed KoreaMed CrossRef
  46. Wright, SA, and Bell, AL. Enhancement of undergraduate rheumatology teaching through the use of musculoskeletal ultrasound. Rheumatology 2008;47:1564-1566.
    Pubmed CrossRef
  47. Kissin, EY, Niu, J, Balint, P, Bong, D, Evangelisto, A, and Goyal, J. Musculoskeletal Ultrasound Training and Competency Assessment Program for Rheumatology Fellows. J Ultrasound Med 2013;32:1735-1743.
    Pubmed CrossRef
  48. Kim, H, Jeon, P, Kim, S, Hong, J, and Kang, Y. Evid Based Complement Alternat Med; 2021. 5591911 p.
    Pubmed KoreaMed CrossRef
  49. Joo, S, Eom, D, and Song, J. A research for recognitions of Korean medical student toward traditional Korean medicine and education. J Soc Prev Korean Med 2016;20:1-8. [in Korean]
  50. Park, S, Choi, E, and Kim, S-Y. A Study on the Korean Medical Students’ Perception of Medical Ethics Education Using Flipped Learning. J Soc Prev Korean Med 2018;22:1-12. [in Korean]
  51. Kwon, SW, Shin, SW, and Lim, B. A survey of students’ satisfaction with education in traditional Korean medicine. J Korean Orient Med 2012;33:1-11. [in Korean]
  52. Jo, H-J, Park, J-S, Roh, JD, and Sung, HK. A Survey on Students’ Perception of Clinical Performance Examination (CPX) in College of Korean Medicine Using Student Standardized Patients. J Soc Prev Korean Med 2020;24:1-13. [in Korean]
  53. Kweon, JH, Sim, SB, Kim, EJ, Hong, JW, and Shin, SW. Evaluation of the Implementation of Clinical Performance Examination in Korean Medicine Education. J Physiol Pathol Korean Med 2018;32:51-61. [in Korean]
    CrossRef
  54. Cha, HY, Kim, NH, Hong, JW, and Shin, SW. Evaluation of the implementation of problem-based learning in Korean medical education. Korean J Orient Physiol Pathol 2012;26:351-359. [in Korean]
    Pubmed KoreaMed CrossRef
  55. Cho, CS. Student Satisfaction Study and Interrater Comparative Study on Patient-Physician Interaction Score of Clinical Performance Examination in Korean Medical Education. Korean J Orient Physiol Pathol 2015;29:152. [in Korean]
    CrossRef
  56. Yang, S-B. Survey research for evaluation of real-time non-face-to-face medical education in clinical Korean medicine. J Intern Korean Med 2020;41:984-992. [in Korean]
    CrossRef
  57. Cho, HW, Hwang, EH, Shin, BC, Sul, JU, Hong, JW, and Shin, SW. The analysis of satisfaction with clinical training and the related factors-especially in oriental rehabilitation medicine-. Korea J Orient Prev Med Soc 2012;16:1-15. [in Korean]
  58. Chang, Y-J, and Hong, S-U. The Efficiency of Objective Structured Clinical Examination (OSCE) as a Method of Clinical Clerkship in the Korean Ophthalmology and Otolaryngology Department. J Korean Med Ophthalmol Otolaryngol Dermatol 2014;27:121-130.
    CrossRef
  59. Lim, KS, Lee, JC, and Park, YG. The result of oriental medical ophthalmology and otolaryngology class applying student-oriented teaching and learning method. J Korean Med Ophthalmol Otolaryngol Dermatol 2010;23:163-173. [in Korean]
  60. Rowe, G, and Wright, G. [Internet]. Expert Opinions in Forecasting: The Role of the Delphi Technique. 2001 [cited 2021 Dec 16]. Available from: http://link.springer.com/10.1007/978-0-306-47630-3_7.
    CrossRef
JAR
Aug 01, 2024 Volume 41:143~208
Current Issue Archives

Stats or Metrics

Share this article on

  • line

Related articles in JAR

Journal of Acupuncture Research

pISSN 2586-288X
eISSN 2586-2898
qr-code Download