Quality Assessment and Implications for Further Study of Acupotomy: Case Reports Using the Case Report Guidelines and the Joanna Briggs Institute Critical Appraisal Checklist
Article information
Abstract
This review aimed to evaluate the quality of case reports where acupotomy was performed according to the CAse REport (CARE) guidelines and the Joanna Briggs Institute (JBI) critical appraisal checklist. Case reports on acupotomy published in Korea from 2013 to October 2020 were included in this review. A total of 28 acupotomy related case reports were selected, and a quality evaluation was verified using the CARE guidelines and JBI critical appraisal checklist. Among the case reports, spinal conditions/diseases were most commonly reported. The overall complete reporting rate for each study was relatively high (median of 63.4% according to the CARE guidelines and 73.4% according to JBI critical appraisal checklist for case reports and 62% for case series). However, low reporting rates were determined in several subcategories namely, “Intervention adherence and tolerability,” “Timeline,” “Diagnostic challenges,” “Patient perspective,” and “Adverse or unanticipated events” for case reports, and “Reporting of the presenting site/clinic,” “Demographic information,” “Statistical analysis,” and “Clear criteria for inclusion” for case series. When reporting cases where acupotomy was performed, it is recommended that the CARE guidelines are followed to improve the quality of research. In addition, new guidelines and tools for the clinical situation of Korean medicine should be developed.
Introduction
Case reports provide an account of the symptoms, signs, diagnosis, treatment, results, and follow-up of an individual patient [1]. They often reveal unexpected conditions/diseases, symptoms, or events and unique therapies; therefore, case reports form the basis for medicine and can give new ideas [1,2].
A group of case reports is called a case series and is a collection of several patient accounts who received similar treatments [2]. They are one of the most basic types of study design, and are particularly useful for determining the effect of an intervention or unusual response [3]. Therefore, a quality evaluation of case series (classified as case reports) was performed in this review.
Case reports describe individualized treatments that are difficult to generalize, and observations may be biased. Nevertheless, with medicine focusing on individual characteristics, the interest in case reports is increasing with a corresponding increase in the number of published articles [4]. Well documented case reports can reflect real world practice to improve critical appraisal, improve patient care documentation [5], and provide a basis for designing clinical studies, such as observational studies and randomized controlled trials.
Some guidelines are developed for the purpose of reporting according to the type of study for example the Consolidated Standards of Reporting Trials (CONSORT) for randomized controlled trials [6]; Strengthening The Reporting of Observational studies in Epidemiology (STROBE) for observational studies, such as cohort studies, case-control studies, and cross-sectional studies [7]; and Meta-analysis of Observational Studies in Epidemiology (MOOSE) [8] and Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) as systematic review and meta-analysis [9]; the CAse REport (CARE) guidelines for case reports; and the Joanna Briggs Institute (JBI) critical appraisal checklist for case reports/case series [10–13].
Acupotomy is a modernized acupuncture tool by Professor Zhu in China that combines two of the nine classical needles, Bongchim and Pichim, which originated from stones and ceramics used for acupuncture. Similar to acupuncture, acupotomy is a microinvasive with the use of a small flat scalpel attached to the tip of the acupuncture needle. In the Korean literature, acupotomy is performed to treat diseases ranging from epidermal to deep muscle lesions by cutting and peeling. In particular, during the Joseon Dynasty, acupotomy gained national support and was developed to treat furuncles. It has been used continuously by many Korean medicine doctors since the official medical device registration in 2002. It is easy to perform, effective, and less painful than surgery; therefore, it is actively used to treat chronic pain conditions/diseases by detaching, or incising soft tissues [14–18].
However, it has been reported recently in case reports where acupotomy has been performed, that it can be used to treat not only chronic pain conditions/diseases such as cervical pain, frozen shoulder, lumbar disc herniation, and arthritis, but it can also treat endocrine conditions/diseases such as obesity, congestive heart failure, and irritable bowel syndrome [19–28]. This indicates an increase in the utilization of acupotomy for various conditions/diseases. Therefore, case reports form the basis for clinical research using acupotomy. However, the stimulating method, treatment area, and equipment used in acupotomy techniques are very different from those in acupuncture treatment, and the description of the intervention is poor in case reports where acupotomy was performed. Therefore, there is a barrier to the application of acupotomy in clinical practice and clinical research. Hence, strictly following the contents of the CARE guidelines (the reporting guidelines) and the JBI critical appraisal checklist (the guidelines for quality assessment) will be great help in clinical research.
Therefore, in this review, case reports of acupotomy in Korea using the CARE guidelines and the JBI critical appraisal checklist were evaluated to assess potential improvements for acupotomy research in the future.
Materials and Methods
Search strategy
The term and technique of acupotomy remains controversial in Korea however, a 2018 study by Yoon et al [18] revealed that the following acupotomy terms Chimdochimsul (37 studies, 66.07%), Dochim (23 studies, 41.07%) are the most widely used in Korea. In English, acupotomy (46 studies, 76.66%), and miniscalpel acupuncture (10 studies, 16.66%) are most widely used. Accordingly, the following search topics: “Acupotomy,” “Miniscalpel Acupuncture,” “Chimdochimsul in Korean,” “Dochim in Korean” we selected. We retrieved articles published in Korea before October 2020 using 6 databases [Oriental medicine Advanced Searching Integrated System (OASIS), Korean Studies Information Service System (KISS), Research Information Sharing Service (RISS), Korea Citation Index (KCI), Science ON, and Korean Medical database (KMBASE)].
Case reports containing the words “case,” “case report,” “case study,” and “case series” in the title were selected 1st, and all abstracts were checked. Considering the year of publication of the CARE guidelines, only case reports published after 2013 were selected for the final study population, and studies not related to acupotomy were excluded after the original text had been checked.
Data collection and quality evaluation
The CARE guidelines, published in the CARE statement in 2013, comprised 13 topics and 30 sub-items. However, Riley et al [5] described the quality assessment method in 2017, and subsections of abstract information and patient information were combined and changed to 13 topics and 28 sub-items. For accurate quality assessment, case reports were evaluated using the items stated by Riley et al. Although the CARE guidelines have limitations in that it is not a tool to evaluate the quality of the report, it is a reporting guideline. Therefore, an additional evaluation was performed using the JBI critical appraisal checklist.
The JBI critical appraisal checklist is a quality assessment method created with the agreement of experts. The case report checklist was comprised of a total of 8 sub-items, and the case series checklist contains 10 sub-items [11–13]. In this study, we verified case report and case series checklists to increase the accuracy of the evaluation.
Two reviewers read the final selected case reports and case series and evaluated them using the CARE guidelines and the JBI critical appraisal checklist. The 1st evaluation was conducted using the CARE guidelines, with each case report evaluated as “Reported-Sufficient” if it was completely reported; “Reported-Not sufficient” if it was reported, but insufficiently; “Not reported” if it was not reported; and “Not applicable” if it could not be applied. The study was re-evaluated using the JBI critical appraisal checklist. If it was clearly described, it was evaluated as “Yes,” “No” if it was not presented, “Unclear,” if it was not clear, and “Not applicable” if it could not be applied. Disagreements were resolved by discussion and further disagreements were resolved through the judgment of a 3rd researcher.
Analysis of characteristics of case reports and case series
The characteristics of the included studies have been reported. Condition/disease, acupotomy terms used, and acupotomy needle were investigated. In addition, condition/disease was divided into musculoskeletal and other conditions/diseases, and the number of published studies was counted in each condition/disease category
Analysis of quality evaluation results
For quality evaluation analysis, each case report and case series were 1st evaluated with the 28 sub-items of the CARE guidelines. After classifying the studies as described earlier, this data was converted to a percentage. For this, the maximum, minimum, and median values were determined. The number of studies in each of the 28 sub-items of the CARE guidelines was then evaluated and classified.
The 2nd evaluation and classification was made using the JBI critical appraisal checklist and the number of sub-items were converted to percentages; the maximum, minimum, and median values were checked. In each of the sub-items, the number of studies evaluated as “Yes,” “No,” “Unclear,” and “Not applicable” were analyzed.
Following an evaluation of adherence to the CARE guidelines and the JBI critical appraisal checklist, sub-items with a case report rate exceeding 50% were classified as “Not reported” and “Not sufficient,” and were analyzed to determine whether improvement was required in the future.
Results
Study selection
A total of 765 articles were retrieved and 668 duplicate studies were removed. The titles and abstracts of the remaining 97 studies were reviewed. Among them, 52 case reports and case series were selected. This excluded 7 protocol and pilot trial studies, 6 clinical trials, 2 controlled clinical trials, 4 randomized controlled trials, 3 retrospective studies, 10 original articles, 1 animal research article, 11 review articles, and 1 article without full text. On the 52 selected studies, 29 were published from 2013 onwards. There was 1 case report unrelated to acupotomy which was excluded leaving 28 case reports and case series where acupotomy was performed (Fig. 1). Among them, 23 cases identified case reports in the title and 5 were case series. The characteristics of each case report, conditions/diseases, acupotomy needle size, and terms used are summarized in Tables 1 [23–50] and 2 [23–50].
A flow chart of study selection according to the preferred reporting items for systematic reviews and meta-analyses.
Characteristics of Case Report and Case Series on Acupotomy.
Classification of the Conditions/Diseases.
Quality assessment as per sub-items of the CARE guidelines
Level of quality of the case reports and case series
On analyzing each case report and case series based on the evaluation criteria for sub-items of the CARE guidelines, it was determined that there were case reports classified as “Sufficient” and “Not sufficient” in the “Reported” section, with a maximum of 92.6%, a minimum of 71.4%, and a median of 79.6%. Overall, the reporting level was high, however, “Reported-Sufficient” showed a maximum of 88.9%, minimum of 48.1%, and median of 63.4%. Among the case reports and case series classified as “Reported-Not sufficient,” when only partially reported, the report rate was 30.8% at maximum, 3.7% at minimum, and 16.2% at median. The “Not reported” rate was 28.6% at maximum, 7.4% at minimum, and 20.4% at median (Table 3 [23–50]).
Classification of Each Case Report and Case Series as per the CAse REport Guidelines.
Level of quality according to the 28 sub-items
Evaluation according to the checklist criteria for each of the 28 sub-items, revealed that Sub-item 10c (Intervention adherence and tolerability) was not reported in any case report. Additionally, Sub-items 7, “Depict important dates and times in the case” (85.7%); 8b, “Diagnostic challenges, (85.7%); and 12, “The patient should share their perspective or experience whenever possible” (78.6%), were evaluated as “Not reported” in more than 50% of the studies.
Among the sub-items classified as “Reported-Not sufficient” and “Not reported,” Sub-items 2, “The key elements of this case in 2–5 words” (78.6%); 5a, “Demographic information of the patient” (85.7%); 5c, “Medical, family, and psychosocial history” (96.4%); and 10b, “Important follow-up test results” (71.4%) were insufficiently reported.
Sub-items 8d, “Prognostic characteristics (e.g., staging), where applicable” and 9c, “Changes in intervention (with rationale)” were not applicable for 13 studies and 23 studies, respectively; therefore, they were evaluated as “Not applicable” and excluded (Table 4; Fig. 2).
Classification of Each Case Report and Case Series According to the CAse REport Guideline Sub-Items.
Classification of the studies as per CARE guideline sub-items. CARE, CAse REport.
Quality assessment for sub-items of the JBI critical appraisal checklist
Level of quality of the case reports and case series
On analyzing the 23 case reports based on the evaluation criteria of the sub-items of the JBI critical appraisal checklist, the report rates of case reports classified as “Yes” were a maximum of 100%, minimum of 50%, and median of 73.4%. The case reports classified as “No” had a maximum report rate of 50%, minimum of 0%, and median of 26.1%. Analysis of the 5 case series revealed the report rates classified as “Yes” to be a maximum of 80%, minimum of 40%, and median of 62%. Those classified as “No” had a maximum report rate of 60%, minimum of 20%, and median of 32% (Table 5 [23–50]).
Classification of Each Case Report and Case Series According to the Joanna Briggs Institute Critical Appraisal Checklist.
Level of quality according to the 8 sub-items for case reports and the 10 sub-items for case series
Evaluation according to the checklist criteria for each of the 8 sub-items revealed that Sub-item 2, “Was the patient’s history clearly described and presented as a timeline?” was not clearly reported in 82.6% of the case reports. The 2nd lowest reporting rate (52.2%) was for Item 7, “Were adverse events (harms) or unanticipated events identified and described?”
The sub-items “Was the post-intervention clinical condition clearly described?” reported in 100% of the case reports and “Does the case report provide takeaway lessons?” in 95.7% of the reports, had the highest reporting rates.
On evaluating the case series according to the checklist criteria for each of 10 sub-items, the following sub-items were not reported: 9, “Was there clear reporting of the presenting site(s)/clinic(s) demographic information?” (100%); 10, “Was statistical analysis appropriate?” (80%); and 1, “Were there clear criteria for inclusion in the case series?” (60%). Sub-item 5, “Did the case series have complete inclusion of participants?” was unclear in 60% of the case series (Tables 6 and 7).
Classification of Each Case Report According to the Joanna Briggs Institute Critical Appraisal Checklist Sub-Items.
Classification of Each Case Series According to the Joanna Briggs Institute Critical Appraisal Checklist Sub-Items.
Discussion
On analyzing 28 case reports since 2013 where acupotomy was performed, there were 9 case studies of spinal conditions/diseases reviewed. Among them, 6 studies discussed herniated intervertebral discs. Carpal tunnel syndrome was the 2nd most commonly reported terminology, with a total of 3 published studies. In case reports where acupotomy was performed, “Intervention, adherence, and tolerability,” “Diagnostic challenges,” “Patient perspective,” “Timeline,” “Adverse events or unanticipated events” were insufficiently reported. Case series reporting the “Presenting site(s)/clinic(s) demographic information,” “Statistical analysis,” “Clear criteria for inclusion” and “Complete inclusion of participants” were lacking.
Compared with other articles, quality assessment studies applying the CARE guidelines, the median percentage of sufficient reporting was 66.7% in the Journal of Sasang Constitutional [51], 61.54% in the Journal of Pediatrics of Korean Medicine [52], 39.3% in the Journal of Oriental Neuropsychiatry [53], 61.54% in the Journal of Korean Medicine [54], 69.23% in the Journal of Obstetrics and Gynecology of Korean Medicine [55], 62.5% in the Journal of Korean Medicine for Obesity Research [56], and 34.78% in the Journal of Korean Medicine Rehabilitation [57]. The reporting rate of case reports where acupotomy was performed (63.4%) was higher than the articles in the Journal of Oriental Neuropsychiatry and the Journal of Korean Medicine Rehabilitation, but similar to other articles which exceeded 60%.
However, with a recent interest in personalized treatment, the number of case reports published in medical journals is also increasing [4], and the number of journals focusing only on case reports (such as British Medical Journal Case Reports and BioMed Central Journal of Medical Case Reports) are also increasing [58–60]. Accordingly, to improve the value of case reports, in addition to accurately and transparently prepared and critically evaluated case reports, a checklist was formulated through agreement between experts [5]. This checklist has been translated into various languages, and has been established worldwide as the CARE guidelines [61]. For improving the quality of case reports published in Korea, the CARE guidelines were also translated and distributed in Korea in 2015 [62].
Acupotomy is being actively studied as a modern acupuncture therapy [63]. Therefore, with the publication of the CARE guidelines in 2013, this review of the overall quality of case reports began by evaluating the adherence to the CARE guidelines observed in Korean published case reports where acupotomy was performed and adhered to. Following the CARE guidelines, analysis was also performed using the critical appraisal checklist published by the JBI. This organization distributes evidence-based healthcare guidelines to nursing, midwifery, medicine, and allied health specialists in more than 80 collaborative centers in more than 90 countries. The checklist created by JBI and collaborators has been approved by the JBI Scientific Committee following a thorough peer review [11–13].
Compared with the other 7 articles in Korean medicine, quality assessment using the CARE guideline subcategories showed that, overall, insufficient reporting rates were similar. In particular, “Intervention adherence and tolerability” and “Diagnostic challenges” were not reported by more than 90% of case reports in 6 articles in Korean medicine. However, in the case reports describing acupotomy, the reporting rate of “The informed consent” and “Adverse events or unanticipated events” was higher than in the other 7 articles. Thus, it could be confirmed that the Korean medicine doctor is receiving consent from the patient and reporting adverse events relatively well. It is expected that a clear description of intervention adherence and tolerability, diagnostic challenges, patient perspective, timeline, adverse events, or unanticipated events, should be provided in a case report or other clinical studies on acupotomy in the future.
Adherence to CARE guideline Item 10c, “Intervention adherence and tolerability,” is a measure of whether an individual’s behavior, such as consuming drugs, lifestyle habits, or visiting a hospital, is consistent with the recommendations of the medical staff [64]. Tolerability refers to the degree to which side effects or discomfort are tolerated during treatment. Alternatively, if the tolerability is good, it can be expected that the degree of adherence will increase and the treatment effect will increase. However, adherence and tolerability related to acupotomy were not reported in the 28 case reports assessed. Acupotomy is an invasive treatment rather than a medication therefore, adherence may decrease due to side effects, which should be reported. Compared with the evaluations reported by other quality assessment articles (100% of the Journal of Sasang Constitutional [51] and Pediatrics of Korean medicine [52] reports, and 69.2% of the Journal of Oriental Neuropsychiatry [53], 93.94% of the Journal of Korean Medicine [54] and 92.68% of the Journal of Obstetrics and Gynecology of Korean Medicine [55] reports), the rate of evaluations not reported was high, as well as case reports where acupotomy was performed.
The 8b category, “Diagnostic challenges,” includes cases in which an accurate diagnosis is made only after condition/disease development, those cases in which the diagnosis method has not been developed, those with diagnostic challenges such as restriction on the use of diagnostic equipment due to financial constraints, or those with communication restrictions due to language and cultural differences. Only 1 case report described the limitations and difficulties of the diagnostic method; 3 case reports evaluated them as “Not sufficient.” This is also related to the fact that it is not easy to use diagnostic medical devices in Korean medical institutions. Acupotomy has a higher risk of side effects, suggesting that imaging should be more actively performed.
Item 12, “Patient perspective,” aims to convey to the readers the following: the patient’s reasons for seeking treatment, what changed after treatment, and what changed in their quality of life. As changes occur from doctor-centered medicine to patient-centered medicine, sharing opinions from the patient’s point of view is considered important, but has not yet been described in detail in case reports where acupotomy has been performed. In particular, new acupuncture therapies, such as acupotomy, which are not yet familiar to patients, need to share patients’ perspectives by expressing their opinions on treatment.
CARE guideline Item 7 and JBI critical appraisal checklist Item 2, depict important dates and times as a timeline to enable easy perusal of the patient’s history. It includes core elements such as the patient’s medical and family history, diagnostic evaluations, therapeutic intervention, care received from other clinicians, follow-up, and outcome. The more recently published the case report, the higher the rate of reporting timelines. Providing this information has improved clinicians’ awareness of the crucial timelines required for recording treatment and patient parameters.
JBI critical appraisal checklist Item 7 assesses whether adverse or unanticipated events have been described after the intervention; it is an important item in a case report, especially when using a new or unique treatment. However, no more than 50% reported this item in case reports where acupotomy was performed, similar to the reporting rate of the CARE guidelines Item 10d (53.6%), indicating that currently there is limited information on the side effects of acupotomy. The acupotomy size reported in this case report was 5–10.5 cm length, with a diameter of 0.5–1.2 mm. This is thicker than that used in general acupuncture, thus, side effects, including damage to the nerves or blood vessels, bleeding [65], or unanticipated events, may be more common. Therefore, it is necessary to provide a detailed description of side effects in case reports where acupotomy was performed.
By JBI critical appraisal checklist subcategory for case series, Item 9, for each condition/disease, has various demographic characteristics, and the population of the case series should reflect this. Therefore, it should be described in detail so that other researchers can confirm that this has been reflected. However, not all case series evaluated in this study has described this item. This is because acupotomy case series had an average of 8.2 participants (range, 2–24); thus, it is difficult to apply demographic characteristics. Item 10 evaluated the appropriateness of statistical means by asking which statistical analysis was used when analyzing the final result and whether this method was suitable for deriving results. However, only 1 statistical method was used, and the remaining case series reported the results without information on statistical analysis. Item 1 suggested the criteria for inclusion and exclusion when recruiting participants. The case series was a group of patients with similar characteristics; hence, it was necessary to present a definite inclusion and exclusion criteria when recruiting a patient. This reporting was still insufficient to establish and describe certain criteria in the most basic patient population. In particular, the description of the exclusion reference point was insufficient.
Item 5 asked whether the case series indicated a complete inclusion of participants. If only patients with therapeutic effects are extracted, the reliability of the study is low. Thus, all patients with the stated inclusion criteria who visited the clinic during the consecutive period should have been included. Approximately 80% of the case series described the question about Sub-item 4—regarding whether the population was collected consecutively. However, the question of complete inclusion was evaluated as being insufficiently reported. Therefore, to increase the reliability and quality of treatment interventions, it must be described completely.
Acupotomy has been widely used in recent years, but the tools, treatment points, and stimulation methods used for intervention are more diverse than in manual acupuncture and have not yet been standardized. In order to increase their utilization and reproducibility in research, detailed descriptions of interventions are required.
Of the case reports that were finally selected for analysis, a median of 79.6% [range (maximum-minimum), 71.4–92.6%] case reports “Reported” the items in the CARE guidelines. When each item in the checklist was subdivided and evaluated as “Reported-Sufficient,” “Reported-Not sufficient,” and “Not reported,” a median of 63.4% [range (maximum-minimum), 48.1–88.9%] case reports could be classified as “Reported-Sufficient.” Using the JBI critical appraisal checklist among 23 case reports and 5 case series, the clearly described reporting rate was found to be 73.4% and 62% at the median, respectively.
This study was the 1st study to evaluate the quality of the case reports/series using the CARE guidelines and the JBI critical appraisal checklist where acupotomy was performed. In particular, we applied the JBI checklist for quality assessment, which was not performed in the previous study for quality evaluation of Korean Medicine case reports. This assessment strategy will play an important role in improving the quality of future acupotomy studies. However, this study has the following limitations. Firstly, as a guideline for reporting on general case reports, there is a limit to apply uniformly to Korean medicine, especially to acupotomy. Therefore, the modification of standards for reporting interventions in clinical trials of acupuncture (STRICTA) [66] for the development of a new tool for proper reporting of acupotomy cases should be considered. It is also necessary to develop reporting guidelines and quality assessment tools for qualitative evaluation of case reports on Korean medicine interventions. Secondly, although 2 authors conducted a quality assessment through discussion, with consensus, it is difficult to accurately assess the items because subjective opinions may have been introduced. In addition, the CARE guidelines are limited because it is a guideline for reporting, not a tool for quality assessment. However, the JBI checklist was devised for quality evaluation, which helped to improve the accuracy of the evaluation analysis. Thirdly, only case reports published after 2013 (following the development of the CARE guidelines) were evaluated in this review. However, despite these limitations, it was worthwhile to evaluate the quality of acupotomy research to confirm the characteristics of case reports and case series published since 2013.
Conclusion
If a clear and thorough case report is provided, the intervention can be reproduced and utilized in actual clinical practice. In particular, a detailed description of acupotomy is required, as it is a modern acupuncture therapy that has been actively researched but standardized techniques have not yet been established. Therefore, in this study, quality assessment was performed using the CARE guidelines and the JBI critical appraisal checklist, and compared with other quality assessment studies. Therefore, insufficient reporting parts were identified in case reports where acupotomy was performed.
Based on this study, we recommend following the CARE guidelines, which are reporting guidelines, for future case reports where acupotomy will be performed. While documenting a case series, as required by the items of the JBI checklist, validity and accuracy must be complied with to improve the quality of research and reduce the risk of bias. Additionally, new guidelines and reporting intervention tools for acupotomy must be developed and tailored to fit the clinical situation of Korean medicine.
Notes
Conflicts of Interest
The authors have no conflicts of interest to declare.
Funding
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (no.: NRF-2019R1F1A1059310). This work was also supported by grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (Grant No. HF20C0173).
The funding source will have no input on the interpretation or publication of the study results.