Outcome Assessment for a Telemedicine-Based Second Opinion Program for Midwest China.

Through an examination of the records of a telemedicine-based second opinion program in county-level hospitals in central and western China, the impact of this service on the diagnosis and treatment of cancer-related diseases was analyzed and evaluated. In this study, all 135 cancer-related cases were included in the analysis. The basic characteristics of the patients were described, the opinions of the original and second diagnosis and treatment were compared, the rate of consistency between them was calculated, the therapeutic regimens were analyzed and the differences between groups were tested. In 94.07% of the cases, the reason for the doctor's request for second opinion service was to assist in the formulation of therapeutic regimen. 64.44% of cases were confirmed with the diagnosis and 17.78% therapeutic regimen by the second opinion service. 126 cases obtained improved therapeutic regimens, and there were statistically significant differences in treatment methods in the diagnosis changed group. Comparing with other international SO studies, the diagnostic consistency rate obtained in this study was lower but not the lowest. The therapeutic consistency rate was quite low, due to the high proportion of original therapeutic regimens missing. This telemedicine-based second opinion program has brought beneficial improvements to the diagnosis and treatment of cancer-related diseases in county-level hospitals in central and western China.

What do we already know about this topic?

The Second Opinion (SO) project is usually carried out among medical institutions with similar technical level in developed countries. The usual function of SO is to revise and improve the diagnosis and treatment opinions of rare and complicated diseases. But there were few SO projects held in China and the evaluation of its effect are even less.

How does your research contribute to the field?

This study focuses on revealing the gap between county-level hospitals in the central and western regions of China and the top hospitals in the first-tier city in terms of tumor diagnosis and treatment capacity, and also works as a reflection of the insufficient capacity on diagnosis and treatment of severe diseases of county-level hospitals.

What are your research’s implications toward theory, practice, or policy?

This research reveals the defects in the ability of cancer diagnosis and treatment of county-level hospitals in central and western China, especially in the formulation of treatment plans. It can indicate the key points that should be paid attention to in the future capacity-building policy of county-level hospitals.

Research Background

In recent years, the Chinese government has put forward higher requirements for the capacity of diagnosis and treatment services of county-level hospitals, which are responsible for the diagnosis and treatment of a large number of common diseases at the grass-roots level. The Instructions of the General Office of the State Council on Promoting the Construction of Hierarchical Medical System clarify the functional orientation of the health services of county hospitals and guide all government authorities to strengthen the comprehensive capacity building of county hospitals.[1] On this basis, the National Health and Family Planning Committee has formulated and promulgated the Basic Standards for Medical Service Ability of County Hospitals and Recommended Standards for Medical Service Ability of County Hospitals, which provide the basic requirements for cancer diagnosis and treatment in county-level medical institution. To be able to undertake the diagnosis and differential diagnosis, clinical staging, surgical treatment and medical treatment of tumor diseases such as lung cancer, gastric cancer, esophageal cancer, colorectal cancer, primary liver cancer, leukemia, breast cancer, meningioma, renal cancer, prostate cancer, cervical cancer and endometrial cancer.[2]

However, in reality, the existing capacity of diagnosis and treatment of cancer-related diseases in county-level medical institutions cannot fully meet the government standards. According to a study by Hua et al,[3] 65.9% of the cancer patients in Shandong Province were diagnosed and 70.0% were treated in county-level medical institutions. Of all patients 42.0% obtained both diagnosis and treatment in county-level medical institutions; however 9.3% of patients chose to visit more than 3 higher-level medical institutions to confirm the diagnosis, mainly because they did not believe the diagnosis results of county hospitals (54.8%) and did not trust the treatment ability of county hospitals (33.3%).[3] Song et al[4] and Dong et al[5] analyzed the origins of cancer patients hospitalized in Beijing and found that 45.81% of patients came from outside Beijing region and their use of surgery and radiotherapy, with high technical and professional experience requirements, were significantly higher than that of local patients.[4,5] Although these researches did not provide direct reasons for the cross-regional health care seeking behavior, draw conclusions from Hua’s study that has been cited above, we also may get a glimpse of the weakness of cancer diagnosis and treatment in the primary medical services level.

With increasingly effective information network construction, telemedicine can more conveniently enhance the capacity of county hospitals services and promote investment in high-quality medical resources.[6] In China, most telemedicine consultations are implemented between hospitals. This inter-institutional consultation essentially is a Second Opinion (SO) service for peer consultation, which also includes 2 other types of projects: Patient-initiated SO program and Payer mandated SO program.[7] SO services are increasingly common in other countries such as Australia. At present, the international evaluation of SO services among peers mostly covers diagnosis inconsistency rate, surgical reduction rate, and the impact on clinical management, cost-effectiveness and satisfaction analysis.[8,9]

In many international SO service evaluation studies, the doctors who make the original diagnosis of tumor diseases usually come from regional medical centers or university hospitals, while the SO services are usually provided by specialized medical centers,[10] and there is little difference in the comprehensive diagnosis and treatment level between these institutions. In this study, the SO service is provided by doctors in national medical centers to doctors in county hospitals in central and western provinces, and there is a big gap between these 2 groups of doctors’ professional skills and experience in diagnosis and treatment. Therefore, the research is carried out in the context of obvious differences in institutional level, which is different from the usual SO program evaluation research. At the same time, the evaluation of diagnosis and treatment ability of county hospitals in China is still relatively rudimentary, so that few studies have focused on the differences in diagnosis and treatment opinions which were made by both types of doctors. This study takes a SO program in China as a sample and evaluates its outcomes and effects on cancer-related cases. It is hoped that the effect of the project implementation can be systematically analyzed and the cancer diagnosis and treatment ability can be quantitatively measured. Also, the role of telemedicine service in diagnosis and treatment guidance can be objectively described, so as to provide scientific evidence for the improvement of the service capacity of county-level medical institutions.

Method

Setting

In 2017, the Beijing Youmai Charitable Foundation set up a public welfare project to provide catastrophic disease SO services to county hospitals in poverty-stricken areas in central and western China. Through telemedicine consultation, the doctors of Youmai Clinic provide SO services for selected cases and offer diagnosis and corresponding treatment advice, based on the patient’s medical records and related examinations and lab test results provided by the local hospital. Doctors who are working in this clinic come from Peking Union Medical College Hospital (designated by the Ministry of Health as Grade III A Hospital, 1 of the national guidance centers for the diagnosis and treatment of severe diseases). All of them, are senior chief physicians, have advanced technical qualifications and many years of clinical experience. Doctors receiving SO services come from county hospitals with intermediate or advanced technical qualifications and less clinical experiences especially in cancer diagnosis and treatment. The procedure of SO service is as follows:

Step 1. A doctor in a county hospital applies for SO service.

Step 2. The clinic physician’s assistant prepares the list of relevant medical records needed for consultation and asks the doctors in county hospitals to gather and upload them.

Step 3. The case and its collected records are assigned to a specialist of Youmai, who will review all the records.

Step 4. Both institutions agree on the consultation time and have telemedicine consultation at the given time. The assigned specialist of the Youmai clinic provides the SO service and makes the consultation record.

The whole working cycle is about 3 to 5 working days and usually, a consultation involves several cases. The SO service is realized through the third party’s free video communication software which is open to the public.

Study Population

This study included all 135 cancer-related cases (out of a total of 1044 consultation cases) involved in the SO program from April 2018 to July 2018. The patients came from 67 county-level hospitals in 18 provinces of central and western China. The study was based on the consultation records of each patient. The records included: province, hospital name, case code, the consultation department, consultation time, consultation purpose, patient’s name, gender, age, initial diagnosis and treatment plan, second diagnosis and treatment adjustment advice given by clinic doctors. The medical outcome of patients and the local doctor’s feedback decisions and reactions were not recorded.

Statistical Analysis

Descriptive analyses using the chi-square test to identify possible significant differences between the original and second opinion were performed using Stata SE version 14.

Results

Patients Characteristics

The majority of patients were male (53.33%) and the mean age at presentation was 57.11 (±16.49 SD) years. Among the tumor sites of patients, lung cancer (n = 28, 20.47%) was the most; laryngeal cancer (n = 1, 0.74%) and bladder cancer (n = 1, 0.74%) were the least. The distribution of other sites is shown in Table 1. The most important reason for seeking SO service was the need for treatment advice (n = 127, 94.07%), followed by help in diagnosis (n = 19, 14.07%) and guidance and improvement of existing treatment regimens (n = 12, 8.89%). Of all the cases, only 50 (37.04%) were diagnosed by local county hospitals, while the rest were diagnosed by other local hospitals or in other provinces.

Table 1.

Study Cases Baseline Characteristics (n = 135).

Characteristics	N (%)
Male	n = 72 (53.33%)
Mean age at presentation	57.11 ± 16.49 years
Lesion location
Lung	n = 28 (20.47%)
Reproductive system	n = 19 (14.07%)
Hepatobiliary system	n = 16 (11.85%)
Breast	n = 13 (9.63%)
Brain	n = 12 (8.89%)
Gastric	n = 12 (8.89%)
Colorectal	n = 9 (6.67%)
Renal	n = 8 (5.93%)
Lymphatic system	n = 6 (4.44%)
Blood and marrow system	n = 5 (3.70%)
Esophagus	n = 3 (2.22%)
Skin	n = 2 (1.48%)
Throat	n = 1 (0.74%)
Pancreas	n = 1 (0.74%)
The purpose of the consultation
Diagnostic confirmation	n = 19 (14.07%)
Confirmation of present treatment regimens	n = 12 (8.89%)
Get therapeutic advice	n = 127 (94.07%)
Original diagnosis made by local county hospitals	n = 49 (36.29%)

Changes Resulting From Second Opinions in Diagnosis and Treatment

The majority of reviews resulted in confirmation (n = 87, 64.44%) of the original opinion for diagnosis, while a small number of cases (n = 9, 6.67%) resulted in changed diagnosis. However 19 cases did not record a prior diagnosis, 14 cases did record neither prior nor second diagnosis, and 6 cases recorded original diagnosis but no second diagnosis. Within the 9 cases whose initial diagnosis was changed, the discrepancies had few impact on 2 patient outcome, a potentially major impact on 3 patient outcomes and had an identical impact on 4 patients (Table 2).

Table 2.

Outcomes After Second Opinion Referral.

Outcomes	Total (n = 135)
Diagnosis
Confirmed	n = 87 (64.44%)
Original diagnosis made by the county hospital	n = 36 (26.67%)
Changed	n = 9 (6.67%)
Original diagnosis made by the county hospital	n = 5 (3.70%)
Others	n = 39 (28.89%)
Therapeutic advice
Confirmed	n = 24 (17.78%)
Changed	n = 43 (31.85%)
No prior advice	n = 59 (43.70%)
Others	n = 9 (6.67%)

A large proportion of cases (n = 59, 43.70%) lacked initial therapeutic advice, only a minority of cases (n = 24, 17.78%) were confirmed, and many cases (n = 43, 31.85%) received changed therapeutic advice. Another 9 cases (6.67%) had other anomalies: 2 of them had the original treatment plan, but they were rejected without any further treatment advice by second evaluation, and 7 cases recorded neither initial nor second treatment advice (Table 2).

Changes in Therapeutic Advice

In 126 cases, suggestions for improvement of treatment regimens were made. Depending on the complexity of the disease and the quality of the initial therapeutic advice, the second treatment regimen for each case involved 1 or more clinical diagnostic and therapeutic methods (Table 3).

Table 3.

Changes in Therapeutic Advice.

Therapeutic advice	Total (n = 126)	Diagnosis confirmed cases (n = 80)	Diagnosis changed cases (n = 9)	Other cases (n = 37)	Chi square (P-value)
Surgery related advice	n = 29 (23.02%)	n = 17 (21.25%)	n = 5 (55.56%)	n = 7 (18.92%)	n = 5.87 (.053%*)
Radiotherapy related advice	n = 10 (7.94%)	n = 8 (10.00%)	n = 0 (0%)	n = 2 (5.41%)	n = 1.57 (.457%)
Chemotherapy related advice	n = 34 (26.98%)	n = 30 (37.50%)	n = 2 (22.22%)	n = 2 (5.41%)	n = 13.34 (.001%***)
Other therapy related advice	n = 74 (58.73%)	n = 52 (65.00%)	n = 2 (22.22%)	n = 20 (54.05%)	n = 6.58 (.037%**)
Medical examination related advice	n = 51 (40.48%)	n = 33 (41.25%)	n = 6 (66.67%)	n = 12 (32.43%)	n = 3.58 (.167%)

P < .1. **P < .05. ***P < .01.

A total of 29 cases received advice on whether to undertake surgery, of which 20 cases received suggestions to undertake surgical treatment (among them, 11 cases had had the original diagnosis confirmed, 2 cases had had the original diagnosis changed, and 7 cases did not record an original diagnosis). In 9 cases the advice after the evaluation was to cancel the surgical treatment (among them, 6 cases had had the original diagnosis confirmed and 3 cases had had the original diagnosis changed).

Ten cases received advice on whether to undertake radiotherapy, of which 8 cases were advised to undertake radiotherapy (among them, 6 cases had had the original diagnosis confirmed, and 2 cases recorded no original diagnosis). Two cases were advised to stop radiotherapy, and the original diagnosis of both cases was confirmed.

A total of 34 cases received advice on whether to undergo chemotherapy or not, of which 29 cases were recommended for chemotherapy (among them, 25 cases had had the original diagnosis confirmed, 2 cases had had the original diagnosis changed, and 2 cases did not record an original diagnosis). About 5 cases were advised to stop chemotherapy, and the original diagnosis of all cases was confirmed.

A total of 74 cases received advice to undertake other treatment besides surgery, radiotherapy, and chemotherapy. Of these, 63 cases were advised to add other types of treatment (among them, 43 cases had had the original diagnosis confirmed, 2 cases had had the original diagnosis changed, and 18 cases did not record an original diagnosis). Eleven cases were advised to reduce or eliminate other types of treatment (among them, 9 cases had had the original diagnosis confirmed, and 2 cases recorded no original diagnosis).

A total of 51 cases received advice on whether to undergo further medical examinations. Of these, 47 cases were advised to undergo additional examinations (among them, 31 cases had had the original diagnosis confirmed, 4 cases had had the original diagnosis changed, and 12 cases did not record an original diagnosis). About 4 cases were advised not to undergo further clinical examinations (among them, 2 cases had had the original diagnosis confirmed, and 2 cases had had the original diagnosis changed).

Pearson’s chi-square test was performed on figures for the treatment recommendations obtained from different diagnostic status groups. There were statistically significant differences in the recommendations regarding surgery, chemotherapy, and other treatment options for the groups whose diagnoses had been changed.

Discussion

This Second Opinion Program Provide a Positive Promotion on the Diagnosis of Tumor Diseases to County Hospitals in Central and Western China

In the international evaluation studies of second opinion projects carried out by other scholars, the consistency rate of diagnosis is mostly between 70% and 90%.[11,12] The lowest rate is in a study on testicular cancer where the consistency rate of diagnosis reported is 60.5%.[13] The diagnostic consistency rate obtained in this study was 64.44%, which was lower than the general level, but not the lowest of all studies. If only the original diagnosis cases (49 cases) given by county hospitals were evaluated, the consistency rate would be 73.47%, which accords with the evaluation results of most studies. However, this result cannot conceal the problems in the diagnostic capacity of county hospitals for cancer diseases. In nearly 30% of the 135 cases, the tumors were not initially diagnosed. The absolute number is even greater than the number of cases diagnosed by the county hospital and confirmed by the SO service. It can be seen that in more than half of the consulting cases the county hospitals have not given a satisfactory diagnosis, which may show that county hospitals in the central and western China have deficiencies in the capacity for cancer diagnosis.

SO Services Play an Important Role in the Revision and Improvement of Therapeutic Regimens for Cancer in County Hospitals in Central and Western China

In this study, less than 20% of the original treatment regimens were confirmed by the SO program. Some of the treatment regimens of the remaining cases needed to be adjusted and changed, but an even larger proportion of the treatment regimens needed to be proposed directly by the SO service experts. Other scholars’ research shows that the proportion of treatment adjustment and improvement involved in the SO program is mostly between 30% and 50%.[14] The 31.85% value of this study is in line with this general range. However, similar to the problem of diagnostic capacity, and even more serious, in 43.7% of all cases, county hospitals were not able to come up with effective original treatment plans, which makes the confirmation rate of original treatment programs very low. It is foreseeable that without the provision of SO services, these cancer patients would face serious problems in their treatment. They may have received inappropriate treatment in county hospitals or they may have gone to other hospitals for treatment.

SO Diagnosis Results Have a Significant Impact on Follow-Up Treatment Methods

In cases where the original diagnosis was confirmed, a high proportion (65%) of patients received adjustments to chemotherapy and other treatments (clinical therapy except surgery, radiotherapy, and chemotherapy), while in cases where the diagnosis was changed, a high proportion (55.56%) of patients received adjustments to surgery advice. This study suggests that diagnostic changes will have a significant impact on the adjustment of surgical treatment options for cancer patients in county hospitals. However, many international studies on SO evaluation have focused on the role of SO in reducing unnecessary elective surgery. For different diseases and projects, the rate of reduction in elective surgery obtained in those studies was generally 10% to 50% of cases.[15-17] Schrader’s study of testicular cancer showed that the rate of reduction of unnecessary surgery was higher than 85.71% in misdiagnosed cases (6 out of 7 misdiagnosed patients had the surgery canceled).[18] In our study, there are both reduction of unnecessary surgery and suggestions for necessary surgical treatment, totaling more than 50% of all patients with diagnostic changes. This means that more than half of the misdiagnosed patients in county hospitals are likely to receive unnecessary surgical treatment or not receive appropriate surgical treatment, which will have a great impact on the clinical outcomes for the patients.

The location and types of tumors have a significant impact on the consistency of diagnosis and treatment advice. This study involved 14 types of tumors, and the number distribution of the tumors is also quite variable. The results of the current evaluation will change if the distribution of tumors changes. Stratified analysis based on disease types can obtain more accurate and stable results, but the sample size is too limited to support stratification.

In addition, many similar research projects have further tracked the adoption and use of secondary treatment opinions in the original hospital, as well as the satisfaction of doctors and patients. However, as this study is based solely on the analysis of the re-diagnosis records, follow-up investigation and a doctor-patient subjective investigation was not possible. It is expected that a follow-up study will be implemented to carry the study forward.

Conclusion

Although the SO project application scenarios involved in this study are limited, this telemedicine-based second opinion program has brought beneficial improvements to the diagnosis and treatment of cancer-related diseases in counterpart county hospitals in central and western China. We may recommend that this SO project should continue to be used as a beneficial supplement to clinical practice to protect patients’ rights and interests especially since the capacity of diagnosis and treatment of cancer diseases in county-level hospitals in central and western regions is still limited.

Limitations

It is difficult to evaluate the overall capacity of county hospitals in cancer diagnosis because this study is based on a SO project and cannot provide data on the overall size of cancer patients in County hospitals. We tried to search for literature on the proportion of cancer patients applying for telemedicine in all cancer patients or the proportion of general patients applying for telemedicine in all patients but failed.

Also, in this study, many patients came to the county hospital with a diagnosis from other hospitals. This is because the Chinese medical security system does not have particularly strict restrictions on patients’ medical treatment behavior and patients have a lot of freedom to choose hospitals on their own.[19,20] Therefore, the source of the original diagnosis can be problematic, that is, it may be a hospital with a higher level of technology, or it may be another hospital similar to the county hospital. The quality of the original diagnoses is therefore uneven. This will have a relatively large impact on the evaluation of SO diagnosis. It is desirable to have a hierarchical description based on the type of institution originally providing a diagnosis. However, due to the limited time and research funding, the sample size involved in this study is too small to achieve a more detailed calculation.

The location and types of tumors have a significant impact on the consistency of diagnosis and treatment recommendations. This study involved 14 types of tumors, and the number distribution of tumors is also quite different, the results of the current evaluation will change with the constitution of tumor disease. Stratified analysis based on disease types can obtain more accurate and stable results, but due to the same reasons, the sample size is limited, which is not enough to support stratification.

In addition, many similar types of research have further tracked the adoption and use of secondary treatment opinions in the original hospital, as well as the satisfaction of doctors and patients.[21-24] However, as this study is based on the data of the SO records, it lacks the design of follow-up investigation and doctor-patient subjective investigation, which is expected to be supplemented in the follow-up study.