Information System Success of the Iranian Integrated Health Record System Based on the Clinical Information System Success Model.

Background: The Integrated Health Record System, locally known as the "SIB," is the most used information system for recording public health services provided to the Iranian population. The objective of this study was to evaluate the success rate of the SIB using the Clinical Information System Success Model (CISSM).
Methods: This is a psychometric and evaluation study. The CISSM has a 26-item instrument that assesses 7 constructs in 3 following stages: (1) the socio-technical stage (facilitating conditions, social influence, information quality, and system performance(; (2) the integrated stage (system use dependency and user satisfaction); and (3) success outcome stage (net benefit). A Persian version of the CISSM instrument was validated and applied in this study. Based on this instrument, the reliability and the validity of the CISSM were assessed. The SIB success rate was evaluated using a validated CISSM. The study participants were 758 SIB users from different disciplines and different levels.
Results: Assessment of content validity, construct validity, internal consistency, and test-retest reliability showed acceptable psychometric properties of the CISSM instrument. The results demonstrated that the SIB success rate was in the moderate range (59.6%). Facilitating conditions and information quality were strong predictors of use dependency and user satisfaction, while both of these 2 constructs significantly influenced net benefit.
Conclusion: The SIB success was in the moderate range, and it needs to be enhanced. Therefore, Iranian healthcare policymakers should consider working on the most important factors influencing SIB success (facilitating conditions, information quality, use dependency, and user satisfaction) to improve SIB success.

• The integrated health record system (SIB) is the most widely used information system for recording public health services, which covers more than 80% of the Iranian population.

• Despite the widespread use, the information system success of the SIB has not been systematically evaluated.

• The clinical information system success model (CISSM) is a model for the assessment of the success of clinical information in a mandatory environment (e.g. SIB). The CISSM assesses 7 constructs in 3 stages: (1) socio-technical stage; (2) integrated stage; and (3) success outcome stage.

• The Persian version of the CISSM instrument is a valid and reliable tool to be used for the evaluation of the SIB.

• The most critical factors in SIB success are “facilitating condition”, “information quality”, “CIS use dependency” and “user satisfaction”.

•SIB success rate is at a moderate level and needed to be improved by considering the above-mentioned factors.

Introduction

Nowadays, using the information system in the health sector provides considerable opportunities to improve patient care quality, reduce process errors, better clinical documentation, and increase healthcare efficiency (1,2). In order to access the improvement of the health system in Iran, an Integrated Health Record System (SIB) was provided in 2015 to facilitate health service distribution, come up with the requirements for the accomplishment of the referral system, access the achievement of the data given on public health, and conclusively enhance the quality of health services. The SIB was put forward and performed in Health Centers in Iran. All health-related data of the population collected at the time of delivering primary health services are recorded in the SIB (3). At present, exceeding 73 million people in Iran have registered in the SIB. This information system has been set up in more than 36,000 urban and rural areas, employing more than 130,000 healthcare personnel (4).

Nowadays, despite many well-designed clinical systems, few are successful at the implementation stage (5), and there are some risks related to information systems in health sectors. These risks may include the high cost of these systems and the undesirable effect of inefficient information systems on personnel and patients. Accordingly, a thorough assessment of health information systems is a significant need (1). A systematic literature review has introduced various methods and frameworks for evaluating health information systems. The result of this study shows that none of these methods and frameworks alone were sufficient for a comprehensive evaluation (6). In the current study, the clinical information systems success model (CISSM) was used because assessing the success of the SIB was considered. In 2013, Garcia-Smith and Effken designed the Clinical Information Systems Success Model (CISSM), inspired by the Delone and Mclean information system success model, to evaluate clinical system success (5).

Based on the CISSM, a study was conducted to evaluate the success of an electronic health record from the viewpoint of personnel of nursery homes (7). In another study, the effectiveness of nursing information system implementation was assessed using CISSM (8).

Several study projects were carried out to evaluate the Iranian Integrated Health Record System (3,9,10). Usability evaluation (3), the study of goals’ achievement (9), acceptance of SIB among users are some of these evaluation studies (10). To the best of our knowledge, no study has yet been published for the success evaluation of SIB. Therefore, the purpose of this study was to evaluate the success rate of the Iranian Integrated Health Record System (SIB) based on the Persian version of the clinical information success model and instrument.

Methods

This study is conducted between December 2018 and August 2019 after obtaining approval from the Ethics Committee of Iran University of Medical Sciences (IR.IUMS.REC.1397.319). The Integrated Health Record System (SIB) users voluntarily participated in this research and signed written informed consent forms before entering the study.

The clinical information system success model evaluates the clinical information system's success in mandatory environments. This model, inspired by the D&M model, was designed by Garcia-Smith and Effken in 2013. There are 3 stages in the CISSM, including the sociotechnical, the integration, and the success outcome stage. The success outcome stage consists of only 1 construct called Net Benefit. Factors of the integration stage are CIS use dependency and user Satisfaction. System Performance (Reliability, Perceived Ease of Use, and Accessibility), Information Quality (Content Completeness, Perceived Usefulness, Format, and Accuracy), Social Influence (Social Support, Service Support), and Facilitating Conditions (Work Processes, Perceived Behavioral Control) are constructs of the Sociotechnical stage (5).

This study was conducted in 3 stages. In the first stage, the psychometric properties of a 26-item instrument (CISSM instrument) were evaluated (5) based on the standard protocol of writing suggested by Streiner and Kottner (11). In the second part of the study, the validity and the reliability of the CISSM were assessed according to the didactic presentation of structural equation modeling by Cristhian M. Ringle et al. (12). Finally, the success rate of the SIB was evaluated.

Psychometric Evaluation of CISSM

Psychometric evaluation of the CISSM instrument started by the forward and backward translation process based on the World Health Organization protocol (13) followed by assessing instrument's content validity using Waltz and Bausell method for content validity index (CVI) and Lawshe content validity ratio (CVR) method based on 15 medical informatics experts' data. The CVR was evaluated using a 3-point Likert scale to grade the importance of every item in the instrument. The CVI was assessed using a 4-point scale rating to grade the relevancy of each item in the instrument. Cut points of 0.49 and 0.79 were considered acceptable for CVR and CVI, respectively (14,15).

Construct validity was evaluated using an exploratory factor analysis (EFA). The EFA was assessed based on 406 SIB users' data, including physicians, nurses, midwives, and health professionals from 3 cities affiliated to Ahvaz Jundishapur University of Medical Sciences.

To assess test-retest reliability, 90 SIB users completed the instrument twice in 2 weeks, and the intraclass correlation coefficient was calculated. Internal consistency was assessed using the Cronbach alpha.

Assessing Validity and Reliab ility of CISSM

In the second part of the study, using the structural equation modeling in SmartPLS, the quality of the measurement model and hypotheses of the clinical information system success model were tested based on another sample of 379 SIB users.

CISSM hypotheses are listed below (5):

H1. System performance can measure CIS use dependency

H2. System performance can measure user satisfaction

H3. Information quality can measure CIS use dependency

H4. Information quality can measure user satisfaction

H5. Social influence can measure CIS use dependency

H6. Social influence can measure user satisfaction

H7. Facilitating condition can measure CIS use dependency

H8. Facilitating condition can measure user satisfaction

H9. User satisfaction and CIS use dependency can predict each other

H10. User satisfaction can predict net benefit

H11. CIS use dependency can predict net benefit

To assess measurement model quality, the first step was examining convergent validity by observing the average variance extracted (AVEs) (12). When AVEs are greater than 0.5, the results of model convergence will be satisfactory (16). Observing the Cronbach alpha and composite reliability as values for internal consistency was the next stage. Both values should be greater than 0.7 (12). The discriminant validity of the structural equation modeling can be assessed by Fornell and Larcker criteria. In this method, AVE's square root in every construct should be greater than between-construct correlations (17). R2 indicates adjusted model quality (12); according to Cohen’s suggestion, models with R2 more than 26% have a large effect (18). The Stone-Geisser indicator (Q2) and Cohen’s indicator (f2) are model accuracy indicators that should be more than 0 and 0.35, respectively (12). Finally, the goodness-of-fit indicator for the observed model was calculated by measuring the square root of R2 and AVE's product for each latent variable. The adequate amount of the GOF is 0.36 (19). For hypotheses testing, path coefficient and T statistics were considered (12). CISSM hypotheses are illustrated in Figure 1.

Fig. 1

CISSM test results

Abbreviations: β: regression coefficient, ***: t student > 1.96,*: t student < 1.96

Measurement of SIB Success Rate

Finally, the data used in the EFA and model validation were combined. This data, consisting of 785 SIB users, was used to evaluate the SIB success rate from the users' perspective. The total scores obtained from the whole questionnaire to the sum of maximum points that can be obtained were considered as success rate. The result of the success rate based on the user’s perspective was categorized as follows: first (under 25%), second, third, and fourth quartile considered poor, low, moderate, and appropriate success rate, respectively (20). In this study, comparison of CISSM constructs in gender, educational and occupational groups was done.

Statistical Analysis

The EFA was analyzed through SPSS 20. Structural equation modeling was done using SmartPLS M2.

Results

Based on CVR and CVI results, all the items were necessary and relevant, and no items were removed from the instrument. The 406 participants' age ranged from 20 to 59 years, with a mean of 36.4 (SD: 7.4). A total of 260 participants (64%) were female; 58% of the participants were health workers, 19.5% midwives, 12% physicians, and 10.5% were of other professions. Using the EFA, 7 factors with 21 items were identified. These factors were called "facilitating condition" (7 items), "social influence" (5 items), "information quality" (3 items), "system performance" (3 items), “user satisfaction” (1 item), “system use dependency” (1 item), and “net benefit” (1 item). According to the study results, because the Cronbach alpha and ICC results were in acceptable range, internal consistency and reproducibility were approved (Table 1).

Table 1

Items’ Characteristics

Factors	Items	SD1	CR2	CA3	AVE4	ICC5 for test & retest
Facilitating condition	q23: Completion of admission assessment	0.860	0.918	0.896	0.619	0.906
	q19: I can document close to the time of patient encounters	0.840
	q20: I can document close to the location of patient encounters	0.836
	q21: I can make referrals to other services are efficiently	0.796
	q22: I am usually able to complete the assessment documentation within 24 h	0.786
	q18: The CIS is compatible with other systems I use	0.735
	q17: I have the knowledge necessary to use the CIS	0.629
Social influence	q14: Service Support staffs understand the specific needs of clinicians	0.908	0.921	0.892	0.702	0.808
	q13: Service Support staffs are always willing to help clinicians	0.900
	q15: Service Support staffs provide enough training for clinicians	0.881
	q9: My supervisor has been helpful in the use of the CIS	0.751
	q11: The "super-user" on my unit has been helpful in the acceptance of the CIS	0.732
Information quality	q7: The CIS information is presented in a useful format	0.890	0.906	0.845	0.763	0.855
	q6: The CIS provides reports that seem to be just about exactly what I need	0.883
	q8: The CIS is accurate	0.847
System performance	q2: The CIS can be counted on to be “up” and available when I need it	0.891	0.866	0.768	0.684	0.687
	q1: The CIS is easy to use	0.850
	q4: The CIS is not subject to frequent problems and crashes	0.731
CIS use dependency	q24: I am dependent on the CIS to document assessments, plan and monitor patient care	1.000	1.000	1.000	1.000	0.762
User satisfaction	q25: Overall, I am satisfied with the CIS	1.000	1.000	1.000	1.000	0.793
Net benefit	q26: I believe that the CIS helps me to comply with hospitalnursing documentation quality standards	1.000	1.000	1.000	1.000	0.821

1 Standard loadings, 2 Composite reliability, 3 Cronbach alpha, 4 Average variance extracted, 5Intraclass correlation coefficient

Assessing Validity and Reliability of CISSM

In the model validation process, according to the results of Table 1, all of the AVEs were greater than 0.5, so convergent validity was satisfactory in all latent variables. Because the Cronbach alpha and composite reliability for all factors were larger than 0.7, the model's internal consistency was acceptable (Table 1). Table 2 is a Fornell and Larcker table. In this table, the AVE's square root in every construct was higher than between-construct correlations, and thus discriminant validity was provided.

Table 2

Fornell and Larcker Table

Latent Variable	CIS use Dependency	Net Benefit	User Satisfaction	Facilitating Conditions	Information Quality	Social Influence	System Performance
CIS use Dependency	1
Net Benefit	0.739	1
User Satisfaction	0.716	0.818	1
Facilitating Conditions	0.733	0.766	0.785	0.816
Information Quality	0.581	0.628	0.694	0.647	0.874
Social Influence	0.43	0.542	0.563	0.595	0.618	0.838
System Performance	0.588	0.635	0.705	0.74	0.653	0.591	0.827

The adjusted model had good quality because of R squares more than 26% (Table 3). Based on the result of the Stone-Geisser indicator (Q2) and the Cohen’s indicator (f2), model accuracy was perfect (Table 3). Good goodness-of-fit model index is seen in Table 3.

Table 3

Model Fitness Index

Index	Net Benefit	CIS Use Dependency	User Satisfaction	Facilitating Conditions	Information Quality	Social Influence	System Performance
(f2)	1	1	1	0.512	0.479	0.531	0.363
(Q2)2	0.697	0.569	0.658
GOF	0.829	0.773	0.847
(R2)	0.717	0.597	0.688

1 Cohen’s indicator, 2 Stone-Geisser indicator

According to the T statistics of path coefficients, all hypotheses were approved except relationship among social influence and user satisfaction and relationship among system (Goodness of fit indicator) performance and CIS use dependency (Fig. 1).

a. Measurement of SIB Success Rate

According to the result of this study, the SIB success rate from the users' perspective was moderate (59.6%).

According to Appendix Table 1, there is a significant mean difference between SIB users with a doctoral degree and other educational groups in all factors except social influence. Appendix Table 2 shows the success scores in the occupational groups. According to this table, no specific pattern can be identified for the results of the mean differences. Appendix Table 3 shows that the mean scores of females are higher than those in males in all factors, but these differences are not significant.

Discussion

Main Findings

This study aimed to evaluate the success rate of the Iranian Integrated Health Record System (SIB). To the best of our knowledge, the present study is the first to assess SIB success. The instrument used in this study was the CISSM instrument. The current study showed that the CISSM instrument has good validity and reliability in the SIB users' population. According to this study's results, the validity and the reliability of the clinical information system success model in the SIB users’ population were acceptable, and this model is useful to evaluate the success of the SIB. The most important predictors of the CISSM for SIB success were facilitating condition, information quality, system use dependency, and user satisfaction. According to users’ opinion, the SIB success rate was in the moderate range (59.6%). Since system support (the concept of Social Influence) is quite beneficial for the SIB, social influence scores are higher than other factors in other educational groups. Consequently, there is no significant mean difference between users with a doctoral degree and other users in this factor.

Specifically, facilitating condition had a significant and positive effect on user satisfaction and CIS use dependency (Fig. 1). These study findings revealed that the facilitating condition had the most impact on the SIB success in the socio-technical stage. It means that for the Iranian health policymakers to enhance the SIB success, the facilitating condition (record the data in a time and place close to the visiting location, referrals to other services, help the user to organize their work, and providing the necessary knowledge to users) must be given most extreme consideration. The significant effect of the facilitating condition on the integration stage is consistent with the study of Sebetci and Cetin, but the facilitating condition could only predict user satisfaction in the original CISSM study (5,21).

In the current study, social influence significantly influences CIS use dependency but not user satisfaction (Fig. 1). It shows that having good service support, helpful supervisors, and supportive organs can lead to CIS use dependency. This finding is in line with the original CISSM study (5). In a research by Sebetci and Cetin, social influence had a significant effect on user satisfaction, but its impact on CIS use dependency was not significant (21).

Information quality had a significantly positive effect on both constructs of the integrated stage, but its influence is weaker than the facilitating condition (Fig. 1). In this way, exact reports, useful format, and accuracy are predictive for user satisfaction and CIS use dependency. It is consistent with the original CISSM study (5). Sebetci and Cetin research demonstrate that information quality has only a significant influence on CIS use dependency (21). The Cronbach alpha for information quality was 0.72 and 0.61 in Tabesh and Tara study. The correlation between this construct and user satisfaction was significant in Tara study (correlation coefficient was 0.319) (20,22).

Similar to the original CISSM study, our finding revealed that system performance could influence only user satisfaction (Fig. 1). It shows being available, easy to use, and not having frequent problems is a forecasting factor for user satisfaction. The influence of system quality was significant on user satisfaction and was not significant on CIS use dependency in Sebetci and Cetin study (21). In Tabesh study, the Cronbach alpha was 0.74 for this construct. The relationship between system performance and other constructs was not evaluated (20). The Cronbach alpha was 0.72 in Tara study for system performance, and its correlation to user satisfaction was significant (correlation coefficient was 0.32) (22).

Net benefit in the success outcome stage is the users’ perception of how much the SIB helps them do standard documentation (5). Both latent variables in the integration stage (user satisfaction and CIS use dependency) significantly influence net benefit, but user satisfaction is a stronger predictor. This is consistent with the results of the original CISSM study and Sebetci and Cetin (5,21). It demonstrates that the degree of user satisfaction and user integration into the SIB in the health system can predict SIB success. The correlation between user satisfaction and net benefit was significant (correlation coefficient was 0.41) in Tara study (22).

Strengths and Limitations

The present study is the first study to evaluate the Iranian integrated health record system (SIB) success rate from the users’ perspective. For this purpose, the validity and the reliability of the clinical information system success model and its instrument were evaluated in the SIB users’ population. Furthermore, the users who participated in this study were SIB users from different professions. There are some limitations in this research. This study was performed on users of just 3 cities affiliated to Jundishapur University of Medical Sciences. It is necessary to use the opinions of more users from different universities to measure this clinical information system's success. Besides, in this study, convenience sampling was used, and future studies require random sampling.

Conclusion

According to the results of this study, SIB success rate from the viewpoint of SIB users was in the moderate range and needs to be enhanced. Therefore, Iranian healthcare policymakers should consider working on the most important factors influencing SIB success (facilitating conditions, information quality, use dependency, and user satisfaction) to improve SIB success.

Acknowledgement

We would like to express our appreciation to health officers of Ahvaz Jundishapur University of Medical Sciences for their participation and support in this study. We are also grateful to the CISSM creators for granting permission to adapt the Persian version of CISSM instrument for this study.

Etical Approval

This study approved by the Ethics Committee of Iran University of Medical Sciences (IR.IUMS.REC.1397.319).

Conflict of Interests

The authors declare that they have no competing interests.

Appendix Table 1

Standardized mean differences of success score between education groups

Factor	Education	PhD	Master	Bachelor	Associate degree	Diploma
Facilitating Conditions	PhD	0
	Master	0.68 (0.22, 1.12)	0
	Bachelor	0.66 (0.31, 0.99)	-0.05 (-0.43, 0.33)	0
	associate degree	0.56 (0.17, 0.94)	-0.11(-0.52, 0.31)	-0.06 (-0.37, 0.25)	0
	Diploma	0.60 (0.27, 0.95)	-0.02 (-0.40, 0.36)	0.02 (-0.23, 0.28)	0.08 (-0.23, 0.39)	0
Social Influence	PhD	0
	Master	0.13 (-0.31, 0.57)	0
	Bachelor	0.14 (-0.19, 0.47)	0.02 (-0.36, 0.40)	0
	associate degree	0.14 (-0.24, 0.52)	0.02 (-0.40, 0.44)	-.002 (-0.31, 0.31)	0
	Diploma	0.00 (-0.33, 0.33)	-.11 (-0.50, 0.27)	-0.13 (-0.39, 0.12)	-0.13 (-0.44, 0.18)	0
Information Quality	PhD	0
	Master	0.13 (-0.31, 0.57)	0
	Bachelor	0.36 (0.02, 0.69)	0.21(-.17,.59)	0
	associate degree	0.60 (0.21, 0.98)	0.42(-.01,.84)	0.21(-.09,.52)	0
	Diploma	0.79 (.44, 1.13)	0.62(.23, 1.10)	0.43(.17,.69)	0.23(-.08,.54)	0
System Performance	PhD	0
	Master	0.54 (0.09, 0.98)	0
	Bachelor	0.49 (0.15, 0.83)	-0.06 (-0.44, 0.32)	0
	associate degree	0.55 (0.17, 0.93)	0.03 (-0.39, 0.45)	0.09 (-0.22, 0.40)	0
	Diploma	0.65 (0.31, 0.99)	0.15 (-0.23, 0.53)	0.20 (-0.05, 0.46)	0.12 (-0.19, 0.43)	0
Use Dependency	PhD	0
	Master	0.50 (0.05, 0.94)	0
	Bachelor	0.72 (0.38, 1.06)	0.15 (-0.23, 0.53)	0
	associate degree	0.69 (0.30, 1.07)	0.16 (-0.26, 0.58)	0.03 (-0.28, 0.33)	0
	Diploma	0.74 (0.39, 1.08)	0.23 (-0.16, 0.61)	0.10 (-0.15, 0.36)	0.07 (-0.24, 0.38)	0
User Satisfaction	PhD	0
	Master	0.44 (0.00,0.88)	0
	Bachelor	0.49 (0.15, 0.83)	0.02 (-0.36, 0.40)	0
	associate degree	0.74 (0.35, 1.13)	0.29 (-0.14, 0.71)	0.26 (-0.05, 0.57)	0
	Diploma	0.87 (0.52, 1.21)	0.42 (0.03, 0.80)	0.4 (0.14, 0.66)	0.14 (-0.16, 0.46)	0
Net Benefit	PhD	0
	Master	0.60 (0.15, 1.05)	0
	Bachelor	0.48 (0.14, 0.81)	0.15 (-0.53, 0.23)	0
	associate degree	0.60 (0.21, 0.98)	-0.04 (-0.46, 0.38)	0.12 (-0.19, 0.43)	0
	Diploma	0.74 (0.39, 1.10)	0.11 (-0.27, 0.49)	0.27 (0.01, 0.52)	0.15 (-0.16, 0.46)	0

Appendix Table 2

Standardized mean differences of success score between occupation groups

Factor	Occupation	Physician	Midwife	Health Worker	Health Home Worker	Other
Facilitating Conditions	Physician	0
	Midwife	0.44 (0.06,0.81)	0
	Health Worker	0.78 (0.40,1.16)	0.33 (0.00,0.65)	0
	Health Home Worker	0.65 (0.31, 0.99)	0.25 (-0.03, 0.53)	-0.05 (-0.33, 0.23)	0
	Other	0.59 (0.14, 1.03)	0.14 (-0.25, 0.53)	-0.21 (-0.60, 0.18)	-0.12 (-0.48,0.23)	0
Social Influence	Physician	0
	Midwife	0.01 (-0.36,0.38)	0
	Health Worker	0.13 (-0.24,0.50)	0.11 (-0.21, 0.43)	0
	Health Home Worker	0.00 (-0.33,0.33)	-0.00 (-0.28, 0.27)	-0.11 (-0.39, 0.17)	0
	Other	0.16 (-0.27,0.59)	0.13 (-0.26, 0.52)	0.02 (-0.37, 0.41)	0.13 (-0.23, 0.49)	0
Information Quality	Physician	0
	Midwife	0.27 (-0.09,0.64)	0
	Health Worker	0.31 (-0.06,0.68)	0.03 (-0.29, 0.35)	0
	Health Home Worker	0.75 (0.40,10.08)	0.48 (0.20, 0.76)	0.46 (0.17, 0.74)	0
	Other	0.11 (-0.32,0.54)	-0.16 (-0.55, 0.23)	-0.19 (-0.58, 0.20)	-0.63 (-0.99,0.26)	0
System Performance	Physician	0
	Midwife	0.27 (-0.10,0.64)	0
	Health Worker	0.63 (0.25,1.00)	0.35 (0.03, 0.67)	0
	Health Home Worker	0.68 (0.34,1.02)	0.43 (0.15, 0.71)	0.12 (-0.16, 0.40)	0
	Other	0.34 (-0.10,0.77)	0.07 (-0.32, 0.46)	-0.29 (-0.68, 0.10)	-0.37 (-0.73,-0.0)	0
Use Dependency	Physician	0
	Midwife	0.52 (0.15, 0.90)	0
	Health Worker	0.65 (0.28, 1.03)	0.15 (-0.17, 0.47)	0
	Health Home Worker	0.74 (0.40, 1.08)	0.26 (-0.02, 0.54)	0.12 (-0.16, 0.40)	0
	Other	0.35 (-0.08,0.79)	-0.17 (-0.56, 0.22)	-0.31 (-0.71, 0.08)	-0.41 (-0.77,-0.05)	0
User Satisfaction	Physician	0
	Midwife	0.37 (0.00, 0.74)	0
	Health Worker	0.54 (0.17, 0.92)	0.19 (-0.13, 0.51)	0
	Health Home Worker	0.78 (0.44, 1.12)	0.44 (0.16, 0.72)	0.25 (-0.03, 0.53)	0
	Other	0.37 (-0.06,0.80)	0.00 (-0.39,0.39)	-0.19 (-0.58, 0.20)	-0.44 (-0.80,-0.08)	0
Net Benefit	Physician	0
	Midwife	0.34 (-0.03,0.71)	0
	Health Worker	0.46 (0.09, 0.83)	0.14 (-0.18, 0.46)	0
	Health Home Worker	0.64 (0.30, 0.98)	0.32 (0.04, 0.60)	0.17 (-0.11, 0.45)	0
	Other	0.30 (-0.13, 0.73)	-0.04 (-0.43, 0.35)	-0.18 (-0.57, 0.21)	-0.35 (-0.71,0.00)	0

Appendix Table 3

Standardized mean differences of success score between genders

Factor	Gender
Facilitating Conditions	-0.24 (-1.65, 1.16)
Social Influence	-0.28 (-1.38, 0.83)
Information Quality	-0.06 (-0.71, 0.59)
System Performance	-0.43 (-1.06, 0.20)
Cis Use Dependency	-0.02 (-0.26, 0.21)
User Satisfaction	-0.23 (-0.49, 0.03)
Net Benefit	-0.10 (-0.33, 0.14)