Content Validity of National Post Marriage Educational Program Using Mixed Methods.

BACKGROUND: Although the validity of content of program is mostly conducted with qualitative methods, this study used both qualitative and quantitative methods for the validation of content of post marriage training program provided for newly married couples. Content validity is a preliminary step of obtaining authorization required to install the program in country's health care system.
METHODS: This mixed methodological content validation study carried out in four steps with forming three expert panels. Altogether 24 expert panelists were involved in 3 qualitative and quantitative panels; 6 in the first item development one; 12 in the reduction kind, 4 of them were common with the first panel, and 10 executive experts in the last one organized to evaluate psychometric properties of CVR and CVI and Face validity of 57 educational objectives.
RESULTS: The raw data of post marriage program had been written by professional experts of Ministry of Health, using qualitative expert panel, the content was more developed by generating 3 topics and refining one topic and its respective content. In the second panel, totally six other objectives were deleted, three for being out of agreement cut of point and three on experts' consensus. The validity of all items was above 0.8 and their content validity indices (0.8-1) were completely appropriate in quantitative assessment.
CONCLUSION: This study provided a good evidence for validation and accreditation of national post marriage program planned for newly married couples in health centers of the country in the near future.

Introduction

Marriage as one of the most important institutions in the world establishes intimate relationship, regulate sexual relations and encourage commitment between spouses. Research suggests a relationship between marriage satisfaction and positive consequences on spouses’ health, upbringing healthy children, living happily and consequent health outcomes on community (1, 2). Marital dissatisfaction and divorce is proven to increase consumption of more alcohol and drugs (3), occurrence of stroke, heart attack, depression, (4,5) and negative effects of one-parent children compared with children of two-parent ones (6).

Family norms and structure has been obviously affected by fast changing family roles and socialization (7). Unfortunately, divorce rate is rising and casting a dark shadow over common lives in the world. Statistics show that the United States, Puerto Rico and Russia with rate of 49.5%, 44.7 and 33.6 are the top 3 divorce rate countries in the world (8). In Iran, 12 divorce cases were reported verses 100 marriage ones in 2007, and 16.5 verses 100 by the end of October 2012. In other words, about one out of every 6 couples in Iran gets divorced (9).

Marital enrichment programs can improve family positive capabilities (10). Divorce increasing rates in Iran during the last 10 years (9) and on the other hand, the positive consequences of marriage programs (11) caused Family Health authorities to intervene with post marital education for newly married couples in public health centers. Converting the raw data, written by the professional experts of the Health Bureau of the Ministry, into training package format was performed as a part of researcher's PhD thesis.

The main objective of thesis was to assess quality and effectiveness of post marriage education.

Preparation of the package (generating data)

Item generation is the most basic element of designing reliable measures (12). Considering content of package as the training instrument, converting raw data into educational package form, was the first step of development of the scale in the process of content validation. To prepare the package, broad foreign and local librarian and electronic resources were studied on the most widespread and successful programs currently running in the world including Premarital Relationship Enhancement and Prevention (PREP)(14,15), Prepare Enrich(16,17), Relationship Enhancement (RE)(18) and the Couple Communication Program(CCP)(19). Finally skills of communication, active listening, control negative emotions, problem solving and conflict resolution, are extracted as the most common skills trained in all of effective and successful programs to enhance marital relationship and prevent distress. Research show that lack of communication skills, marital conflicts and sexual dissatisfaction were the main and common causes of marital dissatisfaction and divorce in the country (20–23). After that, the mainframe work of national post marriage package with primary preventive approach was designed in three main domains of mutual and reciprocal rights of spouses towards each other from Islam's perspective, communication skills and sex education. After studying comprehensively on the scope, the main topics and its pertaining content for each domain was created by researcher regarding the raw format received from the ministry with supervision of an experienced expert of Health Bureau.

Materials and Methods

Study design

This is a mixed qualitative and quantitative methodological study. Four consecutive steps were taken for the content validity process of post marriage program. Firstly, qualitative content developing panel whose 6 experts possessed appropriate content knowledge in marriage subject was executed. Prior to the next panel, 4 knowledgeable professors of thesis were asked to specify whether the content is appropriate, related and clear. Third step was taken with constituting the second panel, involving 13 experienced panelists, to rate the objectives of the program on a 6-point Likert scale with defined cut of point of 0.75. Then supposing the content of program as an educational measurement, quantitative psychometric properties of face validity and CVR and CVI were assessed by third judgment expert panel. Minimum value of CVR for the specified number of panelists was calculated and presented by Lawshe (Table1). According to this table, accepted cut-off point for 10 experts, based on a one-tailed test at the .05 level of significance, was 0.62. The mean of the CVR values of the retained items resulted in CVI index (24).

Table 1:

Minimum Values of CVR and CVR One Tailed Test, p= .05

No. of panelists	Min. Value*
5	.99
6	.99
7	.99
8	.75
9	.78
10	.62
11	.59
12	.56
13	.54
14	.51
15	.49
20	.42
25	.37
30	.33
35	.31
40	.29

-When all say “essential”, the CVR is computed to be 1.00, (It is adjusted to .99 for ease of manipulation) -When the number saying “essential” is more than half, but less than all, the CVR is somewhere between zero and .99

Process of validation

Step1: Six experts with well-developed views on the research topic of psychology, family counseling, heath education fields and experienced in preparation of educational package of Tehran and Ghom universities was contacted by E-mail and telephone and invited to participate in the study. Then invitations to participate in the study along with firsthand version of educational content prepared by the researcher, a brief description of the study and the main goal of panel formation were sent to them by E-mail. After 2 months, the researcher received their professional comments on the area they were asked; mainly resulted in generating, replacing or refining topics and pertaining sub- contents, as the essential step of developing instrument. To achieve the purpose, their comments were wrote down by the researcher and for clarification of vague statements, an interview of 30 minutes was, on scheduled appointment time, performed with each expert separately on three qualitative elements: representativeness, comprehensiveness and clarity which had firstly been asked (25). Then the panelists were asked to vote the results. After valuing and summing the values of each idea, they were prioritized. Representativeness in this study referred to the degree to which each part of content reflected its nominated topic, in other words, the closeness of the contents to the constructs. Comprehensiveness of the entire instrument identified content, which they perceived to be incompatible with its respective topic. Finally, the experts were asked to identify the clarity of wording to ensure that there were clear and understandable. After writing the experts' comments on the paper, their conversation were grouped into common and opposing comments and asked them again until the experts reached a consensus.

Step2: In the second step, the modified version 2, based on the comments of first panel was sent to 4 faculty members of Shahroud University, who were student's thesis professors, by E-mail and they were asked to comment on appropriateness, consistency, adequacy, clarity, simplicity, relatedness and attractiveness. After receiving their comments, content was modified and again sent back to them. This process was repeated 2 times until consensus was achieved on topics and its content resulted in production of version 4.

Step 3: 13 experienced members from wide range multidisciplinary specialties including psychiatry, psychology, family counseling, community medicine, epidemiology, reproductive health, health education, general physician fields and expertise of Health Bureau on the area were intentionally selected. Four of 13 were members of previous step. The same process of invitations of first panel was carried out by E-mail. One expert did not accept the invitation so was excluded from the panel. The whole of modified content of last version was, based on each essential topic and its sub content carrying educational message, segmented into 63 sections. Then specific educational objective, which achieving these objectives altogether was the main goal of the study, in one of three cognitive, affective or behavioral domains of Blooms' taxonomy were created for content of each topic and sent to the members. The panelists were asked to measurethe degree of their agreement in the response to the question “if the specified objective can be achieved with its respective topic and content”. The responses were assessed on a 6-point Likert scales with options of completely disagree, disagree, relatively disagree, relatively agree, agree and completely agree. To determine which objective should be retained or rejected, a criterion for each objective was established. Objective being validated on the agreement scale was considered within validation range if 75% or more of the experts chose the option of “agree or completely agree”. In addition, the experts were requested to give any comment content if they were willing. After receiving, their responses lasted 2 months, the number of “agree” and “completely agreement” options was calculated for each objective, tabulated in table by researcher, and sent to each panelist. Then the panel meeting for final ranking of panelists' ideas was constituted in virtual environment and the objectives out of criteria were deleted.

Step 4: Third panel was completely different from the previous one both in the individuals and in member count, consisted of 10 members of relevant training qualifications and more in executive positions. The CVR indicator of necessity and the CVI representativeness for relevance, simplicity and clarity of questions were, based on expert views, valued in this step (26). With similar process of previous panels, the invitations and modified content according to couples' viewpoints and respective objectives were sent to the knowledgeable members by E-mail. To determine CVR, the members were asked to value the content validity of specific objective representative of its content on three spectra: “It is necessary”, “it is helpful but not necessary, “not necessary”. In order to calculate the CVI for each objective, features like “being relative, “being simple” and clear” should be taken into account using four-choice Likert scale (27). Polit and Beck believe that assessing the element of relatedness will suffice for calculating CVI (28). So to determine the CVI, relatedness was assessed from ranging 1=not relevant to 4=highly relevant (29). The minimum CVR for each item to be considered as acceptable was 0.78 (26).

After constituting the third panel, in order to achieve qualitative face validity, modified version 6, was given to evidence based adequate number of 5 newly wedding couples (10 persons), similar to the target groups of the study, to read and assess it in terms of simplicity, readability and clarity (30, 31).

Results

Findings of step 1

Guided by the interview data of panel 1, one topic of “stress management” was deleted and topics and its content: “barriers to empathy”, “empathy reinforcement” and “four different types of conversation styles” were generated. This step is often viewed as the minimum psychometric requirement for measurement adequacy and is the first step in validation of a new measure (25).

Findings of step 2

Comments of thesis professors of Shahroud University of Medical Sciences were mainly on simplification, clarity and wording, which were corrected.

Findings of step 3

Three objectives as representative its content including “Emotional intelligence”, “the dimension of emotional intelligence” and “effective factors on sexual disorders treatment” being less than defined cut of point criteria, were deleted. The objectives leveled more than cut of point, were eligible to be retained in the package. The second group of validation criteria of a panel is related to panel consensus on item, which is different from panel agreement. However, 3 other objectives for their theoretical features were deleted from the content as consensus was not reached in the meeting. The results of this panel are briefly shown in Table 2.

Table 2:

Percentage of panelist's degree agreement on objective

Degree of agreement	Total No of objectives=63Total No of panelists=12Total No of responds: 63*12=756	Percentage
Completely disagree	1	0.13
Disagree	22	2.90
Relatively disagree	16	2.11
Relatively agree	81	10.71
Agree	282	37.30
Completely agree	354	46.82
Total of “agree and completely agree” responds	636	84%

Findings of step 4

CVR of total 57 objectives as representative its content were all above the cut-off values presented in Lawshe Table (0.62), therefore all content were retained for the next stage. Also based on the results of CVI calculation, all the objectives were accepted, since their CVI values were greater than 0.78. The objectives of first domain and their CVR and CVI indices are shown in the Table 3.

Table 3:

CVR (Content Validity Ratio) and CVI (Content Validity Index) of educational objectives of first domain: Reciprocal couples' rights from Islam's view

No.	List of educational objectives as representatives content of package	CVR	CVI
1	Explain the aims of constituting family from Islam's perspective	1	1
2	Understand the importance of preserving family life	.08	.09
3	Observe reciprocal spousal rights towards each other	1	1

Authorization of the package

After the version 7 was produced in the consequence of assessing face validity on couples' viewpoints, it was submitted to Health responsible authority of ministry. Within 20 days, authenticity of content was confirmed by the respectable director of the Health and Population Bureau of ministry and program was authorized to implementing in the country.

Discussion

The current study aimed to validate content of Iranian couples' tailored post marriage program prior to receiving authorization required for its installment in health care system. For this purpose similar scientific sequential DeVellis' (1991) procedure, alternating between qualitative and quantitative methods, of domain identification, qualitative content generation and quantified judgment were exactly traced (32). Based on Hinkin's guideline (12) and in parallel to Palter's and Boll studies (33, 34), item developmental process was followed out to specify the area of content through a comprehensive literature review by the researcher.

The first panel composed of intentionally six expert panelists experienced in providing marriage packages. Based on Patton's view, the sample size is not necessarily needed to be large in qualitative interview since information richness of the cases and the observational and analytical abilities of the researcher have more effect on the validity, meaningfulness, and insights than sample size (35). Within qualitative interview, mainly being item-oriented developmental step, only three topics generated versus one major topic rejected. This result was, in disagreement with Beck illustration (36) and most studies (31, 37, 38) in which the pool of items was created during qualitative inquiry. This may be attributed to not only original tool development by professional writers but also time consuming and well-processed relevant literature review conducted by the researcher. Sixty-three well-prepared educational objectives written for processed content were given to the 13 multi-disciplinary specialists to rate them with respect to its sub-content on purposely selected 6-point Likert scale. This kind of scale, without neutral rating of forced choicemethod, is preferred when participants are conversant and familiar with the matter (39). Refinement process of this panel resulted in reduction of 6 objectives, 3 due to being lower than determined minimum cut off value of 0.75 and 3 for being out of panelists' consensus. The findings of the last panel indicated that parallel to other well-processed studies (32, 40, 41), the CVR of 57 retained objectives were all above the cut-off value 0.62 for 10 panelists (Table 2). According to Lawshe, if more than half of the panelists indicate that an item is essential, that item has at least content validity (26). Contrary to Rahimi's study in which 24 items were omitted for CVR and CVI indices were lower than 0.62 and 0.8 respectively (31), relevance of all objectives were accepted, since their CVI values were greater than acceptable lower limit of 0.8 recommended by Pilot and Beck (28). Very high-evaluated indices of CVR and CVI, ranging between 0.8 to1.00 reflected high agreement of panelists on the post marriage program content eligible to implement in health care system of country.

No study reviewed used highly credible method of psychometric properties of CVR and CVI for content validity of educational program with converting its content to objectives presented in this paper. Whereas this method was firstly proposed to determine not only content validity of tests and measures but also of procedures, contents and other attainable outcomes provided that the right and correct steps were taken observed in this study (26). We are convinced that using psychometric properties of CVR and CVI indices is useful to validate content of educational program. So this quantitative method for validity of program content is recommended to diligent researchers.

Limitations

In first qualitative panel, which was performed to determine the right retained and rejected content and topics, the purposive sampling method was used. Whereas the most participants of the panel consisted of policy makers, research professors, family health service providers involved or interested in installing the program, so they may be biased. However, the high agreement rate of panelists who were mostly of nondependent executive experts extracted from the qualitative indices show that the research strategy have tried to remove this shortcoming.

Conclusion

The study demonstrated the validation, with high levels of agreement among experts reflected in quantitative indices, exactly expected for its primary professional generation and well-developed process.

Ethical Consideration

After confirmation of Ethics Committee of SHMU and approval of SHMU and authorization signed by Health Bureau, the study was commenced. Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.