Psychometric testing of the Norwegian version of the Nutritional Form For the Elderly among older home-dwelling people.

BACKGROUND: Nutritional screening instruments need to be evaluated in terms of reliability and validity and being able to demonstrate sensitivity and specificity for use in clinical practice and research. The aims of this study were to test the reliability and validity of the Norwegian version of the Nutritional Form For the Elderly (NUFFE-NO) in a sample of older home-dwelling people, and to use the short form of the Mini Nutritional Assessment (MNA-SF) as a standard.
METHODS: A postal questionnaire, including the two instruments, background variables, and health-related questions, was sent to 6033 home-dwelling older people (65+ years) in southern Norway. In total, 2106 persons responded and were included. Data were analyzed statistically regarding homogeneity, concurrent and construct validity, sensitivity, and specificity of NUFFE-NO.
RESULTS: A Cronbach's alpha coefficient of 0.71 and significant item-to-total correlations were obtained as measures of homogeneity. Concurrent validity was assessed by a correlation coefficient of -0.37 (P < 0.001) between NUFFE-NO and MNA-SF. NUFFE-NO could separate known nutritional at-risk groups as a measure of construct validity. A cut-off point of ≥4 for identification of older people at nutritional risk was found for NUFFE-NO with MNA-SF as a standard.
CONCLUSION: NUFFE-NO shows adequate psychometric properties regarding homogeneity and construct validity. MNA-SF was not found to be the most suitable standard to use, because a low correlation coefficient was obtained as a measure of concurrent validity and a lower cut-off point was found compared with another study using the Mini Nutritional Assessment (MNA(®)) as a standard for NUFFE-NO. The obtained cut-off point of ≥4 is not recommended for use in practice or research, because many false positive nutritional at-risk persons would then be identified. Further studies with suitable design have to be performed among older home-dwelling people using the MNA as a standard.

Introduction

Being at risk of undernutrition or being undernourished is a frequent problem among older patients,1–3 but to a less extent among older home-dwelling people.4–6 However, older home-dwelling people receiving home health care have been found to be at high risk for developing undernutrition.7 Therefore, to use a nutritional screening instrument to identify and highlight those who are in need of nutritional assessment and nutritional care may be very useful.8,9 A screening instrument is not designed to assess nutritional status but should indicate potential or actual nutritional problems.10 However, such a screening instrument has to be evaluated in terms of reliability and validity. It also has to demonstrate sensitivity and specificity in order to be used by health professionals in clinical practice8 and research. Reliability of an instrument is the degree of consistency with which it measures the attribute it is supposed to be measuring, and validity refers to the degree to which an instrument measures what it is supposed to be measuring. An instrument’s sensitivity is its ability to identify cases correctly and its specificity is the ability to identify non-cases correctly.11

The Mini Nutritional Assessment (MNA®) is a well known nutritional screening instrument designed for older people12 and is considered to be a reliable13 and valid screening instrument for use in Europe and western health care practice settings.14 A short form of the MNA (MNA-SF), using some of the items in the MNA, has been found to be equal to the full form of the MNA and can be used for quick nutritional screening of older people.15 However, body mass index (BMI) is included in both the MNA and MNA-SF, and may cause some difficulties in performing screening among disabled older persons. Tsai et al16 have found that the MNA could maintain full functionality and enhance the usefulness without BMI. This result was based on the fact that both the MNA with BMI and without BMI could effectively grade the nutritional status of neuropsychiatric patients and show good correlations with BMI, calf circumference, and length of hospital stay. Furthermore, MNA-SF has been found to be easier to administer and less time-consuming when the particular item of BMI was replaced with calf circumference measurement.17

The Nutritional Form For the Elderly (NUFFE) nutritional screening instrument has been developed specifically as a simple instrument for screening in older people. The intention with NUFFE is that it should be an instrument that is easy to use and, therefore, no anthropometric measurements are included. Thereby, it is also suitable for use as a self-report instrument. NUFFE was originally developed in Sweden and has been translated into several languages. It is considered to be an instrument with sufficient evidence of reliability and validity.18–21 The MNA has been found to be a suitable standard in testing procedures both for the original Swedish version19,20,22 and for the Norwegian version (NUFFE-NO)21 regarding concurrent validity, sensitivity, and specificity among older hospital patients.

One testing study23 of NUFFE-NO has been performed in a group of 158 older home-dwelling people with the MNA-SF as a standard to assess concurrent validity, sensitivity, and specificity. Data were collected by a postal questionnaire, and the MNA-SF was considered in that study to be the best choice, because the MNA contains measurements of mid-arm and calf circumference, which are difficult to carry out by the older people themselves without help from professionals. However, the results revealed that using the MNA-SF as a standard, a lower cut-off point for identifying older home-dwelling people at nutritional risk was found compared with the cut-off point found using the MNA as a standard for the Swedish and Norwegian versions of NUFFE among older hospital patients.20–22 Therefore, a new study is necessary in order to test the reliability and validity of NUFFE-NO among older home-dwelling people and to determine whether MNA-SF is a good standard or not regarding validity, sensitivity, and specificity. According to Fletcher and Fletcher,24 it is essential to find a suitable standard when estimating the sensitivity and specificity values required in order to identify a suitable cutoff point. Moreover, a suitable cut-off point is a condition for using an instrument in clinical practice and research with the purpose of identifying people at nutritional risk.

The aims of this study were to test the reliability and validity of the Norwegian version of the NUFFE in a sample of older home-dwelling people and to use the MNA-SF as a standard.

Methods

Design and sample

The study had a cross-sectional design. During the spring and summer of 2010, a postal questionnaire, including an information letter about the study and an invitation to participate, was sent to a randomized sample of 6033 older home-dwelling people, 65 years old in the present year or older, living in five counties in southern Norway. Randomization was accomplished by the National Directory of Residents according to the directory’s common procedures. Answering and returning the questionnaire was considered to be informed consent to participate in the study. A total of 1671 persons responded to the questionnaire. After a reminder to those who had not responded, additional 435 persons answered and returned the questionnaire. Therefore, 2106 (34.9%) older persons were included in the study.

Data collection

The present study is part of a larger project investigating self-care and health among older home-dwelling people, which has been reported in part elsewhere.25,26 The content of the questionnaire used for this study included background variables such as age and gender, five health-related questions (that could be answered by “yes” or “no”), and the NUFFE-NO and MNA-SF nutritional screening instruments.

Nutritional Form for the Elderly

The NUFFE nutritional screening instrument is an ordinal scale with 15 three-point items originally developed in Sweden. The items reflect weight loss, changes in dietary intake, appetite, intake of cooked food, portion size, intake of fruit or vegetables, opportunity to obtain food products, company at meal times, activity, tooth/mouth or swallowing difficulties, fluid intake, gastrointestinal problems, help with eating, number of drugs, and eating difficulties because of impaired health. Each item score ranges between a score of 0 and 2. The most favorable option gives a score of 0, the most unfavorable option a score of 2. The maximum score is 30. Higher screening scores indicate higher risk of undernutrition.18 Some examples of the items with response alternatives of NUFFE are displayed in Table 1.

Table 1

Examples of items and response alternatives using the Nutritional Form For the Elderly

Items	Score 0 (Most favorable response alternative)	Score 1 (Intermediate response alternative)	Score 2 (Most unfavorable response alternative)
3. What is your appetite like now?	Good	Somewhat low Poor
5. What sized portions do you normally eat?	Large or ordinary portions	Fairly small portions	Very small portions
8. Do you normally eat together with anyone else?	Yes	Sometimes	Very seldom

The Swedish version of NUFFE has been tested regarding reliability and validity,18,19 and sensitivity and specificity20,22 among geriatric rehabilitation patients. The Swedish version of NUFFE was shown to be a fairly reliable instrument with evidence of validity. Obtained Cronbach’s alpha coefficients, as a measure of homogeneity, were 0.70 to 0.72.18,19 Concurrent validity was shown by a high correlation coefficient (rs = −0.74, P < 0.001) between NUFFE and MNA. Using the MNA as a standard for estimating sensitivity and specificity values, the following cut-off points were found: <6, ≥6, and ≥13, indicating low, medium, or high risk of undernutrition, respectively. The 6 and 13 cut-off points were based on sensitivity and specificity values of 71% and 86% and of 70% and 98%, respectively.20,22 The NUFFE has been translated into the Norwegian language in accordance with a procedure recommended by Streiner and Norman.27 The translation into Norwegian was performed by two persons speaking Swedish and Norwegian, respectively, and then back-translated into Swedish by another person, who was bilingual. The back-translation was compared with the Swedish original version and considered to be satisfactory. The two languages have the same origin and are similar.

NUFFE-NO has been tested regarding reliability, validity, sensitivity, and specificity with the MNA as a standard among older medical hospital patients. The results showed that NUFFE-NO had sufficient psychometrical properties for performing an institutional screening of older patients with a Cronbach’s alpha coefficient of 0.77 and a correlation coefficient of rs = −0.74 (P < 0.001) between NUFFE-NO and the MNA as a measure of homogeneity and concurrent validity, respectively. When estimating sensitivity and specificity values, the cut-off points <6, ≥6, and ≥11, indicating low, medium, and high risk of undernutrition, respectively, were found with the MNA as a standard. The 6 and 11 cut-off points were based on sensitivity and specificity values of 83% and 73% and of 77% and 83%, respectively.21

When the MNA-SF was used as a standard for NUFFE-NO among a group of older home-dwelling people, concurrent validity was supported with a correlation coefficient of rs = −0.26 (P = 0.001) between NUFFE-NO and MNA-SF. The cut-off point ≥ 4 was found to indicate nutritional risk based on a sensitivity value of 79% and a specificity value of 75%.23

Mini Nutritional Assessment-Short Form

The MNA-SF nutritional screening instrument is a nominal and ordinal scale containing six items, ie, questions about appetite, weight loss, mobility, psychological stress of acute disease, neuropsychological disease, and BMI. These items are the six first items of the 18 items in the full MNA. The maximum score for the MNA-SF is 14. Scores of 12 and above indicate satisfactory nutritional status. A score of 11 or below suggests a risk of undernutrition. MNA-SF is considered to be as effective as the MNA for nutritional screening of older people15 and compares well with the MNA.28 Both instruments are found to be sensitive, specific, and accurate in identifying nutritional risk in older people.13,28 In the present study, the first five items were used in the questionnaire in addition to two questions about weight and height. BMI was calculated when analyzing the data.

Statistical analyses

PASW Statistics version 18 was used to perform the statistical analyses. Statistical significance was determined as a P value of <0.05. The reliability of NUFFE-NO was assessed as homogeneity or internal consistency by calculating the Cronbach’s alpha coefficient27,29 and item-to-total (Spearman rank) correlations between each item and the total scale. The correlation between the individual item and the total scale was calculated when the particular item was omitted from the total scale.27 In addition, the Cronbach’s alpha coefficient27,29 was estimated for the MNA-SF.

The validity of NUFFE-NO was assessed as concurrent validity (as a measure of criterion-related validity) and construct validity.27 Concurrent validity was assessed by calculating the Spearman rank correlation between the total scales of NUFFE-NO and MNA-SF. Construct validity was assessed by comparing median scores of NUFFE-NO for groups with expected high and low scores, respectively, ie, between “known groups”.11 Such groupings consisted of people receiving food distribution or not, receiving home nursing or not, having chronic disease/handicap or not, being satisfied with life or not, and with perceived good health or not. Differences in scores between groups were tested using the Mann–Whitney U-test.

Sensitivity, specificity, and positive and negative predictive values were estimated11,24,27 in order to determine the cut-off point for NUFFE-NO when identifying older people at nutritional risk. To find this cut-off point, the standard used was being at nutritional risk or not, measured by MNA-SF. Sensitivity, specificity, and positive and negative predictive values were calculated for each cut-off point of NUFFE-NO according to Fletcher and Fletcher.24 A receiver-operating characteristic (ROC) curve was constructed, whereby the sensitivity values for each cut-off point of NUFFE-NO were plotted (y axis) against 1-specificity values of each cut-off point of NUFFE-NO (x axis). The optimum point will be found near a “shoulder” of the ROC curve in the upper left corner.11,24,27 The accuracy of the optimum point11 and the area under the curve for the optimum point was calculated.

Ethical considerations

The authors were guided by the intentions of the Declaration of Helsinki30 and ethical standards31 when designing and performing the study. The main project, that consisted of two projects among older home-dwelling people living in rural25 and urban26 areas, respectively, was approved by the Regional Committee for Medical Research Ethics in southern Norway (REK sør-øst D 2009/1299 and REK sør-øst A 2009/1321), and approval was also given to use the data in the present study (REK sør-øst D 2011/2588).

Results

Participants

The study group (n = 2106) consisted of 1063 (50.5%) women and 1043 (49.5%) men, with a mean age of 74.5 ± 6.9 years and a median age of 73.0 years (interquartile range 69.0–79.0).

Reliability

Homogeneity of NUFFE-NO, as a measure of reliability, reached a Cronbach’s alpha value of 0.71 and significant item-to-total correlations (Table 2). The homogeneity of MNA-SF was reflected in a Cronbach’s alpha coefficient of 0.45.

Table 2

Item-to-total (Spearman rank) correlations of the Norwegian version of Nutritional Form For the Elderly (n = 1907)

Item number	Item content	rs	P value
1	Weight loss	0.26	<0.001
2	Changes in dietary intake	0.33	<0.001
3	Appetite	0.41	<0.001
4	Intake of cooked food	0.26	<0.001
5	Portion size	0.36	<0.001
6	Intake of fruit or vegetables	0.25	<0.001
7	Possibility of obtaining food products	0.20	<0.001
8	Company at meals	0.25	<0.001
9	Activity	0.33	<0.001
10	Tooth/mouth and swallowing difficulties	0.21	<0.001
11	Fluid intake	0.19	<0.001
12	Gastrointestinal problems	0.26	<0.001
13	Eating assistance	0.09	<0.001
14	Number of medications	0.23	<0.001
15	Difficulty to eat due to impaired health	0.27	<0.001

Validity

The concurrent validity of NUFFE-NO was reflected in a significant Spearman rank correlation coefficient of −0.37 (P < 0.001) between the total scale of NUFFE-NO (n = 1907) and the total scale of MNA-SF (n = 1915). The construct validity of NUFFE-NO was reflected in significant differences between obtained median scores for groups with expected high and low scores, respectively (Table 3).

Table 3

Norwegian version of Nutritional Form For the Elderly scores for groups with expected high and low scores

Groups with expected high scores	n	Median (interquartile range)	Groups with expected low scores	n	Median (interquartile range)	P value
Food distribution	42	6.50 (4.75–10)	No food distribution	1865	3 (2–5)	<0.001
Home nursing	89	8 (6–11.50)	No home nursing	1818	3 (2–5)	<0.001
Chronic disease or handicap	824	4 (3–6)	No chronic disease or handicap	1083	3 (1–4)	<0.001
Not satisfied with life	216	6 (3–9.75)	Satisfied with life	1691	3 (2–5)	<0.001
Perceived ill health	135	6 (4–11)	Perceived good health	1679	3 (2–5)	<0.001

Sensitivity and specificity

Based on sensitivity, specificity, positive, and negative predictive values (see Tables 4 and 5) and the ROC curve, the cut-off point ≥ 4 of NUFFE-NO was found for identifying people at nutritional risk with the MNA-SF as a standard. The accuracy of this cut-off point was found to be 74%. The area under the ROC curve for this particular cut-off point was 0.73 (95% confidence interval 0.69–0.76).

Table 4

Sensitivity, specificity, and positive and negative predictive values for the Norwegian version of Nutritional Form For the Elderly using the Mini Nutritional Assessment-Short Form as a standard (n = 1812)

Cut-off pointsNUFFE-NO	Sensitivity %	Specificity %	Positive predictive value %	Negative predictive	value %
0	99	6	14	98
1	97	22	16	98
2	89	41	19	96
3	82	60	24	96
4	71	74	30	94
5	59	84	37	93
6	48	92	48	92
7	37	95	55	91
8	30	97	63	90
9	25	98	73	89
10	20	99	81	89
11	15	99	84	88
12	12	99	88	88
13	11	99	96	88

Abbreviation: NUFFE-NO, Norwegian version of Nutritional Form For the Elderly.

Table 5

Calculated sensitivity, specificity, and positive and negative predictive values and accuracy of the cut-off point ≥ 4 for the Norwegian version of the Nutritional Form For the Elderly, indicating risk of undernutrition using the Mini Nutritional Assessment-Short Form as a standard (n = 1812)

	People at nutritional risk according to standard MNA-SF (scores ≤11)(n = 247)	People at no nutritional risk according to standard MNA-SF (scores ≥12)(n = 1565)
People at nutritional risk according to NUFFE-NO(scores ≥4)(n = 583)	An = 176	Bn = 407
People at no nutritional risk according to NUFFE-NO(scores <4)(n = 1229)	Cn = 71	Dn = 1158
Sensitivity: A/A + C = 71%
Specificity: D/B + D = 74%
Positive predictive value: A/A + B = 30%
Negative predictive value: D/C + D = 94%
Accuracy: A + D/(A + B + C + D) = 74%

Abbreviations: NUFFE-NO, the Norwegian version of Nutritional Form For the Elderly; MNA-SF, the Mini Nutritional Assessment-Short Form.

Discussion

In this study, the NUFFE-NO was tested for its reliability and validity, including sensitivity and specificity, in a sample of older home-dwelling people. MNA-SF was used as a standard to assess concurrent validity and to estimate sensitivity and specificity values.

Homogeneity was chosen as a measure of reliability, because it is a method requiring only one test administration.11,27 The Cronbach’s alpha coefficient of 0.71 was equal to the coefficients obtained in the two Swedish testing studies of NUFFE,18,19 but slightly lower than the coefficient of 0.77 in the Norwegian testing study among older hospital patients.21 The value of 0.71 is considered to be a sufficient value of homogeneity.27,32

All of the item-to-total correlations were statistically significant, and are more than in earlier testing studies of the Swedish,18,19 Hungarian,33 and Norwegian versions of NUFFE.21,23 However, two of the correlation coefficients, for item 11 (fluid intake) and item 13 (eating assistance), were lower than the lowest recommended value of 0.20 for item-to-total correlations according to Streiner and Norman.27 The finding that these two items showed low correlations with the total scale can be explained by the fact that most of the people in this study gave the same answer, ie, that they had a good fluid intake and did not need assistance with eating. This highlights the importance of a heterogeneous sample when performing instrument testing studies. Despite a large study sample, there is a challenge to get a sufficiently heterogeneous sample. Regarding nutritional testing studies, the optimal sample exists when the sample can show all degrees of nutritional status, ie, from no risk of undernutrition to high risk of undernutrition, and, thereby, all response alternatives of the items in the nutritional screening instrument are used. However, fluid intake and eating assistance have relevance in nutritional screening of older people, and, according to Streiner and Norman,27 a high homogeneity is not crucial regarding items that can be seen as causal indicators reflecting the underlying construct, ie, the risk of undernutrition.

A low correlation coefficient of −0.37 was obtained as a measure of concurrent validity when total scores of NUFFE-NO and MNA-SF were correlated. According to Terwee et al,32 a correlation with the “gold standard” has to be at least 0.7. In an earlier study23 using the MNA-SF as a standard for NUFFE-NO among a group of older home-dwelling people, the corresponding correlation coefficient was rs = −0.26. This implies that MNA-SF is not the most suitable “gold standard” for NUFFE-NO. An explanation could be that the construction of the two instruments was unequal regarding the number of items. This could also be a reason for the low Cronbach’s alpha coefficient of MNA-SF, which only has six items.27 The content of the items in the two instruments is also quite different. When using the MNA as a standard in two other testing studies19,21 of the Swedish and Norwegian versions of NUFFE among older hospital patients, correlation coefficients of −0.74 were obtained for both studies, which strengthens the full form of the MNA as a suitable standard. The MNA and NUFFE-NO have a more equal number of items and there are similarities regarding the content of the items.

Validity was supported when testing construct validity, because NUFFE-NO could separate known nutritionally at-risk groups. A known group with higher NUFFE-NO scores and thereby at higher risk of under-nutrition contained individuals who received food distribution, ie, meals-on-wheels. Johansson et al5 found that older home-dwelling people at risk of undernutrition had a reduced ability to perform activities of daily living and used home-help services and meals-on-wheels more frequently than did well nourished people. Furthermore, other known groups at nutritional risk that were identified by NUFFE-NO were those receiving home nursing and those having a chronic disease/handicap. These groups being nutritionally at-risk can be explained by the fact that higher care needs are associated with a higher prevalence of undernutrition among older community-dwelling people.7 Another known group at nutritional risk identified by NUFFE-NO included people who were not satisfied with life. According to Johnson,34 an association (albeit not statistically significant) was found between being at nutritional risk and having a lower level of satisfaction. In a study by Lengyel et al,35 an association was found between consuming vegetables and fruit and greater life satisfaction in older community-dwelling men. Moreover, an association between not being satisfied with life and being at nutritional risk has been found using the Swedish22 and Norwegian versions of NUFFE36,37 in older hospitalized patients as well as among older home-dwelling people. Likewise, perceived ill health was found to be associated with a higher risk of undernutrition, as reported by other studies.4,22,36,38

A low cut-off point (≥4) for NUFFE-NO was found for identifying older people at nutritional risk with the MNA-SF used as the standard, compared with the cut-off point of ≥6 found in other studies of older hospitalized patients using the MNA as a standard.21,22 The accuracy (74%) of the cut-off point ≥ 4 was considered satisfactory. The area under the ROC curve can also be seen as a measure of accuracy,24 and an area of 0.73 for a cut-off point ≥ 4 was considered to be adequate in order to distinguish people according to an external standard where the area has to be at least 0.70.32 In the earlier testing study of NUFFE-NO among older home-dwelling people, using the MNA-SF as a standard, exactly the same cut-off point of ≥4 was found. However, the area under the ROC curve was larger, ie, 0.77.23 According to Söderhamn et al,36 the NUFFE-NO using a cut-off point ≥ 6, the MNA, and the MNA-SF were found to identify approximately the same number of nutritionally at-risk patients. However, the present study and that by Tomstad et al23 highlight the fact that the MNA-SF and MNA are not equivalent when used as standards for testing sensitivity and specificity of NUFFE-NO in order to determine a cut-off point.

Limitations

The present study has a number of limitations. The response rate was rather low and, in accordance with Buckwalter,39 it is a challenge to include older persons in studies. Therefore, it may be assumed that those who did not respond were older and thereby could be less healthy and, therefore, may be at risk of undernutrition. It would have been advantageous in this investigation if more people who were nutritionally at-risk had been included. However, the sample size was considered to be adequate for estimation of sensitivity and specificity values of 71% and 74%, respectively,40 because the expected prevalence of home-dwelling people at nutritional risk, according to other studies,37,38 is around 20%. A high negative predictive value (94%) was obtained for the cut-off point ≥ 4. According to Fletcher and Fletcher,24 such a value indicates a low prevalence of the issue of concern.

A cross-sectional design with a single data collection has limitations in terms of testing stability and equivalence as measures of reliability, because high inter-rater reliability (as a measure of equivalence) is sufficient indication of a reliable nutritional instrument.41 However, the NUFFE-NO has been tested previously among older hospital patients regarding its stability by test-retest with good or very good agreement for most items.21

Use of the MNA as a standard is preferable because its construction and number of items are more similar to the NUFFE-NO than to the MNA-SF. However, data collection in this study, which was done by a postal questionnaire, raises questions regarding the choice of a standard. To use the MNA was out of the question, because the measurement of mid-arm and calf circumference included in this instrument is difficult for participants to perform by themselves. Therefore, the alternative choice was to use the MNA-SF. However, it was also desirable to test the MNA-SF as a standard once again among older home-dwelling people in order to be able to compare the present results with the testing results presented by Tomstad et al23 who used the MNA-SF as a standard for the NUFFE-NO.

Conclusion

The NUFFE-NO shows adequate psychometric properties regarding homogeneity and construct validity. MNA-SF was not found to be the most suitable standard to use, because a low correlation coefficient was obtained as a measure of concurrent validity and a lower cut-off point was found compared with another study21 using the MNA as a standard for the NUFFE-NO. Despite adequate values of sensitivity, specificity, and accuracy for the cut-off point ≥ 4 obtained, it should not be recommended for use in practice and research, because many false-positive nutritionally at-risk persons would then be identified. Further studies of suitable design need to be performed among older home-dwelling people to test the NUFFE-NO regarding its reliability and validity, including sensitivity and specificity, using the MNA as a standard.