Julia Bowman1,2, Lise Mogensen3, Elisabeth Marsland4, Natasha Lannin2,5,6. 1. Research and Evaluation Service, Justice Health and Forensic Mental Health Network, Malabar, New South Wales, Australia. 2. Occupational Therapy Department, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia. 3. School of Medicine, University of Western Sydney, Campbelltown, New South Wales, Australia. 4. School of Science and Health, University of Western Sydney, Penrith South, New South Wales, Australia. 5. Occupational Therapy Department, Alfred Health, Melbourne, Australia. 6. John Walsh Centre of Rehabilitation Research, Northern Clinical School, Sydney Medical School, University of Sydney, St Leonards, New South Wales, Australia.
Abstract
INTRODUCTION: Goal setting is a complex skill. The use of formal goal writing procedures (including the use of the SMART goal model) has been advocated. However, a standardised method of writing and evaluating SMART goals is currently lacking. This study comprised of two phases. The aims of phase one was to (i) develop the SMART Goal Evaluation Method (SMART-GEM) based on a SMART goal model; and (ii) investigate the content validity of the SMART-GEM. The aim of phase two of the study was to test the inter-rater reliability of the SMART-GEM. METHODS: Development of the SMART- GEM involved defining and constructing evaluation criteria suitable for auditing goal statements. A content validity assessment was conducted using an expert panel of Occupational Therapists (n = 10). Inter-rater reliability of the SMART-GEM was examined using a purposive sample of multiple raters (n = 24). RESULTS: The SMART- GEM was rated as having good content validity (individual items CVI ranged from 0.90 to 1.00; total SMART- GEM CVI = 0.99, ρ = 0.05). Agreement between raters on individual items ranged from poor (κ = 0.254) to excellent (κ = 0.965) and agreement of overall grades was fair to good (κ = 0.582). Inter-rater agreement on total scores was found to be very good (ICC = 0.895, 95% CI = 0.743 to 0.986, ρ = 0.001) with excellent internal consistency (α = 0.995). CONCLUSION: The SMART-GEM demonstrated good construct validity and very good inter-rater reliability on total score and shows promise as a standardised method to writing and evaluating clinical goals.
INTRODUCTION: Goal setting is a complex skill. The use of formal goal writing procedures (including the use of the SMART goal model) has been advocated. However, a standardised method of writing and evaluating SMART goals is currently lacking. This study comprised of two phases. The aims of phase one was to (i) develop the SMART Goal Evaluation Method (SMART-GEM) based on a SMART goal model; and (ii) investigate the content validity of the SMART-GEM. The aim of phase two of the study was to test the inter-rater reliability of the SMART-GEM. METHODS: Development of the SMART- GEM involved defining and constructing evaluation criteria suitable for auditing goal statements. A content validity assessment was conducted using an expert panel of Occupational Therapists (n = 10). Inter-rater reliability of the SMART-GEM was examined using a purposive sample of multiple raters (n = 24). RESULTS: The SMART- GEM was rated as having good content validity (individual items CVI ranged from 0.90 to 1.00; total SMART- GEM CVI = 0.99, ρ = 0.05). Agreement between raters on individual items ranged from poor (κ = 0.254) to excellent (κ = 0.965) and agreement of overall grades was fair to good (κ = 0.582). Inter-rater agreement on total scores was found to be very good (ICC = 0.895, 95% CI = 0.743 to 0.986, ρ = 0.001) with excellent internal consistency (α = 0.995). CONCLUSION: The SMART-GEM demonstrated good construct validity and very good inter-rater reliability on total score and shows promise as a standardised method to writing and evaluating clinical goals.
Authors: Hninyee Win; Samantha Russell; Betsy C Wertheim; Victoria Maizes; Robert Crocker; Audrey J Brooks; Ruben Mesa; Jennifer Huberty; Holly Geyer; Ryan Eckert; Ashley Larsen; Krisstina Gowin Journal: JMIR Form Res Date: 2022-03-31
Authors: Ahmad Hatim Sulaiman; Zuraida Ahmad Sabki; Mohd Johari Jaafa; Benedict Francis; Khairul Arif Razali; Aliaa Juares Rizal; Nor Hazwani Mokhtar; Johan Arif Juhari; Suhaila Zainal; Chong Guan Ng Journal: Healthcare (Basel) Date: 2020-07-24
Authors: Panagiotis D. Bamidis; Evdokimos I. Konstantinidis; Andoni Beristain Iraola; Roberto Álvarez Sánchez; Santiago Hors-Fraile; Despoina Petsani; Michail Timoleon; Unai Díaz-Orueta; Joanne Carroll; Louise Hopper; Gorka Epelde; Jon Kerexeta Journal: Int J Environ Res Public Health Date: 2021-06-26 Impact factor: 3.390
Authors: Hashem Abu Tariah; Amal Saud Aljehani; Dana Yasser Alenazi; Dona Abdularhman Alturaif; Malak Nabit Alsarhani Journal: Occup Ther Int Date: 2020-04-25 Impact factor: 1.448