| Literature DB >> 23173849 |
Margo L Barr1, Jason J van Ritten, David G Steel, Sarah V Thackway.
Abstract
BACKGROUND: In Australia telephone surveys have been the method of choice for ongoing jurisdictional population health surveys. Although it was estimated in 2011 that nearly 20% of the Australian population were mobile-only phone users, the inclusion of mobile phone numbers into these existing landline population health surveys has not occurred. This paper describes the methods used for the inclusion of mobile phone numbers into an existing ongoing landline random digit dialling (RDD) health survey in an Australian state, the New South Wales Population Health Survey (NSWPHS). This paper also compares the call outcomes, costs and the representativeness of the resultant sample to that of the previous landline sample.Entities:
Mesh:
Year: 2012 PMID: 23173849 PMCID: PMC3536693 DOI: 10.1186/1471-2288-12-177
Source DB: PubMed Journal: BMC Med Res Methodol ISSN: 1471-2288 Impact factor: 4.615
Comparison of survey methods, 2011 NSW Population Health Survey and 2012 NSW Population Health Survey
| Landline RDD sample frame for each of the administrative strata were generated using “best fit” postcodes for the geography (exchange district and charge zone) associated with the Australian Communications and Media Authority (ACMA) phone number ranges for NSW [ | The RDD mobile sample frame was developed using all known Australian mobile prefixes and then using proprietary software [ | ||
| Stratified two-stage cluster sample design, with: strata defined by health administration areas; simple random sampling of clusters (household telephone numbers) within each stratum; and simple random sampling of population elements (household residents) within each cluster. | Two-stage cluster sample design with simple random sampling of the mobile telephone numbers (adult population element) and simple random sampling of children in household (child population elements). | ||
| The questionnaire included questions on: health behaviours, health status, social determinants, demographics (including number of adults and children in the household) and landline phone ownership ("How many residential telephone numbers do you have? Do not include mobile phone numbers or dedicated FAX numbers or modems."). The actual questions in the questionnaire are available on the survey website. | |||
| 3000 persons per quarter with equal numbers in each of the strata | 2000 persons per quarter | 1000 persons per quarter | |
| Business landline numbers, non-NSW residential numbers | Business mobile numbers, non-NSW residential mobile numbers or mobile numbers owned by a child under the age of 16 years. | ||
| Data collection was undertaken using SAWTOOTH WinCati version 4.2 and trained interviewers from the in-house NSW Ministry of Health’s CATI facility. | |||
| The interviewers rang the randomly ordered landline numbers consecutively to try and contact households and convince the household and the respondent to participate in the survey. Up to 12 attempts were made to establish contact and if possible secure an interview with the selected respondent within a household. | The interviewers rang the randomly ordered mobile phone numbers consecutively to try and contact the owner of the phone. Because mobile numbers could be located anywhere in Australia initial calls were timed to accommodate different time zones across Australia. Up to 12 attempts were made to establish contact and if possible secure an interview with the mobile phone holder. | ||
| One person from the household was randomly selected for inclusion in the survey. If the selected respondent was a child under the age of 16 years, a parent or carer completed the interview on their behalf. | The mobile phone holder was selected. If the owner of the mobile phone was a parent of a child under 16 years of age they were asked at the end of the interview if they or the main carers would agree to being contacted at a later date to undertake an interview about one of their children chosen at random. | ||
| Adjust for differences in the probabilities of selection among subjects (using household size and number of landline phones in household). | Adjust for differences in the probabilities of selection among subjects (using number of mobile phones owned by respondent and ratio of mobile phone sample to mobile phone population and number of children in the household). | ||
Call outcome information and rates for by sample frame and overall (combined)
| T=Total phone numbers used | 21350 | 17534 | 38884 |
| I=Complete Interviews (1.1) | 2171 | 1224 | 3395 |
| P=Partial | 0 | 0 | 0 |
| R=Refusal and break off (2.1) | 868 | 457 | 1325 |
| NC=Non Contact (2.2) | 660 | 238 | 898 |
| O=Other (2.0, 2.3) | 1163 | 767 | 1930 |
| e: estimated proportion of cases of unknown eligibility that are eligible. | |||
| UH=Unknown Household (3.1) | 4553 | 5450 | 10003 |
| UO=Unknown other (3.2-3.9) | 0 | 0 | 0 |
| NE=Not eligible | 11935 | 9462 | 21397 |
| | |||
| | |||
| | |||
| | |||
| | | | |
| Average survey length (mins) | 17.2 | 15.6 | |
| Average call costs (per completed interview) | $7.45 | $38.90 | |
| Average interviewer time costs (per completed interview) | $23.68 | $35.53 | |
| Total average costs (call costs plus interviewer time costs) | $31.13 | $74.42 | |
| Telephone numbers used to get a contact: T/(I+R+NEI+NEB) | 1.9 | 2.1 | |
| Telephone numbers used to get an eligible contact: T/(I+R) | 7.0 | 10.5 | |
| Telephone numbers used to get a completed interview: T/I | 9.8 | 14.4 | |
| | | | |
| Response Rate 1: I/(I+P) + (R+NC+O) + (UH+UO) | 23.1% | 15.0% | 18.6% |
| Response Rate 2: (I+P)/(I+P) + (R+NC+O) + (UH+UO) | 23.1% | 15.0% | 18.6% |
| Response Rate 3: I/((I+P) + (R+NC+O) + e(UH+UO) ) | 35.1% | 31.5% | 33.1% |
| Response Rate 4: (I+P)/((I+P) + (R+NC+O) + e(UH+UO) ) | 35.1% | 31.5% | 33.1% |
| | | | |
| Cooperation Rate 1: I/(I+P)+R+O) | 51.7% | 50.0% | 50.7% |
| Cooperation Rate 2: (I+P)/((I+P)+R+O)) | 51.7% | 50.0% | 50.7% |
| Cooperation Rate 3: I/((I+P)+R)) | 71.4% | 72.8% | 72.2% |
| Cooperation Rate 4: (I+P)/((I+P)+R)) | 71.4% | 72.8% | 72.2% |
| | | | |
| Refusal Rate 1: R/((I+P)+(R+NC+O) + UH + UO)) | 9.2% | 5.6% | 7.2% |
| Refusal Rate 2: R/((I+P)+(R+NC+O) + e(UH + UO)) | 14.0% | 11.7% | 12.8% |
| Refusal Rate 3: R/((I+P)+(R+NC+O)) | 17.9% | 17.0% | 17.4% |
| | | | |
| Contact Rate 1: (I+P)+R+O / (I+P)+R+O+NC+ (UH + UO) | 44.6% | 30.1% | 36.5% |
| Contact Rate 2: (I+P)+R+O / (I+P)+R+O+NC + e(UH+UO) | 68.0% | 62.9% | 65.1% |
| Contact Rate 3: (I+P)+R+O / (I+P)+R+O+NC | 86.4% | 91.1% | 89.1% |
Notes to Table 2: AAPOR Categories [28] are as follows: Interview (I) = Complete interviews (1.1); Refusal (R) = Respondent refusal (2.112), Household refusal and break off (2.1); Non-contact (NC) = Respondent never available(2.2), away for duration of survey (2.21); Other (O) = Respondent physically or mentally unable to complete interview (2.32), Non-translated language(2.333), Other non-refusal : hang up said nothing/ terminated by interviewer/technical problems (2.3); Unknown Household (UH) = Engaged busy (3.12), No answer (3.13), always answering machine (3.14); Not eligible (NE) = Fax data line (4.2), Non-working number (4.3), unusual tone (4.31), Business, government office, other organizations (4.51), Non-eligible respondent: not in NSW/mobile owned/answered by child (4.7);
Calculation of each rate for Overall = (RA* (Na+λNabA))+ (RB * (Nb+(1-λ)NabB)) where R frame rate; N population proportion; λ=overlap adjustment (set 0.5); A landline sample frame; B denotes mobile sample frame; a landline-only phone users; b mobile-only phone users; ab denotes both mobile phone and landline users.
Comparison of the demographic profile of the mobile frame and the landline frame respondents
| Age groups | 0-15 | 8.5 | 12.3 | 11.4 | 6.0 | 15.8 | ||||
| 16-24 | 17.1 | 10.8 | 12.4 | 0.5 | 4.9 | |||||
| 25-34 | 41.8 | 16.6 | 23.1 | 1.6 | 6.4 | |||||
| 35-44 | 12.3 | 16.0 | 15.0 | 5.2 | 8.0 | |||||
| 45-54 | 10.1 | 19.3 | 16.9 | 7.3 | 14.3 | |||||
| 55-64 | 7.3 | 14.9 | 12.9 | 16.8 | 22.6 | |||||
| 65-74 | 2.5 | 7.9 | 6.5 | 23.3 | 17.3 | |||||
| 75-high | 0.3 | 2.2 | 1.7 | 39.3 | 10.6 | |||||
| Sex | Male | 48.4 | 48.3 | 0.052 | 48.4 | 0.052 | 42.9 | 38.0 | ||
| Female | 51.6 | 51.7 | 51.6 | 57.1 | 62.0 | |||||
| Aboriginality | Aboriginal | 5.1 | 1.8 | 0.76 | 2.6 | 0.78 | 2.4 | 2.2 | ||
| Non-Aboriginal | 94.9 | 98.2 | 97.4 | 97.6 | 97.8 | |||||
| Country of birth | Australia | 60.8 | 79.4 | 1.00 | 64.9 | 76.6 | 80.1 | |||
| Overseas | 39.2 | 20.6 | 35.1 | 23.4 | 19.9 | |||||
| Marital status | Married | 31.3 | 61.8 | 54.0 | 45.3 | 56.0 | ||||
| Widowed | 1.9 | 3.5 | 3.1 | 28.7 | 10.5 | |||||
| Separated | 3.5 | 3.2 | 3.3 | 3.4 | 4.1 | |||||
| Divorced | 7.4 | 7.0 | 7.1 | 10.8 | 12.6 | |||||
| Never married | 55.8 | 24.5 | 32.5 | 11.8 | 16.8 | |||||
| Income | < $20,000 | 19.0 | 0.32 | 9.9 | 12.0 | 46.8 | 19.7 | |||
| $20,001-$40,000 | 14.7 | 15.7 | 15.4 | 24.5 | 18.9 | |||||
| $40,001-$60,000 | 16.8 | 14.3 | 14.9 | 9.3 | 16.2 | |||||
| $60,001-$80,000 | 14.2 | 13.9 | 14.0 | 4.1 | 11.5 | |||||
| $80,000 plus | 35.3 | 46.3 | 43.7 | 15.2 | 33.7 | |||||
Notes: Chi-squared testing, setting the significance level of p<0.05, was used for the comparisons between the mobile phone frame (mobile-only, both and total) sample demographic categories and the total landline frame sample.
Sample comparisons to the latest population profile for NSW
| Age groups | 0-15 | 14.1 | 11.4 | 13.4 | 12.1 | 20.5 | ||||
| 16-24 | 4.1 | 12.4 | 5.8 | 7.3 | 11.6 | |||||
| 25-34 | 5.6 | 23.1 | 10.2 | 13.2 | 13.6 | |||||
| 35-44 | 7.6 | 15.0 | 8.2 | 9.9 | 14.1 | |||||
| 45-54 | 13.0 | 16.9 | 12.7 | 13.4 | 13.8 | |||||
| 55-64 | 21.6 | 12.9 | 19.8 | 17.4 | 11.7 | |||||
| 65-74 | 18.4 | 6.5 | 16.4 | 14.1 | 7.8 | |||||
| 75-high | 15.6 | 1.7 | 13.5 | 12.5 | 6.9 | |||||
| Sex | Male | 38.9 | 48.4 | 0.85 | 40.1 | 0.07 | 42.8 | 0.20 | 49.3 | |
| Female | 61.1 | 51.6 | 59.9 | 57.2 | 50.7 | |||||
| Aboriginality | Aboriginal | 2.2 | 0.86 | 2.6 | 0.94 | 2.6 | 0.96 | 2.6 | 0.96 | 2.5 |
| Non-Aboriginal | 97.8 | 97.4 | 97.4 | 97.4 | 97.5 | |||||
| Country of birth | Australia | 79.4 | 64.9 | 0.42 | 77.1 | 0.07 | 73.4 | 0.30 | 68.6 | |
| Overseas | 20.6 | 35.1 | 22.9 | 26.6 | 31.4 | |||||
| Marital status | Married | 54.1 | 54.0 | 0.76 | 51.3 | 51.5 | 0.08 | 49.4 | ||
| Widowed | 13.7 | 3.1 | 12.2 | 11.1 | 5.8 | |||||
| Separated | 4.0 | 3.3 | 3.9 | 3.7 | 3.1 | |||||
| Divorced | 12.3 | 7.1 | 11.7 | 10.2 | 8.3 | |||||
| Never married | 15.9 | 32.5 | 20.9 | 23.5 | 33.4 | |||||
| Income* | < $20,000 | 24.0 | 12.0 | 23.4 | 21.9 | 0.05 | 13.7 | |||
| $20,001-$40,000 | 19.8 | 15.4 | 19.2 | 18.5 | 19.8 | |||||
| $40,001-$60,000 | 15.1 | 14.9 | 15.3 | 14.7 | 16.9 | |||||
| $60,001-$80,000 | 10.4 | 14.0 | 10.8 | 11.2 | 19.8 | |||||
| $80,000 plus | 30.8 | 43.7 | 31.3 | 33.7 | 29.8 | |||||
Notes: # Calculation numbers for combined frame = ((Sa+λSabA)+ (Sb+(1-λ)SabB) where S =sample; λ=overlap adjustment (set to 0.5); A landline sample frame; B denotes mobile sample frame; a landline-only phone users; b mobile-only phone users; ab denotes both mobile phone and landline users.
* Census income information was converted from weekly income to annual income for the comparison.
χ2 testing, setting the significance level of p<0.05, was used for the comparisons between the sample demographic categories and the population profile (2011 census).