Health complexity assessment in primary care: A validity and feasibility study of the INTERMED tool.

BACKGROUND: Health complexity includes biological, psychological, social, and health systems. Having complex health needs is associated with poorer clinical outcomes and higher healthcare costs. Care management for people with health complexity is increasingly recommended in primary health care (PHC). The INTERMED complexity assessment grid showed adequate psychometric properties in specialized settings. This study aimed to evaluate INTERMED's validity and feasibility to assess health complexity in an adult PHC population.
METHOD: The biopsychosocial health care needs of 230 consecutive adult patients from three Brazilian PHC services were assessed using the INTERMED interview. Participants with a total score >20 were classified as "complex". Quality of life was measured using the World Health Organization Quality of Life BREF (WHOQOL-BREF); symptoms of anxiety and depression using the Hospital Anxiety and Depression Scale (HADS); social support using the Medical Outcomes Study-Social Support Survey (MOS-SSS); comorbidity levels using the Charlson Comorbidity Index (CCI). We developed two questionnaires to evaluate health services use, and patient perceived feasibility of INTERMED.
RESULTS: 42 participants (18.3%) were classified as "complex". A moderate correlation was found between the total INTERMED score and the total scores of WHOQOL-BREF (rho = - 0.59) and HADS (rho = 0.56), and between the social domains of INTERMED and MOS-SSS (rho = -0.44). After adjustment, the use of PHC (β = 2.12, t = 2.10, p < 0.05), any other health care services (β = 3.05, t = 3.97, p < 0.01), and any medication (β = 3.64, t = 4.16, p < 0.01) were associated with higher INTERMED scores. The INTERMED internal consistency was good (ω = 0.83), and the median application time was 7 min. Patients reported satisfaction with the questions, answers, and application time.
CONCLUSION: INTERMED displayed good psychometric values in a PHC population and proved promising for practical use in PHC.

Introduction

“Health complexity” can be defined as “interference with the achievement of expected or desired health and cost outcomes due to the interaction of biological, psychological, social, and health systems factors” [1]. These factors interact dynamically and non-linearly in an idiosyncratic manner for each individual [2-4]. The importance of incorporating the assessment of health complexity into the management of PHC patients for delivering high-quality care with desirable outcomes is increasingly recognized worldwide [5-7]. This is particularly critical in PHC settings that include one of the most medically and socially vulnerable groups of patients in the world, such as Brazil [5]. Despite this acknowledgement, healthcare complexity assessment has been suboptimal or has not been integrated into PHC practice at all, partially because efficient methods for its evaluation in adult PHC are still lacking [8].

The INTERMED Complexity Assessment Grid (adult version) is an efficient tool for assessing biopsychosocial complexity and improving the communication flow between professionals, patients, and services [9]. According to a systematic review, it is one of the best instruments of its kind [10]. Its development was methodologically robust [11], with validation in diverse populations (in- and outpatients in secondary, tertiary, and emergency services, with a range of health problems [12, 13]), using different versions (face-to-face interview [14], self-assessment [15], pediatric [16], adults [14], and elderly [13]), and showing predictive validity regarding mortality [17], healthcare costs [18], and quality of life [19]. Case managers already use INTERMED to identify and coordinate comprehensive care for people with high levels of health complexity [20]. However, to date, only one small study (n = 55) assessed the psychometric properties of INTERMED in a PHC context, which focused mainly on the self-assessment version, and of which no full-text article is available [8].

This study aimed to evaluate the psychometric properties of the INTERMED Complexity Assessment Grid Adult Interview version in a Brazilian PHC population. We hypothesized that INTERMED could have adequate validity and feasibility (applicability and acceptability) in a PHC population.

Methods

Sampling technique

We aimed for a 10:1 ratio (i.e., 10 patients per each of the 20 INTERMED items) [21, 22], resulting in an estimated minimum of 200 patients and their health care records. Assuming a dropout rate of 15% in a planned second wave of the collection (not reported here), we decided to recruit an additional 30 people. To maximize the chance of gathering representative data, we deployed a quota sampling method by dividing the population into gender and age groups (18–30, 31–40, 41–50, 51–60, and ≥60 years) [23].

Recruitment procedures

Patients were recruited from three PHC services located in Ribeirão Preto city (São Paulo, Southeast region of Brazil).

All adults (age ≥ 18 years) who sequentially arrived at the reception of one of the health services, whether for medical appointments, to pick up medication at the service’s pharmacy, to accompany another patient, or to schedule appointments, were approached by a researcher. To be recruited, patients were required to live in the area covered by the service, have their health records there, and speak, read, and write Portuguese at a sufficient level to complete the instruments. Patients who were unable to understand the interview (e.g., due to cognitive impairment or learning disability) or did not complete all the instruments were excluded. When a patient chose not to participate in the study, the researcher invited the next sequential patient. We obtained informed written consent before participation, including consent to review the health care records.

Instruments

Participant demographic characteristic questions

We collected five types of demographic information from the participants: gender, age, ethnicity, education, and occupation.

INTERMED

The INTERMED tool is a semi-structured interview that synthesizes data from four health-related aspects (domains): 1) biological, 2) psychological, 3) social, and 4) health system, assessed in the context of time (history, current state, and vulnerability/prognosis) [14, 24, 25]. Within each of the four domains, there are five specific variables (items), totaling 20. Each item has specific clinical anchor points described, ranging from 0 (no vulnerability/only health education) to 3 (severe vulnerability/need for immediate or intensive care) [24] (Fig 1). In previous studies carried out in specialized services, a 20/21 cutoff was proposed to differentiate “complex” from “non-complex” cases [26].

Fig 1

INTERMED domains and temporal context with their variables.

The INTERMED interview involves 16 lead questions related to the four domains and one question about satisfaction with the interview. Based on the information obtained from the answers to the 16 questions mentioned, the health professional assesses the four items of vulnerability/prognosis (one for each domain). A health professional is free to follow the topic guide, skip, or modify specific questions according to what a patient said spontaneously. The Portuguese Brazilian version of these questions was used, which underwent cultural adaptation and proved valid for use in an inpatient population in Brazil [27]. Two researchers responsible for data collection and/or health record review, an occupational therapist (CAO), and a nurse (LLC) were trained in using the INTERMED by the authors of its Brazilian Portuguese version [27].

Hospital Anxiety and Depression Scale (HADS)

The HADS is a 14-item scale designed to assess anxiety (HADS-A) and depression (HADS-D) symptoms in medical patients [28]. Items are rated on a 4-point severity scale (0 to 3), where the higher score indicates a worse condition. The total score is the sum of the 14 items, with 7 items per subscale. For each subscale, the score is the sum of the seven items (ranging from 0 to 21). The HADS was used in previous research to assess the validity of the INTERMED psychological domain [11].

Medical Outcomes Study–Social Support Survey (MOS-SSS)

The MOS-SSS is a 19-item scale designed to assess social support in medical patients [29]. The 19 items cover four domains (emotional/informational support, instrumental support, positive social interaction, and affection). Items are rated on a 7-point Likert rating scale (1–7). The overall score is the mean score of all items and ranges from 1 to 7; a higher score indicates a higher level of perceived social support.

WHO Quality of Life–Bref (WHOQOL-BREF)

The WHOQOL-BREF is a 26-item scale consisting of four domains: physical health (7 items), psychological health (6 items), social relationships (3 items), and environmental health (8 items), as well as QOL and general health items [30]. Items are rated on a 5-point rating scale (1–5), which is stipulated as a five-point ordinal scale. The scores are then transformed linearly to a scale of 0–100.

Charlson Comorbidity Index (CCI)

The CCI evaluates the comorbidity level. It consists of 19 selected conditions, including 18 physical health conditions and dementia, which are weighted from 1 to 6 and summed to an index on a 0–33 scale [31]. A higher score reflects the greater number and the seriousness of comorbid diseases.

Questions to assess health system use

To assess health system use, we developed a questionnaire for the evaluation of health services use, with six dichotomous questions based on the SABE study [32]. The questions explored three aspects of service use within a six-month period (prior to the interview). Divided in three domains: 1) PHC use (“Have you consulted with a PHC professional in the last six months, excluding today?”); 2) other health care services (than PHC) (“Have you been admitted to a hospital?”; “Have you consulted with a specialist physician?”; “Have you had a consultation at a specialized mental health service?”; “Have you had a consultation at an A&E department?”) and; 3) use of any medication (“Do you take any medications?”). In the PHC use variable, patients scored with “1” if they had been in an appointment within six months before the survey, excluding the day of the interview. For the variable of other health care services (than PHC), which had multiple questions, only one "yes” answers were counted, regardless of the corresponding questions. This is because the aim was to know whether the patient had used such services, and not the quantity or type of services used.

Feasibility questionnaire

To examine the patient-perceived feasibility of INTERMED use, we developed a questionnaire with seven questions, each with five Likert response options [33], divided posteriorly into dichotomous groups (satisfactory and unsatisfactory in relation to feasibility). The questions focused on the acceptability of INTERMED (the understanding of each question, how to answer the question, and the length of the interview), and applicability (the relevance of asking the questions within each of the four domains). The feasibility questionnaire was administered shortly after the patient completed the INTERMED.

Data collection procedures

The study was conducted between November 2018 and June 2019. To determine the order of data collection across the three PHC units, we followed the daily routines of those units for a week. After this, data collection took place in each PHC unit for two months, from Monday to Friday, between 7 a.m. and 5 p.m. One researcher (CAO) administered all the listed instruments, first the INTERMED, then the questionnaire about the feasibility and, finally, the other instruments (HADS, MOS-SSS, WHOQOL-BREF, and CCI). For each participant, the CAO measured the time taken to apply INTERMED. All data were collected and managed using Research Electronic Data Capture (REDCap) [34], a web-based platform data capture tool hosted at the Ribeirão Preto Medical School of University of São Paulo (Department of Social Medicine) - https://research.fmrp.usp.br/.

Health records review

Following primary data collection using the listed instruments, participants had their health records (both paper and electronic) jointly reviewed by two researchers (CAO, LLC) using their clinical and INTERMED knowledge. The purpose of analyzing the health records was to understand if PHC health professionals can obtain biopsychosocial information from existing patient data, without a need for conducting INTERMED interviews (i.e., as a mean to evaluate the practical applicability of the tool evaluate, as an aspect of feasibility). INTERMED questions were applied to health records, and any information that could be filled out completely on the instrument was marked as present. The health records data did not influence the INTERMED interview scores. In previous research, INTERMED was used in a similar way [35].

Statistical analysis

All data analyses were conducted using the free and open software Jamovi version 1.6.12.

Descriptive statistics of the sample and INTERMED’s feasibility

The remaining data were summarized using simple descriptive statistics. We calculated frequencies for: 1) demographic characteristics of the study population (i.e., gender, age, ethnicity, education, and occupation categories); 2) patient responses to the seven questions on the acceptability and applicability of INTERMED, and 3) information on each of the INTERMED variables identified in the health records [35].

Convergent validity

We performed the Shapiro-Wilk test to examine the distribution of INTERMED data associated with the other four instruments (i.e., HADS, MOSS-SSS, WHOQOL-BREF, and CCI). The results did not meet the prerequisites for normality and homogeneity; therefore, Spearman’s correlation analysis was performed. Spearman coefficients (rho) ranging from 0.10 to < 0.40, from 0.40 to < 0.70, from 0.70 to < 1.00, were interpreted as weak, moderate and strong respectively [36].

Predictive validity

Hierarchical multiple linear regression analysis was used to test if “health system” use could predict INTERMED-based biopsychosocial complexity. One dependent variable was entered into the model (i.e., continuous INTERMED scores) and the three independent dichotomous variables (i.e., PHC use, other health care services (than PHC), and use of any medication). We conducted a study in three stages to select the best predictive model: first, we built the model through forward selection using the AIC as a criterion and controlling the parameters of age and sex. Second, we analyzed the coefficient determination ratio to detect the influential points and build a new model without the influential points using the same AIC criterion [37, 38]. Finally, we analyzed the Shapiro-Wilk test to diagnose the model and to confirm that the withdrawal of the influential outliers was consistent with the normality requirement of the regression model.

Internal consistency

The internal consistency of INTERMED was measured by omega coefficient and with values ranging from <0.5, from 0.5 to 0.6, from 0.6 to 0.7, from 0.7 to 0.8, from 0.8 to 0.9, and ≥ 0.9, which were interpreted as unacceptable, poor, questionable, acceptable, good, and excellent, as per the interpretation of an alpha coefficient [39].

Ethics approval

The Research Ethics Committee of the Community Health Center of the Ribeirão Preto Medical School of the University of São Paulo approved the study (n° 99566718.0.0000.5414 in 10/2018).

Results

We invited two hundred and forty-three (243) patients, of whom five did not agree to participate, and two hundred and thirty-eight (238) agreed. Eight people were excluded because they chose not to complete all the questionnaires. Table 1 shows the socio-demographic characteristics of the 230 participants [mean age = 45.92 (±15.43) years, 56.1% female, 53.5% reported being white, 43.5% reported incomplete higher education or complete high school education, and 40.4% were employed].

Table 1

Socio-demographic characteristics of the 230 participants, PHC patients.

Characteristic		Frequency	%
Age group	18–30	39	17.0
	31–40	59	25.7
	41–50	45	19.6
	51–60	43	18.7
	60+	44	19.1
Gender	Female	129	56.1
	Male	101	43.9
Ethnicity	White	123	53.5
	Black	23	10.0
	Brown	84	36.5
Education level	Illiterate/incomplete primary education	20	8.7
	Primary education /Incomplete secondary education	51	22.2
	Secondary education/high school incomplete	38	16.5
	High school/incomplete higher education	100	43.5
	Graduated	21	9.1
Occupation	Employee	93	40.4
	Unemployed	53	23.0
	Retired	50	21.7
	Freelance	30	13.0
	Student	4	1.7

The INTERMED minimum total score value was zero, the maximum value was 38, the mean was 13.57 (±7.54), and the median was 13. The INTERMED profiles of the participants according to the clinical anchor points of each item (see S1 Table). A total of 42 (18.3%) participants were classified as “complex”, according to the 20/21 cutoff score [21]. Ninety-two patients (40.0%) presented physical/mental multimorbidity, 34 (14.8%) were considered to have health complexity, and 32 (13.9%) had only physical multimorbidity, of whom 6 (2.6%) were “complex”. Of the remaining 106 patients without multimorbidity, 2 (0.9%) were considered “complex”.

Validity

With regard to concurrent validity, there were moderate correlations between the total INTERMED score and its psychological domain with HADS (ranging from 0.46 to 0.59, p < 0.05), and between the INTERMED social domain and the total HADS score (0.41, p < 0.05). There was a moderate inverse correlation between the INTERMED social domain and MOS-SSS (-0.44, p < 0.05), the total INTERMED score and its psychological domain with WHOQOL-BRIEF (-0.44, p < 0.05), as well as the INTERMED biological domain with the physical domain of WHOQOL (-0.58, p < 0.05) and the total WHOQOL score (-0.59, p < 0,05) (Table 2).

Table 2

Spearman’s correlation coefficients between INTERMED and other tools.

	Biological		Psychological		Social		Health system		INTERMED total score
	rho	p	rho	p	rho	p	rho	p	rho	p
HADS:
Total	0.31	<0.01	0.59 a	<0.01	0.41	<0.01	0.26	<0.01	0.56	<0.01
Anxiety	0.34	<0.01	0.57	<0.01	0.38	<0.01	0.22	<0.01	0.55	<0.01
Depression	0.21	<0.01	0.50	<0.01	0.36	<0.01	0.26	<0.01	0.46	<0.01
MOS—SSS	-0.12	<0.05	-0.35	<0.01	-0.44	<0.01	-0.17	<0.01	-0.38	<0.01
CCI	0.09	0.08	-0.02	0.18	-0.01	0.40	-0.12	0.20	0.08	0.75
WHOQOL-BREF:
Total	-0.44	<0.01	-0.57	<0.01	-0.36	<0.01	-0.26	<0.01	-0.59	<0.01
Physical	-0.58	<0.01	-0.52	<0.01	-0.32	<0.01	-0.28	<0.01	-0.63	<0.01
Psychological	-0.23	<0.01	-0.44	<0.01	-0.36	<0.01	-0.20	<0.01	-0.44	<0.01
Social	-0.20	<0.01	-0.41	<0.01	-0.24	<0.01	-0.21	<0.01	-0.42	<0.01

HADS = Hospital Anxiety and Depression Scale; MOS-SSS = Medical Outcomes Study–Social Support Survey; CCI = Charlson Comorbidity Index; WHOQOL-BREF = World Health Organization Quality of Life–BREF.

a Values in bold represent moderate Spearman correlation.

The omega coefficient was 0.834, suggesting good internal consistency [39-41]. After deleting each of the 20 items from INTERMED, the omega coefficient values ranged from 0.817 to 0.836. Four items showed no decrease in the original omega coefficient value when deleted: “treatment experience” (0.835), “resistance to treatment,” “access to care” (both 0.836), and “job and leisure problems” (0.837).

To verify whether the “health system” use (PHC, other health care services (than PHC), and use of any medication) can predict participants’ levels of complexity based on the INTERMED criterion, we used hierarchical multiple linear regression. The analysis resulted in model 1 [F(3,23) = 14.1, p < 0.01, R2 = 0.16, AIC = 1552] and, after controlling the parameters of age and sex, in model 2 [F(8,22) = 10.2, p < 0.01, R2 = 0.24, AIC = 1529]. The results of the Shapiro-Wilk test were 0.98 (p < 0.01) and 0.98 (p < 0.01) respectively, indicating that the assumptions of normality were not met and, therefore, models 1 and 2 were inadequate. Next, through graphical analysis, we identified influential outliers. These were patients who, regardless of the complexity level, either used the health system sporadically (only in A&E) or used the system in an exaggerated way (with excessive consultations in A&E, plus 10 consultations in PHC, and more than 10 medications). After analyzing and excluding these influential outliers, Model 3 was run [F(8,21) = 16.8, p < 0.05, R2 = 0.39, AIC = 1380], for which the results of the Shapiro-Wilk test were 0.99, p = 0.77, indicating that the normality assumption was met and model 3 was adequate. Overall, after adjusting for confounders and excluding the 12 influential outliers (model 3), we found that the use of PHC, other health care services (than PHC), and the use of any medication were predictors of complexity according to the INTERMED criteria. Table 3 presents the model development.

Table 3

Standard multiple linear regression models for INTERMED and health care use.

Model 1: Complexity level (r2 = 0.16)
Predictor	Coefficient β	IC (95%)	t	P
Intercept a	7.50	5.25; 9.75	6.57	<0.01
Use of any medication	4.24 b	2.35; 6.13	4.42	<0.01
Use of PHC	2.72	0.41; 5.03	2.32	0.02
other health care services (than PHC)	2.74	0.92; 4.56	2.97	0.01
Model 2 after adjustment of confounders: Complexity level (r2 = 0.24)
Predictor	Coefficient β	IC (95%)	t	P
Intercept a	9.68	6.34; 13.01	5.72	<0.01
Use of any medication	3.57	1.61; 5.53	3.58	<0.01
Use of PHC	1.55	-0.67; 3.78	1.38	0.17
other health care services (than PHC)	2.79	1.06; 4.52	3.18	0.01
Age groups:
18–30 vs. >60	-0.44	-3.50; 2.61	-0.28	0.77
31–40 vs. >60	0.37	-2.32; 3.06	0.27	0.78
41–50 vs. >60	2.64	-0.13; 5.41	1.88	0.06
51–60 vs. >60	2.98	0.20; 5.76	2.11	<0.05
Sex
Male vs. Female	-4.48	-6.24; -2.73	5.03	<0.01
Model 3 after adjustment of confounders and withdrawal of residual outliers: Complexity level (r2 = 0.39)
Predictor	Coefficient β	IC (95%)	t	P
Intercept a	3.08	0.32; 5.85	2.20	<0.05
Use of any medication	3.65	1.92; 5.38	4.16	<0.01
Use of PHC	2.12	0.13; 4.11	2.10	<0.05
other health care services (than PHC)	3.05	1.54; 4.57	3.97	<0.01
Age groups:
18–30 vs. >60	-0.40	-3.08; 2.27	-0.30	0.77
31–40 vs. >60	0.95	-1.84; 2.86	0.43	0.67
41–50 vs. >60	0.15	-0.30; 4.60	1.73	0.08
51–60 vs. >60	3.49	0.49; 5.35	2.37	<0.05
Sex
Male vs. Female	5.74	4.20; 7.28	7.35	<0.01

a Represents reference level.

b Values in bold represent the highest predictor value.

Feasibility

All patients reported satisfaction with the questions asked, the answers, and the application time. The range of the application time was 3–32 minutes; the average application time was 8.15 minutes; the median was 7 minutes and up to 14 and 18.15 minutes for 90% and 95% of the patients respectively. The perceived relevance of the domains was as follows: biological (n = 230, 100%), psychological (n = 227, 98.7%), health system (n = 221, 96.1%), and social (n = 215, 93.5%). The health records analysis showed that the psychological, social, and health system domains had incomplete data (S2 Table). Only the INTERMED biological domain had more than 50% of the items already described in the health records.

Discussion

We explored the validity and feasibility of the INTERMED adult interview tool applied in PHC attendees in Brazil, using an adequate sample size, multiple performance metrics, and exploring patients’ opinions. To the best of our knowledge, this is the second in general, and the first as thorough and fully reported assessment of INTERMED in a PHC population. We found moderate Spearman’s correlation coefficients between the four INTERMED domains and other instruments based on a comprehensive approach to health status (HADS, MOS-SSS, and WHOQOL-BREF). Similar results have been reported for INTERMED studies in specialized services (ranging from 0.55 to 0.74) [11, 12]. The correlation with the CCI, which only quantifies health conditions [31], was found to be weak, which reflects the completeness of the INTERMED tool. The good results regarding INTERMED internal consistency are similar to those found in other studies [10]. Using multiple linear regression, we found an association between higher INTERMED scores and higher use of PHC, other health services (than PHC), and use of any medication. These results suggest that the INTERMED tool is valid for use in a PHC setting [33, 42].

Previous research has suggested that health records with biopsychosocial data facilitate evidence-based care planning development, with increased communication between patients and health systems [43-45]. We found that the data already existing in the health records were focused almost exclusively on the biological aspect, which is considered a widely reported problem and an area for improvement [46, 47]. These results mean that INTERMED could help assess, organize, and coordinate all relevant biopsychosocial aspects of the health service network’s information-sharing process [14].

The maximum of 14 minutes needed to complete the interview for 208 (90%) of the participants was shorter than the recommended duration of a single outpatient appointment in Brazil [48], and the median of 7 minutes is compatible with the average PHC appointment duration in 39 countries [49]. The median time being relatively shorter is likely to be related to the fact that 127 (55%) of the interviewed patients were classified as “non-complex” and did not require further clarification after applying INTERMED [49]. This is different from what was found in specialized services, in which the INTERMED application time ranged from 20 to 40 minutes, with a smaller percentage of patients being considered “non-complex” [27]. Another previous study, in a PHC context, also supports the position that “non-complex” consultations are significantly shorter than “complex” consultations [50]. These results regarding the application time were obtained by applicators with the theoretical and practical training proposed by the authors of the INTERMED [25]. The application time results, together with patient perceived feasibility, suggest that INTERMED is a promising candidate for practical use in the Brazilian PHC context [48].

Study limitations

While we utilized the cutoff point applied in all previous studies, the clinical significance of these cutoff points in PHC is unclear. The cutoff in the context of PHC could be established through the application of ROC analysis, by measuring the sensitivity and specificity of different cutoff scores and their relationship to variables found in a larger sample of PHC patients.

Research implications

Given the appropriate psychometric properties of INTERMED in the referred sample, in future research, the authors plan to: 1) evaluate its implementation in routine PHC practice to assist person-centered care planning for better health outcomes; 2) evaluate its implementation in primary and specialized services within the same health service network to enable integrated care for better health and service outcomes; 3) develop digital versions of INTERMED to enable objectives 1 and 2; 4) assess whether the future use of health services can be predicted from an INTERMED score; and 5) evaluate INTERMED psychometric properties in other PHC populations and contexts.

Conclusions

This study showed that INTERMED has adequate psychometric properties to help PHC teams assess the biopsychosocial complexity of health needs. INTERMED could assist PHC professionals and teams in defining patient complexity profiles and developing healthcare planning. The results indicate the need for further studies to assess the potential of INTERMED to enable the delivery of integrated and person-centered care.

Profiles of the 230 PHC patients regarding INTERMED items and their clinical anchor points.

(DOCX)

Click here for additional data file.

Completeness of the INTERMED’s domains in the health records.

(DOCX)

Click here for additional data file.

8 Feb 2021

PONE-D-20-31324

Health Complexity Assessment in Primary Care: a validity and feasibility study of the INTERMED tool.

PLOS ONE

Dear Dr. Oliveira,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please ensure that your decision is justified on PLOS ONE’s publication criteria and not, for example, on novelty or perceived impact.

For Lab, Study and Registered Report Protocols: These article types are not expected to include results but may include pilot data.

==============================

Please submit your revised manuscript by Mar 25 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Stefan Hoefer

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Based on the reviewers comments, I would like you to focus in particular on presenting your study more concisely, and the statistical analysis and results must be presented more clearly. Please try to address all the comments of the reviewers very carefully.

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1) Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2) In your Methods section, please provide a justification for the sample size used in your study, including any relevant power calculations (if applicable).

3) We suggest you thoroughly copyedit your manuscript for language usage, spelling, and grammar. If you do not know anyone who can help you do this, you may wish to consider employing a professional scientific editing service.

Whilst you may use any professional scientific editing service of your choice, PLOS has partnered with both American Journal Experts (AJE) and Editage to provide discounted services to PLOS authors. Both organizations have experience helping authors meet PLOS guidelines and can provide language editing, translation, manuscript formatting, and figure formatting to ensure your manuscript meets our submission guidelines. To take advantage of our partnership with AJE, visit the AJE website (http://learn.aje.com/plos/) for a 15% discount off AJE services. To take advantage of our partnership with Editage, visit the Editage website (www.editage.com) and enter referral code PLOSEDIT for a 15% discount off Editage services.  If the PLOS editorial team finds any language issues in text that either AJE or Editage has edited, the service provider will re-edit the text for free.

Upon resubmission, please provide the following:

The name of the colleague or the details of the professional service that edited your manuscript

A copy of your manuscript showing your changes by either highlighting them or using track changes (uploaded as a *supporting information* file)

A clean copy of the edited manuscript (uploaded as the new *manuscript* file)

4)  We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

5)  Thank you for stating the following in the Acknowledgments/Funding Section of your manuscript:

[This study was financed in part by the “Coordination of Superior Level Staff Improvement

– Brazil (CAPES) – Finance Code 001”. It was also funded by the “Foundation for Support

to Teaching, Research and Assistance at Clinics Hospital of Ribeirão Preto Medical School

of University of São Paulo – Brazil (FAEPA)”. A discussion regarding it occurred during an

international meeting funded by the “Global Challenge Research Fund (GCRF) - Internal

Pump Priming Fund Round 5 of the University of Aberdeen”. The funding sources had no

role in the design, conduct, and reporting of the study.]

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

[CAO

This study was financed in part by the “Coordination of Superior Level Staff

Improvement – Brazil (CAPES) – Finance Code 001”. https://www.capes.gov.br/.

It was also funded by the “Foundation for Support to Teaching, Research and

Assistance at Clinics Hospital of Ribeirão Preto Medical School of University of São

Paulo – Brazil (FAEPA)” - https://www.faepa.br/

The funders had no role in study design, data collection and analysis, decision to

publish, or preparation of the manuscript.]

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This study aimed to evaluate the feasibility and validity of the application of the INTERMED interview in primary health care.

The INTERMED is an interview to assess the bio-psycho-social health care needs of patients. An advantage of the INTERMED is that many variables assessed in the interview are directly related to health care decisions. Using a cut-off 20/21 one can identify complex patients in need of integrated care. In general practice – with a high time pressure - the application of the INTERMED before a consultation could be of high importance to detect complexity or bio-psycho-social problem areas that are not detected by the standard consultation. The present study is therefore important and useful. In addition, with n=230 interviewed patients in primary health care the sample appears large enough to evaluate feasibility and validity.

However, I have several concerns regarding the manuscript (abstract, introduction, methods etc.). I therefore recommend a major revision.

In addition, the manuscript has to be revised by a native speaker.

More detailed comments:

Abstract:

• The abstract is hard to understand due to language problems

• The INTERMED has to be shortly explained – was it the interview or the questionnaire? What does it measure?

• The other assessment instruments have to be listed (methods).

• How is “health complexity” defined?

• Please shorten the description of the use of specific correlation coefficients in the abstract

• “Spearman’s correlations located between 0.44 – 0.65”. Which correlations are reported? The INTERMED total score correlated with which variables to which amount? This must be described exactly and clearly. The same holds for the other correlations (or just name a few variables and their correlations – but use exact descriptions).

• 7 minutes for an INTERMED interview is a very short time. How did this happen? (To be explained in the results section or discussion)

Introduction:

• Please ask a native speaker to revise the manuscript

• Please insert line numbers in the revision

• “identifying the epidemiology of patient complexity” – what is meant by this? The reference cited (9) does not refer to an epidemiological study.

• Please reduce the number of references considerably

• “how to effect the identification of complex patients” – I do not understand this sentence.

• The introduction must be re-written. It is not clear what the study wants to show. Perhaps the authors want to say that complexity is frequent in general practices but not so easy to detect? And that the INTERMED interview could be a useful assessment instrument for a GP. However, to date, only one study evaluated the usefulness and validity of the INTERMED interview in the frame of GPs. And, therefore, the present study aims to …

Methods:

• Was the (already recruited) sample divided into groups or the sampling based on various age groups?

• Instruments: The description of the INTERMED is hard to understand. Perhaps the authors could insert a Figure showing the INTERMED grid. The INTERMED interview assesses four domains: biological, psychological, social, and health care use. In each domain, five variables are rated on a scale ranging from 0-3, resulting in 20 scores. Each score ranges from zero evidence for or health service need (0), to a clear and serious disturbance or health service needs (3).

• There are 16 lead questions for the interview. The remaining 4 items are prognostic items – that is, the interviewer gives a prognostic score (without a lead question).

• Why was Spearman’s correlation coefficient used? Why not Pearson for the correlation between INTERMED total score and HADS score, for instance?

• What is meant by Pearson X2? Is it a chi-squared test? What is compared here? A chi-squared test compares groups with dichotomous outcomes. Were Pearson correlation coefficients calculated? Please clarify. The distribution of health care use is probably skewed. Here, a Pearson correlation is probably not the best choice.

Results:

• How many patients refused to participate? What are the differences between non-responders and responders? What were the reasons to refuse?

• Validity: The correlations ranged from 0.44 to 0.65? In Table 3, there are smaller correlations presented.

• I do not understand Table 4. In Table 4, chi-squared statistics and p-values are presented. What was calculated here? Which groups were compared? Were Pearson correlations calculated? Then the correlation coefficients should be shown, together with a p-value.

• Please shorten the paragraph regarding “feasibility”.

Discussion:

• The discussion must be shortened. Some parts are hard to understand due to language

Reviewer #2: !. Is the Pearson chi square test appropriate when one variable is a parametric test score (i.e., scores on the INTERMED current status)?

2. Why was Spearman rho used as the correlation coefficient? Were the data in question rank ordered?

3. Is coefficient alpha appropriate for an instrument that is not tau equivalent? That is, can it be demonstrated that the INTERMED is an unidimensional instrument appropriate for the alpha coefficient?

4. Why was the term "construct validity" used on page 9?

5. What is an "academic clinician" (page 9)?

6. On page 7 is the following statement: "we evaluated the concurrent validity of each of its four domains with other well-validated specific instruments for these domains." The next sentence starts listing the other instruments, the first of which is the Socio-demographic Questionnaire that was "developed for the study." Is the assertion that this instrument was "well-validated?" Or, the adaptation resulting in the Questionnaire for Evaluation of Health Services Use? These scales were not specifically used in the computation of the validity coefficients, but nevertheless there does seem to be a gap between the Term "well-validated" and documentation thereof, including documentation of the validity for criterion variables HADS, MOS-SSS, CCI, and WHOQOL-BREF.

7. Was consideration given to the control of type 1 error in the series chi square tests, and Spearman correlations? If not, why not?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

25 May 2021

We are grateful for all the comments. Our point by point response to each comment can be found in a separate document entitled 'response to reviewers

Submitted filename: Response to reviewers.docx

Click here for additional data file.

26 Jul 2021

PONE-D-20-31324R1

Health Complexity Assessment in Primary Care: a validity and feasibility study of the INTERMED tool.

PLOS ONE

Dear Dr. Oliveira,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 09 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Stefan Hoefer

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Both reviewers belief that your manuscript is of importance and merits publication in PLOS ONE. I kindly ask you to in particular focus on the comments of reviewer 1 and try to thoroughly address the raised statistical issues as well as the language concerns.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I still believe that the study is worth publishing. The manuscript hast been revised and has improved in quality and formal standards. However, it still lacks scientific precision and good linguistic expression. I would therefore recommend a comprehensive second revision based on the following comments:

Abstract:

The manuscript has been revised and the language has improved. However, I still have the impression that the manuscript was not edited by a native speaker (e.g. expressions such as “18.3 % of the patients were classed as complex” in the abstract or “This study aimed to assess… to assess…). Also, for instance, line 232: “Patients were excluded if they were determined to be unable to consent. In my opinion, the language of the manuscript must be improved.

• In the Methods Section of the Abstract the cut-off for complexity must be mentioned.

• The Methods Section of the Abstract must be re-written. Example for a better formulation: Biopsychosocial health care needs were assessed by using the INTERMED interview… Participants with a total INTERMED score >20 were classified as complex / having complex health care needs. Quality of life was measured by …

• The Results Section of the Abstract should be re-phrased. Example: Results showed a high correlation between INTERMED total score (not “whole” score) and quality of life (r=0.59).

• The above formulations are just examples. The next paragraphs also lack scientific precision. I would really ask the authors to scan other scientific papers and use appropriate formulations. The whole study decreases in value if the expressions are not precise and adequate.

• T- and beta-values should be rounded to two decimal points (instead of three).

• the sentence “In a multiple linear regression analysis, the use of PHC (β = 2.121, t = 2.100, p=0.037), services other than PHC (β = 3.052, t = 3.970, p<0.001) and medication (β = 3.654, t = 4.164, p<0.001) predicted the INTERMED score.” is hard to understand. Perhaps a change to “ the use of PHC (), any other health care service (than PHC) () and the use of any medication () were associated with a higher INTERMED score.

• I do not understand the variable “PHC use”. All the patients were recruited in a primary health care setting. So, everybody score a 1 on this variable?

Introduction:

• The introduction must be revised by a good native speaker

Methods:

• Sampling technique: I do not understand the “rule of thumb” – five observations per variable. The validity of the INTERMED was measured by using the total score (and not by calculating correlations with each item).

• Sampling technique: The authors state that previous NTERMED studies used different sample sizes. These studies varied widely in object and settings. So, why should there be a common sample size calculation for all these studies? An INTERMED study does not need specific assumptions for a sample size calculation (compared to other validation studies). For a validation study one can, for instance, calculate the appropriate sample size based on the assumed correlation between the main instrument (INTERMED) with the gold standard with which I compare the instrument. However, very few validation studies do a power calculation prior to the beginning (see for instance https://hqlo.biomedcentral.com/articles/10.1186/s12955-014-0176-2

• One could also just describe that the sample was collected consecutively over a specific time period – if this was the case – resulting in n=230 patients.

• Line 310: Please change “The instruments were administered between November 2018 to June 2019.” Into “The study was conducted between November 2018 to June 2019” or “Patients were included between November 2018 to June 2019” – or something similar. Please adjust other formulations to a more scientific language.

• Line 293: The patients were recruited in a PHC setting. So, each patient scored a 1 on the variable “PHC use”?

• Line 343: adjusting for sex and gender – or controlling for…

• Lines 344-346: This is not to understand. Outliers are removed beforehand (when data appears to be absolutely implausible). Also, the language is difficult to understand. What is meant by “influent”? How many data was withdrawn from the sample? I would recommend to skip step 2 and 3.

• Line 351: Why was a factor analysis conducted? I would recommend to omit this procedure. It is not part of the study aim. Also, in my opinion, the INTERMED is not uni-dimensional. It consists of four different domains.

• Descriptive statistics should be described first. Followed by the more complex statistics.

Results:

• Table 2 should be omitted or changed in a Supplemental Table

• Please round correlation and p-values to two decimal points.

• Line 497: What means “after controlling for type 1 error?”

• Lines 503-505: The INTERMED is not assumed to be uni-dimensional. Somatic issues (for instance) are not on the same dimension as psychological issues. Perhaps the factor analysis should be omitted. It is also not clear, which factor analysis has been conducted.

• The outliers should not be excluded. These are the patients who use the system excessively (or not at all). There is no reason to exclude them. In contrast, the patients with the highest health care use are probably those with the highest INTERMED scores.

• Table 5 must be omitted.

Discussion:

• line 627 ff: a median does not range. The range of the time was 3-32, the median was 7.

• Lines 637 ff: The cut-off could be investigated with the present study data. One could run an ROC analysis with different cut-offs for complexity and look at the sensitivity and specificity of the various cut-offs regarding a specific outcome (this would need a categorization of the patients according to an external criterion). So, the authors do not have to do this. But a clustering method is not the method to investigate the appropriateness of a cut-off.

Reviewer #2: My main concerns were related to statistical and psychometric issues. The authors provided an evidence foundation for the use of the Spearman correlation and provided the omega coefficient for non-tau equivalence. These and all other concerns were adequately remedied.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

27 Sep 2021

We are grateful for all the comments. Our point by point response to each comment

can be found in a separate document entitled 'response to reviewers

Submitted filename: renamed_2f54d.docx

Click here for additional data file.

30 Dec 2021

6 Jan 2022

Thank you for the opportunity to revise our manuscript. We would also like to thank the reviewer 1 for their comment. It was very helpful and insightful. Below we describe how it has been addressed in the revised version of the manuscript.

Submitted filename: renamed_26913.docx

Click here for additional data file.

26 Jan 2022

Health Complexity Assessment in Primary Care: a validity and feasibility study of the INTERMED tool.

PONE-D-20-31324R3

Dear Dr. Oliveira,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Chung-Ying Lin

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

The authors have satisfactorily addressed the last comment made by the previous reviewer. I believe that the present manuscript achieves the standard of publication.

Reviewers' comments:

7 Feb 2022

PONE-D-20-31324R3

Health Complexity Assessment in Primary Care: a validity and feasibility study of the INTERMED tool

Dear Dr. Oliveira:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Chung-Ying Lin

Academic Editor

PLOS ONE