The Application of Formal Axiology to Medical Education through the Hartman Value Profile: A prospective cohort study.

INTRODUCTION: Many criteria such as USMLE scores, applicant resumes, Dean's letters, recommendation letters, personal discussions, interview scores and medical school transcripts can be used to predict the success of a medical trainee in the USA. This information is either relatively objective, or subjective. It would be valuable if we had some objective measures that might predict a successful resident performance early in the process or on the other side to allow remediation or redirection. Actual performance of a resident or fellow is based upon his or her ability to execute sound judgments within the complex healthcare setting. The Hartman Value Profile (HVP) evaluates the structure and the dynamics of an individual value system. This study has the primary goal of determining whether specific indices on the HVP correlate with the management's evaluation of the residents established by the Department of Anesthesiology at Yale University.
METHODS: The protocol developed uses univariate correlations between residents' HVP subscales and their performance scores, which will be determined with the Pearson correlation coefficient or Spearman rank coefficient as appropriate. Demographic and clinical variables will be reported descriptively. A two-sided alpha value of 0.05 will be used for identifying statistically significant findings.
CONCLUSION: The potential benefits are that obtaining specific indices on the HVP would enable management to better engage and work with residents. Experience gained from incorporating the HVP into the residency selection process suggests that it may add objectivity in predicting resident performance during training. Given the potential impact, it could be implemented as an adjuvant tool to the traditional evaluation process. Copyright: © Shiraz University of Medical Sciences.

Introduction

Many criteria can be used currently to select and predict the success of a medical trainee in the USA; these include information from the United States Medical Licensing Exam (USMLE) scores, applicant resumes, Dean’s letters, letters of recommendation, personal statements, interview scores and medical school transcripts. The information provided is either relatively objective, as for USMLE scores, Alpha Omega Alpha Honor Society membership, and class rank, or subjective, as for letters of recommendation, medical school reputation, and clerkship performance. Programs assign varying significance to each component, and then rank applicants based on the overall intuitions generated by this relatively subjective and unstandardized process (1-6). It would be valuable to have more objective measures that might predict a successful resident performance early in the process, or to allow remediation or redirection.

The limitation of this idea is understandable. True performance of a resident is based on the ability to accomplish sound judgments within the intricate healthcare environment. This ability is dependent on character qualities such as intelligence, integrity, adaptability, maturity, leadership, and work ethic (1,2,4,6,7). Almost no publication directly evaluates these qualities or their effects on performance (1-4,6,7).

Dr Robert S. Hartman, a German logician and philosopher, dedicated his work to the idea of organizing goodness. He thought that people’s good judgment brings outcomes at the personal and corporate level. Hence, there is not a decision made in life or business that doesn’t need good judgment (8). The procedure that allows for the estimation of good judgment is the Hartman Value Profile (HVP).

Robert S. Hartman elaborated the Profile in the early 1960’s. The HVP is the most mathematically, scientific and logically based evaluation instrument ever designed for this intention. Where most evaluation instruments estimate an individual’s personality, behavioral style or attitudes, the Hartman offers understanding into an individual’s judgment ability (9).

In a Healthcare setting, prospective candidates should be assessed first on their competence. Competencies are perceived as verified skill sets that are obtained through education, experience, and training. Once it has been decided that candidates have the competencies required to be effective in the position, based on Hartman’s work, the next stage is to assess the candidate’s ability to apply good judgment. Hartman’s formula:

Competency + good judgment= excellence and quality

For Hartman, the formula for excellence in work, including productivity, high quality, and positive morale was:

Work = (Competent skill sets+ good information+ good processes) * good judgment.

As a multiplier effect, Good judgment has a meaningful effect on excellence in the workplace (9).

Hartman’s investigation and anecdotal documentation during 35 years suggest that most success is the outcome of good judgment. The goal of the Profile is to estimate the strength of these qualities so that their utility and growth can be more critically addressed (9).

The HVP is based on formal axiology, a field of psychology that evaluates how individuals value themselves and the nearby environment. It is not an IQ, rational intelligence profile, personality test, or emotional balance profile. Indeed, it demonstrates the constraints of each of these (1).

Instead, it evaluates the structure and the dynamics of an individual’s value system. Robert S. Hartman argued that value is assigned to various concepts or objects according to the following (10):

1. The value of its uniqueness (described as “Intrinsic Value Dimension”)

2. The value of its function or role (or “Extrinsic Value Dimension”)

3. The value of its meaning or purpose (or “Systemic Value Dimension”)

The Intrinsic Value Dimension evaluates the capacity for relational judgment, which is evidenced plainly in good people skills. The Extrinsic Value Dimension evaluates the ability to be brilliant in tasks, projects, processes, and the basic realization of skill competencies. The Systemic Value Dimension evaluates the aptitude for mastery in long-term planning, strategic visioning, structural integrations, implications, and consequences (10,11).

Furthermore, each of these dimensions can be valued or de-valued intrinsically, extrinsically, or systematically. By merging the 6 variations of value judgments for intrinsic, extrinsic, and systemic dimensions, a total of 18 value judgments can be made (10,11).

Axiology demonstrates that these 18 alternatives are not assigned arbitrarily; the relative value that an individual designates to an object, a choice, or a circumstance is based on that individual’s conceptual hierarchy. As this procedure is the essence for the practice of medicine, evaluation of this hierarchical value categorization would be a crucial component of evaluation of medical trainees. The HVP details this conceptual structure and therefore lends insight into how people view themselves, others, and the world around them (1).

Evaluative judgment is defined as the ability, when presented a problem or situation, to observe and understand the dynamics of the circumstance, to determine what actions will improve the situation, and ultimately take action to improve the condition (1).

Broadly, the HVP distinguishes three equitably significant kinds of Evaluative judgment. In an institution environment, all three are significant if there is to be a fair and exhaustive perspective on strategy and tactics. The highest performing and best decision-making groups collectively show all three kinds of judgment as depicted below.

Intrinsic Value Judgment concerns the ability for excellence in relationships and the ability for compassion, care and empathy.

Extrinsic Value Judgment concerns the ability for excellence in tasks, projects, and work processes.

Systemic Value Judgment concerns the ability for excellence in more abstract domains of work and life such as long-term planning, strategic anticipation, structural integrations, implications, and outcomes (9-11).

These results exist within a range from very weak to very strong. They are quantitative and include global scores and scores of the various components. Table 1 lists the components of self-side and work-side judgment. Table 2 features the balance indicators of work-side judgment and self-side judgment. The “self-side” addresses how subjects value themselves. The “work-side” tackles how they value their work (9).

Table 1

Measured components of self- and work-side judgment.

Personal/Self-Side Judgment	External/Work-Side Judgment
1. Understanding what is “important”	Problem solving ability.
2. Self-regard/self-care.	Ability to notice, insight, sensitivities.
3. Self-accepting vs self-criticizing.	Dealing with difficult situations, problem solving energy, innovation.
4. Effects of self-side stress.	Effects of work-side stress.
5. Assertive vs conflict avoidant.	Focus and concentration.
6. Moral clarity.	Directions followed with accuracy.
7. Problem solving style, self-side.	Problem solving style, work-side.
8. Acceptance of change/role identity.	Realisms vs idealism orientation.
9. Meaningfulness of work, self-identity.	General tolerance, acceptance of others.
10. Morale: value of work.	Compassion, empathy, actions of care.
11. Solving personal problems for self.	Trainability- the ability to understand work.
12. Solving practical problems for self.	Dependability, reliability, work ethic.
13. Basic organizational ability.	Understanding big picture implications.
14. Environmental conscientiousness.	Using big picture implications.
15. Overall strength of self-side judgment.	Overall strength of work-side judgment.

Table 2

Balance indicators of self- and work-side judgment

Self-Side Balance	Work-Side Balance
1. Self-esteem/self-confidence	Value of people, relations
2. Self-confidence/Role of Satisfaction	Value of work, tasks
3. Self-image/Motivation	Value of ideas, implications and consequences

Leon Pomeroy compared the axiological patterns of three USA populations who were given the HVP - doctors, college students and psychiatric outpatients. This showed a statistical recognition of unique axiological signatures defining each population. The psychiatric outpatients possessed a very unique axiological signature and were easily distinguished from doctors and students (12).

Incongruence between leaders and followers has been proposed to conduce to dispute and failure. Literature on leadership and followership fails to approach self-efficacy. It was formulated that leader-follower congruence would affect follower self-efficacy. Cooke et al. tested this hypothesis utilizing the HVP assessment to evaluate leader-follower congruence. They deduced that there was a strong positive association among leader-follower congruence and follower self-efficacy (13).

There is more than 30 years of research and validity studies associated with the HVP. In the scoring process, there literally are 12.8 quadrillion possible combinations of response derived by the calculus (9).

The outcomes of the HVP are acquired from logical mathematical norms with quantitative values for each of the constituents in Tables 1 and 2, and they are not founded on the values of any particular populace or group. The outcomes have no bias with respect to sex, age, race, or any other socio-cultural categorization and are highly reliable and reproducible (9).

The HVP is an effective, proven approach for the prediction of performance, and it is utilized extensively in private industry for employee selection and development. Scant literature exists that has used the HVP for trainee evaluation. Its predictive value specifically in evaluating successful candidates at medical training programs, has not been tested.

This study protocol has the primary goal of determining whether specific indices on the HVP correlate with the management’s evaluation of the residents, established by the Department of Anesthesiology at Yale University.

Methods

Personalized HVPs are generated from the way in which subjects rank 2 lists of 18 phrases. The first list is ranked from “best to worst” and the second from “most agree to least agree” (Figure 1) (9). Before completing the forms, applicants are asked to read the instructions next to the phrases. The profiles reflect individual predilection; accordingly, no right or wrong answers exist. The profiles take approximately 15 to 25 minutes to complete.

Figure1

The hartman value profile

Figure 1. “Part I: Phrases”-Value judgment ranking list as the individual relates primarily to the world of work (or the world that is “external”). “Part II: Quotations”-Value judgment ranking lists as it pertains to the individual’s judgments concerning one’s self (or to the “internal” self). (The copyright for this profiling instrument is owned by Robert S. Hartman Institute. It may be used for commercial purposes without paying royalties to the Institute as long as copyright credit is given to the Institute).

An independent consulting group, Ruhmann Associates, will implement and interpret the profiles customizing the HVP to the needs of our institution and program for specific desired outcomes.

Because the results of the HVP include global scores and a myriad of component scores, particular components of the HVP can be emphasized to target candidates who best fit a program’s goals (1).

We propose to have our anesthesiology residents take the HVP. Faculty will separately evaluate the residents and place them in tertiles (upper, middle or lower) depending on their success as residents. We will then data mine the HVP to see if there are particular characteristics that are associated with the successful resident. This may then potentially be used to predict the success of a resident.

There will be a wealth and breadth of information available from the completed HVPs. Although we might ‘mine data’ throughout the project to look for trends and critical indexes, there will be some basic characteristics that we will look out for. These include:

1. Noticing/differentiation: Measures the likelihood of the resident to notice subtle changes in their patient which may indicate that treatment is not working in some critical manner.

2. Following directions: Measures the likelihood of the resident to be careful in following directions regarding elements of treatment and care.

3. Conceptual systemic: Measures the likelihood of the resident to understand the care plan conceptually and explain that to the patient.

4. Actual systemic/integration: Measures the likelihood to integrate into real choices, decisions and actions which have been conceptualized.

5. Problem-solving ability/integration: Measures the likelihood to be a clear and decisive problem solver, solution-finder and strong decision-maker.

6. Problem-solving energy/integration: Measures the likelihood to recognize, organize and mobilize personal resources to impact the situation.

7. Attitude in the workplace: Measures the likelihood to be resilient in the face of negative circumstances.

8. Role identity: Measures the likelihood to deal with change.

9. What is important: Measures the likelihood to have a strong sense of what is important as opposed to that which is peripheral.

10. Self-side Attitude: Measures the ability to be grounded in one’s self and act as a personal ‘foundation’ during times of challenge.

Up to 57 Anesthesiology residents training at Yale New Haven Hospital will be asked to take part in this study, which will take place at Yale University. All residents will be asked to volunteer and, if they agree to take part in the study, they will sign informed consent forms.

The current first, second and third year residents will be asked to complete the HVP online. The independent consulting group will provide the Department of Anesthesiology with a “Mindset” that will be presented to the residents. The Department of Anesthesiology will be provided with login information for the resident to use as they complete the tool. The Department of Anesthesiology will provide information about year of residency and management’s evaluation of the individual (top, middle or lower third). This criteria is established by the Department of Anesthesiology and is consistent with all residents. The top third represents the strongest residents.

First year residents will complete the HVP after they have been on-board for 6 months, or at such time as they receive the first review from management. Management will be requested to rank each resident on a top, middle, and lower third basis. The data will be analyzed to identify critical indices.

The clinical director will try to relieve residents from the operating time to complete the test. Residents may be asked to complete it after clinical hours, but that will be completely voluntary. It is our goal that all testing will be completed by 5:15 PM. Under no circumstances will any testing be performed after 7:15 PM.

1. Recruitment procedures

Residents will be notified of the study in three ways:

1) A letter will be posted in the anesthesia departments.

2) An e-mail will be sent to all residents and

3) Investigators will meet with residents to discuss the study with them.

All three methods will be used to notify residents, because some of them will be on vacation or post-call. Telephone correspondence will not be used to screen potential subjects for eligibility prior to the potential subject coming to the research office. If residents are rotating at different hospitals, phone conversations may be used to answer any questions residents may have.

2. Consent

As the research involves the disclosure of protected information, the In-Training Exam scores, separate subject authorization is required under the HIPAA Privacy Rule. The Compound Consent and Authorization form will be provided.

The investigators will meet with the residents to explain the study, and will provide them with the informed consent. All three methods of contacting the residents will clearly state that enrollment in the study is purely voluntary. Fax signatures will be accepted for consent.

3. Data confidentiality and security

All information and scores will be de-identified, and strict measures will be taken to maintain confidentiality. No anesthesia program directors or chairperson will be permitted access to any private data or the HVP results. The independent consulting group will be responsible for anonymizing the HVP profiles data. While the individual results will not be available to the program leadership, whether the resident has taken the exam or not will be available by necessity. This could potentially cause bias for or against the resident. However, HVP scores and all their profile information will be confidential and could only become known to a training program if a resident decides to notify the program. Residents will have the option to decide whether to receive their profile results or not. Adverse events are not anticipated. Finally, the data will be evaluated and reported in an aggregate fashion.

4. Statistical considerations

Power analysis was performed using the Power Analysis and Sample Size software (PASS, 2005). With 57 subjects for the current pilot study, we will have 88% power to detect a correlation of 0.4 between a HVP subscale with a performance score using a two-sided hypothesis test with a significance level of 0.05.

Demographic and clinical variables will be reported descriptively. Descriptive statistics will include N, mean, median, standard deviation, minimum and maximum values. For categorical (nominal) variables, the number and percentage of subjects will be presented.

Univariate correlations between residents’ HVP subscales and their performance scores will be quantitated by Pearson correlation coefficient or Spearman rank coefficient, as appropriate.

Multivariate canonical correlation analysis will be carried out, which provides a more powerful means to detect the associations between residents’ HVP subscales and performance scores than univariate analysis. The probable duration of project is 2 years.

Two-sided alpha value of 0.05 will be used to identify statistically significant findings. All data will be analyzed using SAS 9.2 (Cary, NC).

5. Ethical statement

The research will be performed according to ethical principles and in compliance with all federal, state and local laws, as well as institutional regulations and policies regarding the protection of human subjects. Approval for this research study has been obtained prior to initiation of the study. This research protocol was approved following expedited review by the Human Subjects Committee of Yale University (IRB Protocol number: 1201009603). The project was found to be of minimal risk and met the approval requirements under University IRB policy and 45 CFR 46, as applicable.

Discussion

The literature regarding the resident selection procedures reports several limitations (3,4,6). A hazard exists of overlooking a candidate cause of incorrect inferences from an application, a letter of recommendation, or interview (1-7,14-17). The first weakness is the fact that most of the data are subjective, including letters of recommendation, school reputation, interview performance, appearance, and dean’s letter (1,4-6).

Researchers have tried to solve this weakness by elaborating composite scores (1,15). Besides, metrics cannot regularly and accurately be applied to subjective values without a variable degree of error.

Intuitions about the significance assigned to the components of a trainee application and interview procedure have been discussed broadly (1-4). Still, these studies are surveys.

Articles confirm that some traits are key to trainee success, and highlight the critical nature of character qualities (1-4,7,16). Successful trainees are defined by the traits of honesty, integrity, good work ethic, adaptability and the ability to organize one’s thought process. Understanding these traits would advance the selection procedure (1,3,4,6,16). Positive characters are especially desirable in candidates for medical training programs. Issues in confirming and assessing these qualities prevail, because independent, objective confirmation is intricate to attain in the absence of a value assessment tool (4,7,16,18).

Utilizing the HVP as an additional tool in trainee evaluation may yield significant clarification by lending insight into these qualities, and ultimately into the ability of a candidate to make sound judgments in the complex healthcare environment.

Research utilizing the HVP as an adjuvant tool to the selection procedure have had positive feedback. By presenting the tool at intervals in the training program, there is further evidence on the impact of the overall program’s effect on the evolving of trainee judgment, organizational skills, trustworthiness, confidence, or passion for one’s profession, gaining awareness into the overall training process (1).

Evidence confirms a positive correlation of these indicators to predicting future prosperity (1). Moreover, we would consider utilizing the HVP in assessing the training process, because it may offer objective data on overall progress.

While axiology demonstrates that value systems are stable across concepts, it also demonstrates that these systems are dynamic over time. Value systems are a product of interactions, events, and relationships, and additional experiences will refine one’s value structure. This conveys special importance in light of the core competencies rendered by the Accreditation Council for Graduate Medical Education (ACGME). According to the ACGME, these core competencies comprise the premise of medical education (19). However, these competencies lack confirmation verifying that they enhance the education procedure (20). The HVP can render specific awareness into these competencies and could examine how residency training tackles the core competencies (1).

Conventional tools utilized to choose and evaluate trainees have notable weaknesses. The HVP is a validated supplementary tool to predict future success in candidates in the business setting. Incorporating the HVP into the medical residency selection and evaluation process suggests that it may add objectivity and refinement in predicting resident performance (1). By obtaining specific indices on the HVP, this would enable management to better engage and work with the residents. It could be useful as well to predict the success of a resident, or equally, allow remediation and redirection during their training. Further evaluation is warranted with longer follow-up times.