Validating curricular competencies in innovation and entrepreneurship for biomedical research trainees: A modified Delphi approach.

INTRODUCTION: Biomedical researchers need skills in innovation and entrepreneurship (I&E) to efficiently translate scientific discoveries into products and services to be used to improve health.
METHODS: In 2016, the European Union identified and published 15 entrepreneurial competencies (EntreComp) for the general population. To validate the appropriateness of these competencies for I&E training for biomedical researchers and to identify program content, we conducted six modified Delphi panels of 45 experts (6-9 per panel). Participating experts had diverse experience, representing such fields as entrepreneurship, academic research, venture capital, and industry.
RESULTS: The experts agreed that all 15 EntreComp competencies were important for biomedical research trainees and no additional competencies were identified. In a two-round Delphi process, the experts identified 120 topics to be included in a training curriculum. They rated the importance of each topic using a 5-point scale from not at all important (1) to extremely important (5) for two student groups: entrepreneurs (those interested in starting their own ventures) and intrapreneurs (those wanting to be innovative and strategic within academia or industry). Consensus (mean importance score >4) was reached that 85 (71%) topics were of high importance for the curriculum. Four topics were identified by multiple panels for both student groups: resiliency, goal setting, team management, and communication skills.
CONCLUSIONS: I&E training for biomedical trainees should address all 15 EntreComp competencies, including "soft skills," and be flexible to accommodate the needs of trainees on different career trajectories. © The Association for Clinical and Translational Science 2019.

Introduction

Today’s biomedical research trainees often need skills in innovation and entrepreneurship (I&E). Those who choose an academic career need I&E skills to equip them for success in an increasingly impact-orientated funding environment. Those who choose a career in industry or government need I&E skills to help them to identify, assess, and capitalize on opportunities to improve human health. To keep pace with the changing training needs to support multiple career options for biomedical researchers, the National Institutes of Health (NIH) has encouraged breadth and flexibility in research training programs [1]. We received a grant from the NIH to develop a competency-based program to introduce biomedical research trainees to innovation and entrepreneurial thinking. To achieve this goal, we first sought to identify the core I&E competencies a successful biomedical researcher should possess and to identify topics to be included in a curriculum to introduce these skills to biomedical research trainees.

In a preliminary review of the literature, we found that few publications addressed competencies for I&E; and we found none that specifically addressed the needs of biomedical researchers. Our study team considered the literature and their combined experience in I&E, to develop an initial working draft of I&E competencies shown in Table 1. After the grant was awarded, we updated our literature review and discovered the work of another group, “EntreComp: The Entrepreneurship Competence Framework” [27]. The EntreComp Framework is unique in the literature in that the aim was to identify competencies that would generate an “entrepreneurial mind-set” for all citizens rather than training entrepreneurs. Fifteen high-level competencies were identified using a rigorous mixed-methods approach, including a literature review and in-depth case study analyses leading to the development of draft competencies and a conceptual model, with validation via expert and stakeholder consultation [27]. Entrepreneurship is defined broadly as “the capacity to act upon opportunities and ideas to create value for others. The value that is created can be social, cultural, or economic” [28]. The authors of the EntreComp Framework suggest that it can be used as a “starting point for the interpretation of the entrepreneurship competence” and that it should be adapted and tailored to address the needs of specific groups. As of March 2018, the Framework had been used in at least 74 training programs [28]. The EntreComp competencies are provided in Table 2.

Table 1.

Innovation and entrepreneurship (I&E) competencies for biomedical researchers: A working draft model

Type	Competencies
Cross-cutting personal competencies	Creativity and cognitive adaptability [2–5] Ethical problem-solving [6–8] Self-leadership and self-management [9–15] Social intelligence [16–18]
Team and project management competencies	Communication [3, 16] Leading creative and diverse teams [2, 19–21] Leading for innovation [2, 3, 22–25] Project management skills [3]
Business competencies	Business acumen and technical knowledge Negotiation Managing ethical and legal issues in business [26]

Table 2.

Delphi panel domain areas and EntreComp competencies

Panel	EntreComp competencies	Definitions a
1. Management (N = 8)	Planning and management	Set goals and define priorities and action plans, while adapting to unforeseen changes
	Financial and economic literacy	Estimate the cost of bringing an idea to fruition, and make financial plans and decisions that ensure the activities can continue over the long term
	Mobilizing resources	Obtain and manage the resources needed to turn ideas into action, and make the most out of limited resources
2. Vision and imagination (N = 9)	Spotting opportunities	Identify opportunities to address problems, challenges, and needs, and judge whether to act on them
	Vision	Imagine the future and visualize strategies to turn ideas into action
	Valuing Ideas	Recognize an idea’s potential to create value, and identify suitable ways to share and protect that idea
	Creativity	Develop multiple ideas and experiment with innovative approaches to solve existing and new challenges
3. Social skills (N = 7)	Self-awareness and self-efficacy	Reflect on personal strengths, weaknesses, needs, and aspirations, and believe in the ability to influence course of events
	Mobilizing others	Inspire relevant stakeholders through effective communication, persuasion, negotiation, and leadership to obtain the support needed to achieve outcomes
	Working with others	Network and cooperate with others to develop ideas and turn them into action
4. Psychological skills (N = 7)	Learning through experience	Reflect on experiences and learn from successes and failures
	Taking the initiative	Initiate activities and take on challenges to achieve goals
	Motivation and perseverance	Be determined and focused to turn ideas into action and reach long-term goals, even under adversity and failure
5. Ethical and decision-making skills (N = 8)	Coping with uncertainty, ambiguity, and risk	Make decisions despite incomplete information and unknown results
	Ethical and sustainable thinking	Assess the consequences of ideas and activities on communities, markets, society, and the environment, and reflect on the sustainability of social, cultural, and economic goals
6. EntreComp competencies (N = 7)	All competencies

Definitions are those presented to the expert panels and are adapted from the EntreComp Framework [27].

We decided to use the EntreComp Framework as a starting point to develop our I&E training program for biomedical research trainees for the following reasons: (1) the aim of the EntreComp Framework to generate an entrepreneurial mind-set was in line with our intent; and (2) all the competencies in our working draft were contained within the Framework. To validate the appropriateness of the EntreComp competencies for I&E training for biomedical research trainees in the USA and to identify course content for our program, we conducted six modified Delphi panels. Delphi panels are typically used to establish group consensus about priorities when many options exist, and modified Delphi processes have previously been used to develop curricula [29, 30–35], which was our purpose. We needed expert opinions upon which to build our curriculum.

Materials and Methods

Compliance

The project was presented to the Washington University in St. Louis (WU) Institutional Review Board who determined that it did not constitute research because it aimed at producing consensus among experts rather than generalizable knowledge from subjects. However, all participants freely agreed to participate in the process and provided permission to publish their names and biographical details, which they reviewed and approved (see Appendix A).

Participants

Panelists were selected using purposeful, non-probability sampling with the goal of recruiting a heterogeneous group of experts from the USA. Forty-five experts were identified (from academia, 25; venture capital, 11; industry, 9). Many had experience in biomedical entrepreneurship as shown in their biosketches (see Appendix A). All experts agreed to participate in a modified Delphi process over 2 months, with a total time commitment of less than 2 hours. A $100 Amazon gift card was offered as an honorarium.

Procedures

Recognizing the breadth of the I&E competencies, and to avoid overburdening panel members, we grouped the EntreComp Framework high-level competencies into five domain areas (Table 2). Six Delphi panels were planned, one for each domain area (Panels 1–5) and one to validate the 15 EntreComp competencies for use in our program (Panel 6).

The project team allocated the 45 experts to the six panels based on their area of expertise and entrepreneurial experience to ensure that each panel had a mix of both content experts and experienced entrepreneurs. Panel size ranged from 6 to 9 and is provided in Table 2. Between April 9, 2018 and July 18, 2018, each panel worked independently and simultaneously with panelists blinded to the identity of other participants. For each round of the Delphi process, individual panelists accessed the surveys hosted in Qualtrics through unique links. Each round allowed a 2-week window to submit responses with reminder e-mails and a 1-month gap between rounds.

For all panels, at the beginning of round 1, all experts were asked to review background information to understand the context and purpose of the Delphi panel project. First, they were introduced to the program goal: to better equip biomedical research trainees for their future careers by teaching them I&E skills. Second, they were introduced to the EntreComp Framework and the 15 high-level competencies that we had grouped into five overarching categories: management, vision and imagination, social skills, psychological skills, and ethical and decision-making skills. Finally, the experts were asked to make the following assumptions: “(1) All course participants are enrolled in a training program to pursue a career in biomedical research. (2) They may work in the university, government, industry, and other settings. (3) They may or may not become entrepreneurs. (4) The course they were considering would be introductory, and approximately 20 hours long.”

The EntreComp Framework Panel

The goal of this panel was to validate the EntreComp Framework as a whole for biomedical researchers. The seven panelists were provided a list of all 15 EntreComp Framework high-level competencies and a brief description of each (Table 2). They were asked whether each competency was relevant for biomedical research trainees (yes/no); in the case of a “no” vote, they were asked to provide a rationale. Panelists were also asked if any additional competencies should be added to the list. A priori, we defined agreement among the experts that the EntreComp competency was relevant to biomedical research trainees as a yes vote from 5/7 panel members. This approach is consistent with other Delphi methodology [36]. We chose to use this variation on the Rand criteria due to our sample size and dichotomous choice. If new competencies were identified by individuals, they would be presented to the panel in a second round to evaluate for consensus.

Panels 1–5

The goal for Panels 1–5 was to identify topics to be included within the training program for the EntreComp competencies assigned to that panel. In round 1, participants were presented with their panel’s EntreComp high-level competencies and brief definitions (Table 2). Then they were asked “What content do you think should be taught? Content can be described as topics, knowledge, skills, or attitudes. Please list as many content areas that you think should be included.”

For each panel, responses from round 1 were analyzed to eliminate redundancy and the responses were summarized. In round 2, panelists were asked to review the summarized list and rate the importance of each topic for an introductory course in entrepreneurial thinking for biomedical research trainees using a 5-point scale from not at all important (1) to extremely important (5). For each topic they provided their rating for two student groups: entrepreneurs (those with an interest in starting their own ventures) and intrapreneurs (those who want to be innovative and strategic within pre-established companies or an academic career) [37]. For each panel, the mean importance score of all expert panelists was calculated for each topic. We defined topics with a mean importance score > 4.0 (very or extremely important) as having a consensus that the topic was “highly important” to teach. This approach is consistent with other Delphi methodology that defines a consensus using a mean score [36]. We adopted a 5-point rating system (rather than 9 point) to make it easier to label each option. Participants were also asked to use open-ended textboxes to provide other content areas they thought were missing from the list.

After analyzing results from round two, it was decided that a third round for Panels 1–5 was not required as the results were intended to guide curriculum development by providing an expert generated list of topics, rather than build consensus.

Results

Validity of EntreComp Competencies for Biomedical Research Trainees

The seven experts on the EntreComp Framework panel agreed that the 15 EntreComp competencies were all important for biomedical trainees. After analyzing results from round 1, we determined that a second round for this validation panel was not required as no new high-level competencies were identified.

Altogether 207 topics were generated across the five panels in round 1, resulting in 120 summarized topics (17–31 per panel) for analysis in round 2. Overall, 36/38 (95%) experts submitted responses for both rounds of the Delphi process.

Course Content

The five panels achieved consensus that 85 (71%) of the 120 topics ranked in round 2 were of high importance to include in the curriculum. These included 42 (49%) topics considered to be of high importance for all biomedical research trainees regardless of future career direction, 35 (41%) additional topics for entrepreneurs only, and 8 (9%) topics for intrapreneurs only (Table 3). In Table 4, topics that reached consensus for entrepreneurs and intrapreneurs are listed by panel grouped by importance for trainee’s career direction (both entrepreneur and intrapreneurs, entrepreneur only, and intrapreneur only). The complete data for all panels are provided in Appendix B.

Table 3.

Number of topics assessed in round 2 of the modified Delphi process by panel

Panel	Total topics scored	Number of topics that reached consensus
		Overall	Both E&I	E-only	I-only
Management	23	15	5	10	0
Vision	17	14	6	8	0
Social skills	27	20	11	7	2
Psychological skills	22	17	12	2	3
Ethical and decision-making skills	31	19	9	7	3
Total	120	85	42	35	8

E, entrepreneurship; I , intrapreneurship.

Table 4.

Innovation and entrepreneurship (I&E) topics that reached consensus, by panel

	Panel 1	Panel 2	Panel 3	Panel 4	Panel 5
	Management	Vision and imagination	Social skills	Psychological skills	Ethical and decision-making skills
Consensus for both entre- and intra-preneurs	Team management skills.Goal setting.Decision making skills.Self-management skills.Project management skills	Communication skills to illicit different perspectives.Identify assumptions of business plan.Problem formulation strategies.Know the state of your industry.How to screen ideas.How to think divergently and come up with many solutions	Formal presentation skills.Active listening skills.Give and receive feedback.Conflict resolution.Goal setting.Team dynamics.Improvisational dialogue skills.Courtesy skills.How to develop meaningful interpersonal relationships.How to be assertive.Awareness of your emotions and ability to control your reactions to those emotions	Make the most of your strengths and capabilities.How to deal with failure.Methods to measure progress to inform future decisions.Self-management skills.Networking skills.Use goal setting as motivational strategy.Grit and perseverance.Be aware of cognitive biases.Develop team trust.Experimental skills to test hypotheses.How to anticipate problems.How to coach others	How to recognize and take stock of your circumstances.How to ethically interact with patients.Ethical interactions with industry and institutions.Ethical pragmatism.Intellectual property issues.Questioning your own and other’s judgment: understanding biases.Know unmet health needs.Develop an experimental mind-set.Know how to interact ethically with other professionals
Consensus for entrepreneurs only	Knowledge of the different types of funding sources, and how and when to apply for them.Team building.How to define the resources you need for your venture.Facilitation skills.Fundamentals of product adoption.Fundamentals of intellectual property.Fundamentals of product approval.How to pitch an idea or otherwise build, improve, and defend business cases.How to create a business plan.How to work with outside partners and institutions	How to identify your minimum viable product (MVP).How to perform market research.Understand what inspires and motivates you to act.Interview techniques for different audiences.Know what investors want to see before investing.Know market trends and underlying models of supply and demand.Know entrepreneurial ventures in your field.How to find uses for existing technologies	Networking skills.Survival skills such as resilience, managing change, and risk management.Negotiating skills.How to build a personal brand.How to utilize body language effectively.How to create and deliver elevator pitches.Establish norms and expectation of the team	Self-efficacy.Person-environment fit	Resiliency skills.Seek outside help to make decisions.Establish your central vision and refer back to it to guide decision making.Anticipate worst case and most likely consequences of your actions.Implementation considerations, being thrifty when expanding in new directions.Stay up to date with global issues.Know a framework for rational economic decision making, such as Net Present Value
Consensus for intrapreneurs only			How to lead when you don’t have a position of authority.How to “manage up”	Give/receive feedback.Sense-making within teams.Emotional intelligence: manage your own and others emotions	Conflicts of interest and ethical issues underlying them.Moral reasoning.Cross-cultural communication

Although each panel was assigned different EntreComp competencies in rounds 1 and 2, several topics were identified as high importance to include in the curriculum for both career groups by multiple panels. For entrepreneurs, these included resiliency/self-management (four panels), goal setting (three panels), team management (three panels), and communication skills (three panels). These same topics were identified by multiple panels for intrapreneurs, resiliency/self-management (two panels), goal setting (three panels), team management (three panels), and communication skills (four panels).

Discussion

The six modified Delphi panels reached our two aims – to identify the I&E competencies a biomedical researcher should possess and to identify topics to be included in the core curriculum to introduce I&E skills to biomedical research trainees. Our findings suggest our introductory training program should address all 15 EntreComp competencies, yet be flexible to accommodate variation in needs of trainees on different career trajectories. One approach to meet the need for breadth and depth of course offerings is to provide both core and elective courses.

Our expert panelists provided guidance about topics to be included in core and elective courses. Half of the 85 topics identified as high importance were ranked as high importance for all trainees, regardless of their career trajectory. Reviewing this topic list suggests that core courses should be designed to teach trainees the following I&E skills: how to identify opportunities for innovation; how to determine their potential for success; how to communicate about your innovation idea to various audiences; how to build and manage teams; how to be aware of the ethical consequences of your decisions and actions; and self-management and resiliency. Topics identified as high importance for only one career group could be considered as electives, allowing trainees to tailor their program to meet their needs. Skills of particular interest to entrepreneurs might be learned through elective courses focused on commercialization such as identifying your funding needs and opportunities, and building a business plan. Elective courses of interest to intrapreneurs might include managing up and social entrepreneurship including cultural competencies. Elective courses also provide opportunity for more in-depth coverage of topics that might be relevant only for some trainees.

With the goal of improving human health through translational innovation, I&E skills are important for the translational research workforce of the future [38]. An important initiative in this regard is the Clinical and Translational Science Award (CTSA) program with the goal of efficiently translating research knowledge into improved health [39]. Several core I&E skills identified by our experts have previously been identified as core competencies for clinical and translational science, including the “soft skills” needed to function as a professional and to work in multi- and interdisciplinary teams [38, 40–42]. Courses in communication and team science are provided at many CTSA sites across the country as part of degree programs in clinical and translational research [38, 40, 41]. Training in the more traditional entrepreneurial skills such as design thinking and commercialization is offered through engineering and business schools [43]. For some there is a tension between the goals of medicine and science and entrepreneurship. Medicine and science are founded on goals such as improving health, creating generalizable knowledge, openness, and transparency [44-46]. These goals may conflict with goals of entrepreneurship where profits and financial motives may lead to secrecy, proprietary claims, and competition rather than collaboration [28, 29, 47]. Evidence has shown that financial incentives and motives can affect decision making, change behaviors, and potentially lead to unethical actions, for instance through conflicts of interests [29, 46, 48, 49]. At the same time, collaborations between industry and academia are now the norm, alongside an emphasis on translational science, suggesting we should not abandon these relationships but rather we need to ensure that individuals are aware of these tensions [47, 49, 50]. This is our rationale for including ethics experts in our panels and for requiring an ethics course for all students, regardless of track. Our web-based introductory program will increase opportunity for pre-and post-doctoral students to access training in I&E skills that are relevant for translational researchers. The final program will include a bootcamp and a team-based capstone project to provide learners opportunities to integrate their new knowledge and skills.

It is notable that the topics of self-management and resilience were rated as highly important for both entrepreneurs and intrapreneurs across multiple panels. Our experts recognized resilience as an important skill to deal with the ups and downs of innovation. Indeed, entrepreneurial resilience has been shown to have a significant positive relationship with success for individuals and businesses [51, 52]. Overall, research suggests that resilience is a modifiable construct and not an inherent, immovable trait [53]. Encouraged by a recent metanalysis that found “a modest but consistent benefit of resiliency training programs in improving a number of mental health outcomes within three months of follow-up” [53], we plan to have a core course to improve resilience skills. We will carefully evaluate the courses as we note that the authors commented that the 13 randomized controlled trials included in the meta-analysis were small and generally of poor methodological quality [53].

Limitations

Our panelists each had I&E experience relevant to biomedical research, but it cannot be assumed that a different panel of experts would reach the same conclusions regarding competencies and program content for an I&E curriculum for biomedical research trainees. We based the definition of highly important topics for learner groups on common sense and common practice [36], and changing this definition might change the conclusions drawn from the study. We provide the complete study data in Appendix B to allow the reader to review all topics suggested by the expert panelists, not just the ones that we identified as being highly important. We focused on identifying topics to include in our curriculum and did not ascertain the level of mastery expected of learners. Our program will be an introductory course for pre- and postgraduate students, and the learning objectives will determine the level of mastery expected.

Conclusion

The six modified Delphi panels identified topics to be included in a basic training program to encourage an entrepreneurial mind-set in biomedical research trainees. We will use these findings to inform the development of our introductory program in I&E training targeting this group, specifically to inform learning objectives, course content, and designation as a core or elective course. While these training recommendations are based on the expert consensus, we will need to assess learner outcomes and reactions to evaluate the success of our program. Additional considerations such as how the course should be implemented (in-person, web-based, team-based), course duration, and the roll of additional components such as mentoring and a capstone project to synthesize learning need further exploration.

Table B1.

Management panel prioritized learning topics for biomedical I&E (ranked by entrepreneur mean importance)

Topic presented to panelists	Entrepreneur		Intrapreneur
	Mean importance (5-point scale)	N rating high (4) or essential (5)	Mean importance (5-point scale)	N rating high (4) or essential (5)
Consensus
Knowledge of the different types of funding sources, and how and when to apply for them (venture capital, grants, angel investors, etc.)	4.83	6/6	3.17	3/6
Team management skills: how to lead, coordinate, and trouble-shoot the combined efforts of a group of individuals	4.67	6/6	4.17	5/6
How to create or identify goals?	4.67	6/6	4.33	5/6
Decision-making skills	4.67	6/6	4.5	6/6
Self-management skills: How to take responsibility for your own well-being and behavior; this includes time-management, organization, self-motivation, self-care, and accountability?	4.67	6/6	4.67	6/6
Team-building: How to identify and evaluate the skill sets and capabilities needed for your team?	4.5	6/6	3.83	3/6
Project management skills	4.5	5/6	4.17	5/6
How to define the resources needed for your venture?	4.33	6/6	3.67	4/6
Facilitation skills: How to guide a group through a process in a way that maximizes individual participation and productivity?	4.33	6/6	3.83	4/6
Fundamentals of product adoption: What it takes to get consumers to adopt a new product (work flow, interface, reimbursement strategies, etc.)	4.17	5/6	3.67	3/6
Fundamentals of intellectual property: When and how to file for IP protection, non-disclosure agreements, etc.?	4.17	5/6	3.83	5/6
Fundamentals of product approval: What is involved in getting a drug or device approved?	4.17	5/6	3.83	3/6
How to pitch an idea or otherwise build, improve, and defend business cases?	4.17	4/6	3.67	3/6
How to create a business plan?	4	5/6	3.33	3/6
How to work with outside partners and institutions, including knowledge of what resources are available via contract research and manufacturing organizations and how to manage contracts?	4	4/6	3.67	3/6
No consensus
Knowledge of regulations	3.83	4/6	3	2/6
Fundamentals of start-up management: How to create a company and administer operations throughout the early stages of development?	3.83	3/6	2.67	1/6
Fundamental business topics: such as finance, accounting, operations, technology, and legal	3.67	3/6	2.67	0/6
How to perform market research: gathering information about consumers, competitors, and current market trends to help inform decisions?	3.67	3/6	3.17	2/6
Interviewing techniques to obtain information from consumers, peers, and competition	3.67	3/6	3.33	2/6
Fundamentals of quality assurance: GLP, GCP, and GMP (good laboratory practices, good clinical practices, and good manufacturing practices)	3.5	3/6	3.17	2/6
Improvisational dialogue skills: how to talk off-the-cuff?	3.33	1/6	3.17	1/6
Familiarity with basic biostatistics tools	3.17	2/6	3.17	2/6

Table B2.

Vision and imagination panel prioritized learning topics for biomedical I&E (ranked by entrepreneur mean importance)

Topic presented to panelists	Entrepreneur		Intrapreneur
	Mean importance (5-point scale)	N rating high (4) or essential (5)	Mean importance (5-point scale)	N rating high (4) or essential (5)
Consensus
Communication skills, to engage with diverse types of people and bring different perspectives to the table	4.78	9/9	4.33	7/9
Identifying the assumptions that your business plan relies upon	4.78	9/9	4.22	8/9
Problem formulation strategies, such as how to identify unmet needs in the world	4.67	9/9	4.11	6/9
How to identify your minimum viable product (MVP): the core solution/product behind your ideas to be subjected to potential customer feedback?	4.67	9/9	3.89	6/9
How to perform market research?	4.56	9/9	3.89	7/9
Knowledge of the state of your industry/technology	4.56	8/9	4.22	7/9
How to screen ideas: identify bases for comparison and define criteria against which to evaluate your ideas?	4.44	8/9	4.33	7/9
Understanding what inspires you and motivates you to act	4.44	7/9	3.78	5/9
How to think divergently, and come up with many alternatives given a situation or problem?	4.33	8/9	4.11	8/9
Interviewing techniques to obtain information from consumers, peers, and competition	4.33	8/9	3.89	5/9
Knowledge of what investors want to see before they invest in a venture	4.33	8/9	3.44	4/9
Knowledge of market trends and the underlying models of supply and demand	4.22	8/9	3.44	3/9
Knowledge of entrepreneurial ventures and companies inside your field	4.22	7/9	3.33	3/9
How to find new uses for existing technologies?	4	7/9	3.89	7/9
No consensus
How to assess the intellectual property landscape?	3.78	6/9	3.11	3/9
How to benchmark technologies and identify better ways of doing what’s already on the market?	3.67	5/9	3.56	5/9
Knowledge of entrepreneurial ventures and companies outside of your field	3	2/9	2.44	1/9

Table B3.

Social skills panel prioritized learning topics for biomedical I&E (ranked by entrepreneur mean importance)

Topic presented to panelists	Entrepreneur		Intrapreneur
	Mean importance (5-point scale)	N rating high (4) or essential (5)	Mean importance (5-point scale)	N rating high (4) or essential (5)
Consensus
Formal presentation skills, including the use of visual aids	4.67	6/6	4.5	5/6
Active listening skills	4.5	6/6	4.83	6/6
How to effectively give and receive feedback?	4.5	6/6	4.33	6/6
Conflict resolution skills	4.5	6/6	4	4/6
Networking skills: how to establish, maintain, and productively use professional relationships?	4.5	6/6	3.83	3/6
Goal setting strategies	4.5	5/6	4.17	5/6
Fundamentals of team dynamics: how individuals interact in groups, and how to get groups of people to efficiently work together?	4.33	5/6	4.67	6/6
Improvisational dialogue skills/how to talk off-the-cuff	4.33	5/6	4	4/6
Survival skills, such as resilience, managing change, and risk management	4.17	5/6	3.67	3/6
Negotiation skills	4.17	5/6	2.83	1/6
Courtesy skills: being polite, sincere, and sensitive to the needs of others	4.17	4/6	4.33	5/6
How to develop meaningful interpersonal relationships, by caring about others and identifying what matters to them?	4.17	4/6	4.17	4/6
How to build a personal brand: establishing and promoting how you want others to perceive you?	4	6/6	3.33	3/6
How to be assertive: to stand up for your own or others’ rights and needs in a respectful way?	4	5/6	4.33	6/6
Emotion management skills: awareness of your emotions and ability to control your reactions to those emotions in an appropriate manner	4	5/6	4	5/6
How to utilize body language effectively?	4	5/6	3.67	3/6
How to create and deliver elevator speeches?	4	5/6	3.5	3/6
The importance of taking the time to establish norms and expectations in the team	4	5/6	3.33	2/6
How to lead when you don’t have an official position of authority?	3.17	3/6	4.33	6/6
How to “manage up”: establishing and maintaining a productive relationship with your boss?	2.83	1/6	4.33	6/6
No consensus
Metacognition skills: How to be self-aware and self-assess one’s own learning?	3.83	5/6	3.67	4/6
How to cultivate an executive presence/charismatic personality ?	3.83	5/6	3	1/6
Methods for coaching or mentoring others	3.67	5/6	3.5	2/6
Cultivating a growth mindset: having confidence in your skills and in your ability to grow	3.67	4/6	3.5	3/6
Knowledge of different motivational strategies	3.67	3/6	3.5	3/6
Knowledge of different leadership strategies	3.5	4/6	3.5	3/6
How to use Twitter effectively>	2	0/6	1.67	0/6

Table B4.

Psychological skills panel prioritized learning topics for biomedical I&E (ranked by entrepreneur mean importance)

Topic presented to panelists	Entrepreneur		Intrapreneur
	Mean importance (5-point scale)	N rating high (4) or essential (5)	Mean importance (5-point scale)	N rating high (4) or essential (5)
Consensus
Identifying your strengths and capabilities, and making the most of them	4.86	7/7	4.71	7/7
How to deal with failure or the threat of failure?	4.71	7/7	4.57	7/7
Methods for measuring progress to inform future decisions, such as “post-mortems” or agile methodologies	4.43	6/7	4.43	6/7
Self-management skills: how to take responsibility for your own well-being and behavior; this includes time-management, organization, self-motivation, self-care, and accountability?	4.43	6/7	4.14	6/7
Networking skills: How to establish, maintain, and productively use professional relationships?	4.29	5/7	4.43	6/7
Using goal-setting as a motivational strategy	4.29	5/7	4.29	5/7
Grit: How to persevere toward goals when it’s tough, and foster a pursuit of passions despite setbacks?	4.29	5/7	4.14	5/7
Knowledge of cognitive biases, such as sunk costs (the unwillingness to leave a project that has failed because one feels that they have put too much effort in it to walk away)	4.29	5/7	4	5/7
Development of interpersonal and team trust to promote candid conversations	4.14	6/7	4.29	6/7
Smart experimentation skills: How to test hypotheses about the environment or about the viability of products or services?	4.14	4/7	4	4/7
How to generate ideas about what could go wrong before a project begins, such as using a “pre-mortem”?	4	6/7	4.14	6/7
How to coach or mentor others?	4	5/7	4	5/7
Self-efficacy: belief in your ability to achieve a goal	4	5/7	3.86	5/7
Person-environment (PE) fit: the idea that matching characteristics, values, and needs between people and their workplaces leads to positive outcomes	4	4/7	3.57	3/7
How to give and receive feedback effectively?	3.86	4/7	4.29	5/7
Emotional intelligence: How to identify and manage your own emotions as well as others?	3.71	5/7	4	5/7
Sensemaking within teams: How to work with a group to make sense of an unexpected event and decide upon a course of action?	3.71	4/7	4.14	5/7
No consensus
Entity (intelligence is unchangeable) vs incremental (intelligence can be increased through effort) theory: the idea that a growth mindset facilitates continued effort after failure rather than helpless responses	3.86	5/7	3.86	5/7
The concept of promotion (pursuing gains) vs prevention (avoiding losses) mindsets when pursuing a goal	3.86	5/7	3.71	4/7
Vicarious learning: using others’ stories to learn in the absence of firsthand experience	3.71	4/7	3.71	4/7
Knowledge of implicit biases: unconscious beliefs about different social groups	3.43	4/7	3.43	4/7
Fundamentals of personality theory: such as big five traits (openness, conscientiousness, extraversion, agreeableness, and neuroticism), individual information processing differences, Myers-Briggs	3.14	3/7	2.86	2/7

Table B5.

Ethical and decision-making skills panel prioritized learning topics for biomedical I&E (ranked by entrepreneur mean importance)

Topic presented to panelists	Entrepreneur		Intrapreneur
	Mean importance (5-point scale)	N rating high (4) or essential (5)	Mean importance (5-point scale)	N rating high (4) or essential (5)
Consensus
How to recognize and take stock of your circumstances?	4.75	8/8	4.25	7/8
How to interact ethically with patients?	4.5	8/8	4.62	8/8
Resiliency skills: How to deal with failure, or the threat of failure?	4.5	8/8	3.75	5/8
Ethical interactions with industry and other partner institutions, including potential sources of conflicts of interest	4.5	7/8	4.15	7/8
Ethical pragmatism: keeping your core ethical principles intact, while dealing with difficult practical matters	4.5	7/8	4	6/8
Intellectual property issues	4.5	7/8	3.75	5/8
Questioning your own and others’ judgment: understanding biases, pretenses, and presumptions	4.375	7/8	4.38	7/8
Knowledge of unmet health needs	4.25	8/8	4	7/8
Seeking outside help to make decisions	4.25	7/8	3.88	4/8
Establishing a central vision for what you want and referring back to that vision to guide decisions	4.25	7/8	3.75	6/8
Anticipating consequences of actions: imagining worst-case and most-likely scenarios; asking how actions will affect others; deciding what will always be uncertain	4.25	7/8	3.63	5/8
Implementation considerations: paying attention to operations and being thrifty when expanding in new directions	4.25	7/8	3.38	2/8
Staying up to date on what is happening around the world	4.125	7/8	3.88	6/8
Developing an experimental mindset	4	8/8	4	6/8
How to interact ethically with other professionals?	4	7/8	4.13	8/8
Frameworks for rational economic decision-making, such as Net Present Value calculations	4	7/8	3.25	4/8
Conflicts of interest and the ethical implications underlying them	3.875	5/8	4.13	6/8
Moral reasoning: identifying and weighing the values and interests at stake in different situations	3.875	5/8	4	7/8
Cross-cultural communication	3.875	5/8	4	6/8
No consensus
Using timelines/milestones to evaluate progress for initiatives that have been undertaken	3.875	6/8	3.88	6/8
Issues of power and authority as they affect ethics and risk	3.875	6/8	3.75	5/8
Issues of financial stressors as they affect ethics and risk	3.875	6/8	3.5	5/8
Ethical impacts on different communities	3.875	5/8	3.88	6/8
Knowledge of the Triple Bottom Line (People, Planet, Profit)	3.875	5/8	3.5	4/8
Cultural sensitivity skills	3.875	4/8	3.88	5/8
Emotional intelligence skills: How to identify and manage your own emotions as well as others’?	3.875	4/8	3.75	4/8
Analyzing personal motivations	3.75	6/8	3.5	5/8
Knowledge of the impact that environmental sustainability and climate change have on the global economy	3.625	3/8	3.75	4/8
Career survival skills and considering the long-term impact of your early choices	3.5	3/8	3.63	5/8
Knowledge of health disparities	3.5	3/8	3.38	4/8
The study of ethical and influential leaders	3.375	5/8	3.25	5/8

Table B6.

Panel 6 responses

Competency	Yes	No	Rationale for no
Planning and management	7	0
Financial and economic literacy	6	1	Financial planning is a managerial role and not necessarily the role of a basic scientist. This is not needed by everyone and in fact should be a task allocated only to the finance staff of a company, start-up, or university
Mobilizing resources	7	0
Spotting opportunities	6	1	While this seems like a “yes” answer, in fact basic scientists or managers with a definite task should focus on that and not be looking for “opportunities” wherever they exist.
Vision	5	2	(1) While important to start with a vision, this needs a better definition in entrepreneurship and innovation. A vision may need to be very flexible in this setting, but are sometimes rigid and can be a hinderance for innovation.(2) Again, many people work in a focused area without the need to visualize the future – I want my staff to get the analysis done, and not worry about how or if we will market certain areas for future growth
Valuing ideas	6	1	Value to me is a monetary return and often i do not need staff to worry about this. It is often sufficient for the boss to say do this and not needed for everyone to be able to demonstrate this
Creativity	6	1	Is your future job to plow through a defined scientific problem using established tools, or is it to develop an independent group (lab, start-up, core facility)?
Self-awareness and self-efficacy	7	0
Mobilizing others	6	1	This is an important leadership trait, but may be delegated to a manager. Effective entrepreneurs are not always the best managers
Working with others	7	0
Learning through experience	7	0
Taking the initiative	6	1	Not everyone needs or has leadership skills
Motivation and perseverance	6	1	Not everyone needs or has leadership skills
Coping with uncertainty, ambiguity, and risk	7	0
Ethical and sustainable thinking	6	1	I don’t believe everyone has the cognitive ability to analyze the consequences of their work