FAMMOCN - Demonstration and evaluation of a framework for the multidisciplinary assessment of organisational maturity on business continuity.

Business Continuity Management (BCM) encompasses effective planning to respond to business interruptions and relaunch business in the short term. This study follows the Design Science Research methodology and proposes a framework to systematise Business Continuity Management and streamline the Business Continuity Plan (BCP) design and implementation. The framework defines metrics providing strategical guidance and assessment of the Business Continuity Management initiatives. The framework provides a Business Continuity Management Model, an Implementation Guide, a Self-Assessment System, and a Measurement System. The model was developed based on a systematic literature review and guidelines from Business Continuity Management frameworks and standards. In the first iteration, we demonstrated and evaluated the framework through a Focus Group with experts in Business Continuity Management. In the second iteration, it was used and evaluated by professionals with responsibilities in Business Continuity Plan implementation, representing various business sectors. As a result, the framework is useful and complete, effective and enhances governance and is scalable and adaptable to organisations. This study concludes that the framework adds value to Business Continuity Management monitoring, gaps identification, and practitioner's guidance on what needs to be planned, done, checked and acted to manage continuity.

Introduction

Not protecting information, or worse losing it, is critical to an organisation’s survival. Hence, organisations must prepare for the eventuality of interruptions in their business processes, especially those supported by Information and Communication Technology (ICT) services.

In this context, there are benefits to implementing a Business Continuity Management (BCM) program (Russo and Reis, 2020a). As a BCM output, a Business Continuity Plan (BCP) is designed to avoid or mitigate risks, reduce the impact of crisis or disaster conditions, and reduce the time needed to restore conditions to a normal operating state (Cerullo and Cerullo, 2004). Consequently, BCM is more than Risk Management, and other components of BCM must be managed effectively to provide a BCP. Therefore, to design a BCP, it is necessary to understand each of the activities of the BCM and gain the organisation’s commitment to improve and prioritise program development activities (Isa et al., 2019). Thereby, the planning for BCM prepares the organisation to maintain the continuity of its services during the occurrence of a disaster by implementing a contingency plan.

For the design of BCP and in the BCM, organisations benefit from the existence of a framework that guides the activities for mitigation of business disruptions. Organisations, when designing a BCP focused on ICT, may experience time or technical constraints due to the need to obtain an effective and urgent Business Continuity (BC) response, but also constraints in accessing BC expert knowledge. These constraints amplify the perception of complexity in the establishment and maintenance of the BCM program, potentiating the non-use of internationally accepted frameworks (Krell, 2006). Whether due to its technicality or to the wide scope of intervention at the organisational level, which as a whole may delay the achievement of a comprehensive view of its BCM capacity in a short time, and prioritise action in the most relevant areas, the BCM program may not be considered a priority and strategic by the organisation’s decision-makers.

The key objective of the study is to develop, demonstrate and validate a Framework for the Multidisciplinary Assessment of Organisational Maturity on Business Continuity (FAMMOCN). FAMMOCN allows for the design and implementation of a BCP, considering the specificity and organisation size, through the multidisciplinary assessment of organisational BC maturity. The framework allows assessing the organisation’s maturity, and its preparation level to ensure BC and the primary activities to be developed. It will guide the revelation and analysis of the elements considered essential in the formulation of a BCP, focused on ICT systems, advocating the mitigation of the identified constraints and gaps.

After the problem identification and definition, we inferred the major challenge is to design a solution providing the information that can support an organisation in obtaining the desired BC response as an outcome. Moreover, the information provided must streamline the BC organisational processes without adding an overload of administrative tasks and unattainable BC knowledge by the organisation’s human resources. Furthermore, to check if a solution answers the identified problem and applies to a wide set of organisations, we defined attributes like clarity, completeness or adaptability to help the demonstration and validation steps. These two requirements must be balanced to allow the design and implementation of BC response according to the maturity and capacity of the organisation, scaled by the pre-defined level of operationality.

This study combines a systematic literature review that aims to collect strategic guidelines to develop a solution with references that considered how BC maturity or preparedness can be assessed and what are the professional best practices. Following the Design Science Research (DSR) methodology proposed by Peffers et al. (2007), we used this theory as a basis for the design and development of the framework. The demonstration and evaluation steps used Focus Groups and Semi-structured interviews. These two techniques used in the DSR process iterations provide a rich discussion and inputs on how to solve the problem, the related concerns and constraints. The interview and the simulation with the framework allowed the framework to be evaluated by organisations. The interviews were guided by a set of predetermined questions related to the research question and by a set of characterising attributes.

The results of the evaluative techniques show that the Group BC experts and the organisation’s professionals interviewed agreed that the framework can add value in supporting and streamlining organisational processes for implementing a BCP.

This paper is organised into the following sections: after the introduction, the second section highlights the background research and a synthesis of the systematic literature review. The third section presents the methodology, the research plan, and the question. Considering that it is not an objective of this paper to present the framework in detail, the fourth section introduces the framework developed. The fifth section reports its demonstration and evaluation steps accordingly to the adopted methodology. The sixth section presents the results and discussion. Finally, the conclusions are mentioned, followed by a list of all the references used.

Background research

The study started with background research related to the constraints in implementing a BCP and adopting a BCM System (BCMS) to cope with the risk of business disruption. Hence, we pursue to understand its potential causes, constraints, and gaps. Restricting company policies and time constraints to finish the BC projects are causes (Fani and Subriadi, 2019). Small and medium-sized enterprises are more susceptible to disasters due to their limited resources and capacities to bounce back from disasters and are constrained by financial, human resources, and technological deficiency (Kato and Charoenrat, 2018). Other constraints are a lack of proactive BC and disaster recovery planning that may lead to loss of reputation and market share, customer service and business process failure, regulatory liability and increased resuming and restoring (Sahebjamnia et al., 2015).

There is an awareness gap of the benefits of BC planning among business Top Managers (Bethany, 2014). Gallo (2021) shows that there is a gap in staff skills on resilience or recovery from interruptions and the importance of the application of a BCMS. When implementing a BCMS, there are also constraints and limitations due to feasibility and implementation issues and cost restrictions (Aronis and Stratopoulos, 2016). Guidance for BCM implementation because of the lack of knowledge in BC or BCM frameworks (Russo and Reis, 2020b; Wong, 2009) is also a cause for delaying the adoption of a BCMS. Nevertheless, there is a gap in organisations that implement and are certified on a Business Continuity Management Standard, especially in the public sector (Hamid, 2018).

In this context and to address the mitigation of the constraints, organisations can select from several International BCM and ICT frameworks and standards, hereinafter referred to as Standards, to guide the implementation of the BCP and a comprehensive BCMS. The International Organization for Standardization (ISO) specifies the requirements for implementing a BCMS (ISO 22301, 2019). Capability Maturity Model Integration (CMMI) proposes the planning for mitigation activities to cope with significant disruptions to business operations (CMMI Institute, 2018). In the BC scope, the primary objective of Control Objectives for Information and related Technology (COBIT) is to provide a plan to enable business and ICT organisations to respond to incidents and quickly adapt to disruptions (ISACA, 2018). Information Technology Infrastructure Library (ITIL) presents the Service Continuity Management Practices to ensure the availability and performance of a service, in case of a disaster (ITIL, 2019). National Fire Protection Association (NFPA) provides fundamental criteria for preparedness through a program that addresses prevention, mitigation, response, continuity, and recovery (NFPA 1600, 2019).

Implementing an adequate and effective BCMS is a challenging, demanding, time-consuming and holistic process (Aronis and Stratopoulos, 2016). With this in mind, there is the need to streamline the organisational process of implementing a BCP and support an organisation in this achievement. Since small and medium-sized enterprises have fewer resources and knowledge for implementing a BCMS, there is a need to provide strategic guidelines for implementing a BCP that can adapt to the organisation’s maturity and capacity.

Hence, first, it is relevant to identify the components of the BCM and the requirements to narrow the identified gaps and ease the constraints. The Systematic Literature Review (SLR) provided the theory (Kitchenham, 2004) to support the achievement of the solution to the problem and its requirements. The SLR identified the BCM components and activities, communicated between 2000 and 2021. It gathered a set of strategic guidelines that complement the Standards for each of the BCM components. Thus, the design of the BCP must consider the support provided in the strategic guidelines suggested by the studies. Altogether, can be combined or integrated into a comprehensive set of guidelines to streamline the organisational processes for the BCP design.

In Table 1, there is a synthesis of the number of publications grouped by BCM component that was used as a basis for the understanding of what are the major concerns and areas of interest in BCM research.

Table 1

Quantitative synthesis of publications, by BCM component.

BCM Component	Number of publications
Administration Support and Commitment	48
Understanding the Organisation	31
Risk Assessment	167
Business Impact Analysis	58
BCM Strategy	121
ICT Strategy and alternatives to critical functions	155
BCP Design and Implementation	163
BC Training	20
BCP Testing, Maintenance, and Analysis	68

All areas are important to be considered in the BCP design. Nevertheless, the analysis of Table 1 revealed a higher number of publications addressing Risk Assessment, BCM or ICT Strategy and BCP Design and Implementation. The BCP or Disaster Recovery Plan (DRP) is mentioned in every SLR publication since they are part of the search string.

Farr and Bailey (2019) explore the interrelation between various programmes in the BC scope, like operational risk management, BCM and other related, as an effective risk structure for an organisation. They outline how uniting these programmes would benefit continuity practitioners. In their research, Sahebjamnia et al. (2018) states that managers need to address specific features of BCP and DRP for implementing effective BCMS by prescriptive rather than descriptive approaches. Despite the approach method, some practical issues need to be addressed, namely attention to process enhancements and changes, or inappropriate approaches to executing processes (Fernando, 2017).

For this purpose and focused on risk, Păunescu and Argatu (2020) outlines the composing elements of a BCM and showed the interactions between these elements meant to ensure the foundations of effective business continuity management. The BC response planning has the most significant impact on building an effective BCM, followed by BCM exercising, maintaining and reviewing and embedding the BCM in the organisation’s culture.

Although some essential aspects are common, Fani and Subriadi (2019) concludes that a BCP should be done according to the needs of the organisation and each would have different stages in designing and implementing a BCP. Some authors focus on success and critical factors, discussing cases of success in BCM adoption and implementation but lacking constant monitoring leading to a failure (Kim and Amran, 2018). Another example is organisations that see BCM as a complex practice will not further pursue it, as they deem extra resources required will further drain their financial resources.

Considering the importance of having a BC response, the understanding of what are the key BC areas to include in the solution will determine its scope and the right information to support the achievement of a BCP. The critical and the success factors will advise on the importance of selecting and defining the activities in the BC components. The strategic guidelines collected in the literature and the Standards will guide the design of the proposed solution presented in this study. We assume the challenge is to develop a framework that respects the knowledge obtained in the literature.

Research plan and questions

Research plan

This research follows the DSR methodology. The first step of the DSR Process Model presented by Peffers et al. (2007) was accomplished by preliminary research to identify and define the problem. The inference of the problem identified drafted the objectives of the solution. This second step was consolidated by integrating the SLR methodology to sharpen the objectives of the artefact. This combination ensures rigour across the DSR Process Model steps (Peffers et al., 2007).

The SLR is part of the research methodology and a prerequisite for a quantitative meta-analysis, summarising existing evidence and identifying gaps in current research. The SLR follows the guidelines provided by Evidence-Based Software Engineering (EBSE) (Kitchenham, 2004), with the help of the online tool Parsifal. The SLR’s objective is to identify the essential BCM components, activities and related strategic guidelines which can streamline the BCP design and implementation, appropriate to the specificities of organisations.

In March 2021, the SLR was completed using the EbscoHost, ScienceDirect, and Scopus databases. The primary search terms were: “business continuity plan” or “disaster recovery plan”. These search terms were combined with “framework”, “guideline”, and “streamline”, along with a set of synonyms. The publication must present the search terms in the title, abstract, or keywords to be included in the review. The publication date ranges from 2000 to 2021.

The primary search returned 10,356 publications. Applying the identification criteria resulted in 1240 potentially eligible publications. 14 publications were manually added since they didn’t fit the search string criteria, although highly cited in BCM papers.

About 30% of the 1254 articles were eligible, referring to strategic guidelines that apply to all organisations, as defined in the identification criteria. The other publications (70%) had specific considerations for a population or context and, therefore, were not included in the quality assessment. All the 398 full-text publications selected were read and analysed for quality assessment to provide more detailed exclusion criteria (Russo et al., 2021).

From the 393 publications included for data extraction, 288 publications focus on comprehensive BC strategic guidelines to apply to all organisations. The other 105 have a focus on BCM components or activities guidelines. Altogether, the publications set can be combined or integrated into a comprehensive set of guidelines to streamline the organisational processes for the BCP design. Nevertheless, we generally considered the guidelines for the proposed artefact from the publications since 2015.

Figure 1 presents the DSR methodology process for FAMMOCN.

Figure 1

DSR methodology process for FAMMOCN.

According to Figure 1, it is possible to see that the DSR guides the iterable process of development of the framework applicable through the Measurement System or the Self-Assessment System, supported by an Implementation Guide. We developed the artefact supported by the literature analysis, structuring a model with BCM components, and defining activities in each component. This way, we gathered a set of strategic guidelines for each activity and proposed them as metrics. For example, some metrics assess design features, and others assess implementation features according to the parent BC activity.

The design and development of the artefact are tested in the evaluation step. We will compare the objectives of a solution to actual observed results from the use of the artefact in the demonstration. The demonstration and evaluation steps used Focus Groups with BC experts in the first iteration. The second iteration was accomplished by conducting semi-structured interviews with organisation professionals that used the framework before, in the organisational context, and during the interview. An informed consent was obtained from all participants in the study.

The communication step started with the problem identification (Russo et al., 2021), and the dissemination of results begins with this paper.

Research question

The research can be systematically conducted if it has validity by using an appropriate process, the findings come from the data, and they answer the research question (Oates, 2006). Therefore, all research projects depend on the research question and available resources (Kitchenham, 2004). As defined in the EBSE approach, the specification of the research question is a critical part of the SLR. It drives the primary studies to include in an SLR, what data must be extracted, and how it is synthesised or aggregated to answer the research questions.

In this context, we formulated the research question to enable a qualitative evaluation with respect to benefits, risks, value, impact or other aspects of adoption (Kitchenham et al., 2015) of the proposed artefact. We design the research question to be used in a qualitative SLR consequentially with less focused research questions. Instead of having many research questions, we chain them all to stress that they are interdependent and should not be interpreted in isolation.

Hence, the research questions whether it is possible:

to support an organisation and streamline its organisational processes,

with the definition of strategic guidelines for implementing a BCP,

which allows the formulation of response, restart, recovery and restoration of business processes, supported by ICT,

at a pre-defined level of operationality,

according to the maturity and capacity of the organisation?

Although the research question is extensive, the research is focused on achieving a set of strategic guidelines to streamline and support an organisation in designing and implementing a BCP. Thereby, the research focus on this part of the research question. The demonstration and evaluation steps of the DSR were specially designed to consider the simulation of use in the context of organisations, collecting data on how they were supported and their organisational processes streamlined.

Thus, the subject of the study relates to the multidisciplinary preparedness of the organisation’s response to achieve a predetermined level of business processes continuity in the face of the various phases of an incident or disruptive event of usual activity.

We split the research question into four sub-questions to streamline the DSR demonstration and evaluation techniques applied:

Do you consider FAMMOCN will support and streamline the organisational processes for implementing a BCP?

Do you consider FAMMOCN will guide an organisation in developing a response, recovery, resume and restoration of business processes, supported by ICT, at a pre-defined level of operationality?

Do you consider that a comprehensive set of strategic guidelines for implementing a BCP, through measurement, is in place with FAMMOCN?

Do you consider FAMMOCN can adapt according to the BC maturity of an organisation?

Table 2 present the set of attributes that we needed to qualify the framework, according to the sub-questions. The “Attribute” column identifies the key concept that will be assessed. The “Description” column complements the attribute and provides some context.

Table 2

Set of attributes for evaluation.

Attribute	Description
Complexity	Concepts with nexus and easy to solve.
Clarity	Intelligible and transparently expressed.
Completeness	Covers the essential issues of a BCMS.
Consistency	A conceptual model that identifies the components and relationships and defines the necessary metrics.
Cohesion	Components, activities and metrics work together holistically to implement a BCP.
Integrability	By alignment with CMMI Maturity concepts.
Alignment	Aligned with relevant standards, frameworks, good practices and regulations.
Scalability	Open and able to grow evenly and support more orientations/metrics.
Dynamic	Considers the impact of changes to maintain the viability of the BCMS.
Feasibility	Potential to be executed, performed or fulfilled in an organisation.
Adaptability	Has the potential to apply to all types and sizes of organisations.
Maintainability	Able to be maintained, improved and updated by a team or individual.
Management support	Allows activities management, aligned with the guidelines defined by Top Management.
Governance support	Considers stakeholder expectations and direction through prioritisation, performance monitoring, and compliance.

As shown in Table 2, some attributes try to capture unique characteristics of how we developed it. Some are focused on the ability to be used and maintained in the medium or long term. There are attributes to assess the capability to adapt and scale to incorporate new areas of interest. These attributes will be used for demonstration and evaluation, therefore they will be presented in context.

Build the framework to solve the problem

Problem identification and motivation

To define the research problem, the Standards were mapped and compared, resulting in a set of areas not formally covered by them. Figure 2 maps the preliminary gaps (red shapes) in the formal definition of activities, in each common component of BCM in the Standards.

Figure 2

BCM components, relevant activities, and gap analysis. Source: adapted from Russo et al. (2021).

Figure 2 represents the Standards in organised activities, objectives or practices, suggesting, in the centre of the circle, the preliminary BC areas or components. Hence, despite the complexity and pertinence of the Standards, it was identified a gap in the definition of BC metrics (Russo et al., 2021). Effectively, the various standards were analysed, to assess the existing gaps in each of them, presented in Figure 2, given the problem under study. In this sense, a framework was developed that, given the reality of organisations, would be effective concerning BC.

Since the scope of the studied Standards differs from one another, they are not comparable with the proposed framework. Instead, they will provide meaningful guidelines for its development. To this end, the proposed framework fills some gaps, considering it is multidisciplinary and aggregating in the problem of developing a BCP.

After reviewing the Standards guidance, it was considered that they are focused on the methods to use, when and by whom, and their guidelines point that organisations must determine what needs to be measured.

In this context, it is considered that there are gaps in the way to assess the capacity or maturity in the activity’s achievement that allows perceiving its level of fulfilment compared to a desirable function for the activity under study, according to the size and capacity of the organisation.

In Table 1, the number of publications addressing measurement issues represents nearly 10% of the SLR studies. This percentage confirms the relevancy of BC measurement. Yet there are few ways of evaluating and measuring how the organisation’s time and resources investment targets the right areas under the BCM components (Green, 2014).

Objectives for the solution

The key objective is to develop and validate a framework that will guide the organisation to what needs to be addressed for the design or implementation of a BCP, with a greater focus on ICT systems. This is proposed to be achieved through the definition of a set of metrics for the multidisciplinary assessment of organisational maturity in the BC area. The specific objectives are:

identify and relate the components and activities of the BCM aiming to identify gaps and achieve a comprehensive framework that allows an overview of the BCP implementation;

identify and outline a set of metrics for each activity of the BCM, to assess the stage of preparation or implementation, the key initiatives guiding the BCM and the formulation of a BCP adequate to the organisation.

Considering the demonstration and evaluation activities, the objectives are:

demonstrate scientific completeness: whether we achieved an answer to the research question and the various BC perspectives were covered by the artefact to solve the identified problem;

evaluate the effectiveness of the proposed artefact use: data analysis of the results of the Focus Groups and interviews.

FAMMOCN – a framework for the multidisciplinary assessment of organisational maturity on business continuity management

The Framework for the Multidisciplinary Assessment of Organisational Maturity on Business Continuity (FAMMOCN) is a set of strategic guidelines that assist an organisation in the BCMS management and implementing a BCP supported by a measurement system.

FAMMOCN applies to organisations that intend to be prepared and respond to incidents or disruptive events. However, also to organisations that seek to improve their BCMS and increase their BC maturity.

Having in consideration the background research, we developed FAMMOCN as a solution to the identified gaps in section 4.1, formed by four components presented in Figure 3:

Figure 3

FAMMOCN structure.

Model: the set of components and activities organised to manage business continuity;

Measurement System: the system that enables strategic guidance defined in the Model through systematic measurement;

Self-Assessment System: the system that enables plain guidance through a self-assessment of selected activities defined in the Model;

Implementation Guide: presents FAMMOCN and its elements, and persuades its adoption and implementation.

Considering the structure of FAMMOCN presented in Figure 3, we outlined a strategy for the various stages of organisational business continuity maturity to address the identified constraints. Thus, applying the Model, FAMMOCN is side-by-side with the organisation in this BC evolution. With the support of the Implementation Guide, an organisation can use the Self-Assessment System in the first steps of the development of a BC response. When the organisation is prepared and adequately conscious and disciplined in BC can use the Measurement System to measure systematically its BC preparedness and response.

Thus, as BC awareness increases, it will also be necessary to increase organisational discipline and improve the underlying BC processes, namely with the PDCA cycle or a BCMS implementation.

FAMMOCN can be used in its basic version, providing the organisation with a Self-Assessment System capable of establishing a starting point for the BCMS implementation. Rapid self-assessment, using FAMMOCN, allows the organisation to visualize its current state and provide internal and external benchmarking. In this way, the objective is for FAMMOCN to support the process of raising the commitment and awareness of Top Management to obtain the necessary investment to establish a suitable BCMS for the organisation.

However, FAMMOCN adjusts and follows the evolution of BC awareness and the need to increase discipline in the processes that guarantee the management of the organisation’s BCMS. In this context, the organisation may choose to apply the full FAMMOCN. Thus, FAMMOCN defines a scalable set of metrics for each activity. Each metric evaluates an initiative that contributes to achieving a level of maturity. Therefore, each metric guides the organisation in what needs to be considered to satisfy the activity objective.

Model

The FAMMOCN’s Model defines the components that comprehend relevant areas in BCM. Thus, the FAMMOCN’s strategic guidelines are grouped into the components that need to be addressed to prepare the BC response to an incident or disaster interruption. The FAMMOCN components and their key objectives are:

Top Management Commitment - transversal to the entire Framework, to the extent that describes the continuous management activities and follow-up to the BCM program. Top Management must demonstrate leadership, commitment and support for the BCMS activities;

Understand the organisation - aims to determine which factors are relevant to the organisation’s mission, which involve the delivery of products and/or services and which affect the expected results of the BCMS. The component considers three domains: Organisation, People, Processes and Technology;

Manage risk - aims to determine risks according to the results of understanding the organisation, assess the impact of risks and opportunities identified and plan risk management according to the defined strategy;

Consolidate the strategy for continuity - aims to define strategies that allow the BC objectives achievement, according to the continuity requirements and available resources;

Plan and structure the continuity response - aims to develop and document the plans and the needed capacity to execute the defined strategy and the BCM program;

Implement and maintain continuity plans - aims to implement the assumptions, actions, solutions and processes necessary to achieve the continuity objectives, according to planning;

Check the continuity management system - aims to ensure that the organisation verifies the adequacy and effectiveness of the BCMS and its requirements;

Improve the continuity management system - aims to ensure that the organisation determines the opportunities for improvement according to the verification performed and implements the actions necessary to achieve the continuity objectives and of the BCMS.

Each component comprises activities that must be developed to achieve the general objectives of the FAMMOCN components. The activities are organised in domains of action for the identification of the activities' context. Thus, each domain groups activities that detail, among others, the actions, tasks, intentions, initiatives, projects, strategies or policies that can be addressed and measured through metrics. Each activity metric defines goals to be achieved. The achievement adds value for an increase in the guarantee of the multidisciplinary preparation of the organisation’s BC response.

Figure 4 presents the components of FAMMOCN. The normal (solid line) and alternative flows (dashed line) identify the path to be followed by the organisation, in the preparation or improvement of organisational processes that may result in an adequate response, recovery, recovery and restoration of business processes. The maturity arrows should show that there is an increase in maturity along with each component, and when following the FAMMOCN in all components.

Figure 4

FAMMOCN model.

FAMMOCN Model in Figure 4 emphasises the identification of the organisational profile as the starting point for the application of FAMMOCN and benchmarking purposes. In FAMMOCN, an essential component is the Top Management Commitment to BC.

Understanding the organisation is vital, as its business processes and information flows as a basis for managing risk and conducting business impact analysis. Consolidating the strategy for continuity, defined by top management with the support of the BC teams, includes a vision for ICT, given the characteristics of the business. Planning and structuring the continuity response covers the documentation of the BCP and the creation of conditions for implementing the BC response. Thus, the organisation can be prepared to implement and maintain continuity plans, implement solutions and conduct exercises, tests and training. Checking the BCMS is important, particularly about how it is being performed. Verification is decisive and preparatory to improving the BCMS, focusing on continuous improvement, corrective processes and change management.

Self-assessment system

In a preparatory phase for continuity and to capture a momentary overview, the organisation can apply the FAMMOCN to self-assess its preparation in BC. Through the Self-Assessment System support application, exemplified in Figure 5, the organisation can establish a starting point for implementing a BCP, measuring its current state of multidisciplinary preparation in the area of BC, in an agile way. At this stage, essential activities are considered in each component, reporting their compliance through written questions in a simplified and direct way.

Figure 5

Self-Assessment System support application.

The organisation answers directly to self-assessment questions in selected activities for each FAMMOCN component, like the example presented in Figure 5. There may be several questions for a single activity. The total score achieved will be the sum of the weighted scores of each component, which reflects compliance with FAMMOCN.

Measurement system

FAMMOCN has an underlying measurement system that incorporates a set of metrics. It allows assessing the stage of preparation, implementation, maintenance, review and continuous improvement of the essential elements which guide the BCM and the design of a BCP adapted to an organisation. The measurement system allows the perception of the current level of maturity in BC of the organisation. Each metric aims to guide the organisation in what needs to be considered to meet the parent’s activity objective.

Figure 6 presents the Measurement System underlying the FAMMOCN. The organisational maturity in BC is evaluated considering the weighting of each component. Each component has a weighted set of activities allowing adjusting the organisational effort modularly by component. An activity has a weighted set of metrics, and each metric has an associated score and maturity level.

Figure 6

FAMMOCN measurement system.

Figure 6 reveals that each metric has a defined structure with a grouping of attributes that characterise the metric and what should be done by the organisation. The green colour on the scorecard shows that the goal has been achieved, ranging between 90% and 100%, and no relevant action is required. The yellow colour represents values of achievement between 60% and 90%, and it is important to understand the evolution. The red colour shows a below 60% performance and implies the need for intervention.

Maturity levels are based on the implicit definition of the CMMI. A metric refers to a maturity level that is reached when its score is equal to or greater than 90%. When all metrics of a given maturity level and activity score is higher than 90%, then the maturity level of the activity is reached. However, as mentioned in Figure 6, a FAMMOCN metric defines several attributes classified as shown in Table 3.

Table 3

Group of attributes in the metrics structure.

Group of attributes	Description
Identification	Identification of the metric, including the weighting of the metric in the activity.
Characterisation	Objective or type of measure.
Self-Assessment	Self-assessment questions and guidance on what should be achieved in each colour of the scorecard.
Quantification	What is being quantified, the calculation formula and the goal to be achieved.
Concretisation and level	Current state of completion (score), the associated maturity level and the priority in implementation.
Responsabilisation	Register those responsible for defining, influencing, or measuring.
Frequency	The frequency at which measurement, reporting, analysis, or review is required.
Evidenciation	Additional attributes that, for example, record the source of evidence or data.

In every attribute defined, there can be registered relevant information that can guide the organisation in fulfilling the objective of the metric, streamlining the organisation processes in the BC scope.

Table 4 presents the structure of the “Manage risk” component and the weight of each activity. We selected this component since it was represented in Figure 5 as part of the Self-Assessment System.

Table 4

Structure of the component "Manage risk".

Domain	Weight	Activity
Governance	0.05	Develop a risk or opportunity management strategy.
	0.10	Develop risk or opportunity management plans.
	0.03	Manage risks or opportunities by implementing planned activities and ensuring continuous consultation with stakeholders.
	0.02	Identify constraints, priorities and compensations in risk management.
	0.03	Design products and services to address prioritized risks.
	0.02	Incorporate and demonstrate appropriate risk management culture and behaviours.
	0.05	Communicate and report risk within the organisation and to stakeholders.
Risk management	0.02	Identify, document, assess and monitor uncertainties, threats and vulnerabilities.
	0.10	Identify and document risks and opportunities.
	0.03	Identify categories of risk or opportunity.
	0.08	Analyse risks and opportunities.
	0.07	Assess, monitor and communicate risk.
	0.05	Assess and monitor risk management.
	0.10	Address risk by planning appropriate risk responses.
	0.03	Evaluate alternative courses of action to respond to the risk.
	0.02	Review the risk assessment process.
Business impact analysis (BIA)	0.10	Conduct Business Impact Analysis (BIA) and assess and estimate the probability, impact and proximity of risks, prioritize risks and understand risk exposure.
	0.03	Identify in the BIA the types of resources, activities and processes necessary for the organisation to fulfil its mission.
	0.02	Evaluate the BIA.
	0.02	Monitor the probability and severity of risks occurring.
	0.03	Review the BIA.

As a summary of Table 4, it is emphasized that the organisation must develop risk management strategies and plan its activity to cope with these risks. Risks must be analysed, evaluated for their impact on the business and treated with the appropriate response that ensures continuity and predefined readiness.

Considering the activities presented in Table 4, we select one example metric used in the FAMMOCN Measurement system. Table 5 shows a metric in the “Manage risk” component of the FAMMOCN Measurement System, which allows evaluating, in detail, the Self-Assessment question 13 referred to in Figure 5.

Table 5

Metric “Identify and document risks and opportunities”.

Attribute	Description
Identifier	MR1
BC component	Manage risk
BC component activity	Identify and document risks and opportunities.
Metric weighting in activity	50%
Metric designation	Identify and document risks.
Information purpose	The organisation must identify the potential risks to which it is subject to be able to address the risks or mitigate the impact of their occurrence.
Measurement type	Efficiency
Measurement period	Yearly
Metric objects	1. Number of identified risks. 2. Number of identified risks that have been described and documented.
Formula	(Number of identified risks that were described and documented/Number of identified risks)∗100
Metric goal	100%
Metric critical value	95%
Metric scorecard	Red: [0.60]; Yellow: [60.90]; Green: [90,100]
Maturity level	1
Priority	Must have
Activity owner	Risk Management Manager
Responsible for the metric	Risk Management Manager
User responsible for measurement	Manager’s Secretariat
Measurement frequency	Monthly
Analysis frequency	Quarterly
Communication frequency	Quarterly
Review frequency	Yearly
Evidence of achievement	1. Document with identified risks 2. Documents with described and documented risks
Source of data	Agenda or minutes of meetings with Risk Managers about the identification, description and documentation of risks. Documents with described and documented risks.
Communication format	2D Pie Chart. A summary of issues limiting achievement should be attached.
Comments	The Manager’s secretariat may have to collect information from the various departmental risk management officers

All attributes identified in Table 5 are mandatory, except the “Comments” attribute. Each attribute can define and describe relevant information that guides the organisation in fulfilling the objective of the metric, streamlining the organisational processes within BC. This streamlining is achieved by reducing the need to consult external sources of knowledge, describing what should be done and how it should be evaluated, reviewed and communicated.

Implementation guide

In the demonstration step of the study, the experts mentioned the need to create a document to guide the framework’s implementation and its adoption by the organisation. Thus, an Implementation Guide was developed, presenting the framework, its components and activities, and the advantages of its adoption. The implementation guide comprises the following sections:

About FAMMOCN

Overview: presents FAMMOCN and its scientific validity;

Objective: presents the objectives of the framework;

Why FAMMOCN: presents the reasons for the adoption of the framework;

Target audience: identifies the organisations that could benefit from the adoption of the framework and the various stages of implementation;

Guiding the organisation: presents the general process that allows guiding the organisation with FAMMOCN.

Structure

Components, domains and activities: presents the constituent parts of the FAMMOCN Model;

Measurement System: presents the system that allows the measurement of the implementation of FAMMOCN, the structure of a metric, the scorecard and maturity levels;

Self-Assessment System: presents the system that allows self-assessment by following FAMMOCN.

Annexe

Components and Activities: describes each of the components and activities of FAMMOCN. Presents the questions of the Self-assessment System and example metrics of the Measurement System.

FAMMOCN demonstration and evaluation

The demonstration activity implies using the artefact to solve one or more instances of the problem (Peffers et al., 2007) and may involve its use in proof-of-concept. The effective knowledge of how to use the FAMMOCN to solve the problem was demonstrated in the Focus Group session and the semi-structured interviews. We adapted the proof-of-concept prototype of FAMMOCN according to the exploratory Focus Group feedback.

The DSR concentrates on the practical relevance and pragmatic validity of a generic design (Van Aken, Chandrasekaran and Halman, 2016). As a guideline for design evaluation, the utility, quality, and efficacy of a design artefact must be rigorously demonstrated via well-executed evaluation methods (Hevner et al., 2004). Conceptually, such evaluation could include any appropriate empirical evidence or logical proof. We conducted the Focus Group session to evaluate the FAMMOCN Model and the Measurement System, not only by the ability to answer the research question but also through the attributes presented in Table 2. The semi-structured interview’s key objective was to evaluate the Implementation Guide and the Self-Assessment System developed in the DSR process iteration.

First iteration – Focus Group

The study used Focus Group in the first iteration of the DSR methodology process. The study follows the Focus Group steps defined by Tremblay et al. (2010) as presented in Figure 7.

Figure 7

Focus Group steps. Source: Tremblay et al. (2010).

The research problem was already formulated, as presented in section 3. Therefore, the next activity shown in Figure 7 is the definition of the number of Focus Groups. Tremblay et al. (2010) suggest the use of a small group for greater participation from each member. The group size was restricted to 5 participants. There have been constraints in scheduling with experts, then the number of Focus groups for the proof-of-concept was limited. The participants were chosen from different fields of expertise to get contributions, among others, in ICT management, ICT governance, risk management, cybersecurity, and continuity planning and solutions.

In the next step, the script of the Focus Group was designed. The research problem and motivation, the question and objectives were first on the agenda of the Focus Group. Second, we presented an overview of the FAMMOCN, its process of the multidisciplinary assessment of organisational maturity on business continuity and the levels of maturity. The FAMMOCN components and activities were explained in some detail. The metric structure of FAMMOCN and some examples of metrics created complete the presentation.

The participants discussed if the information in the example metrics streamlines BC organisational process and provides strategic BC guidance. To help moderation and focus, it was presented a survey with dichotomic questions for a set of attributes, like clarity, completeness or adaptability. Finally, the research question was decomposed into smaller related questions to refine the understanding of the artefact to answer the problem from the expert’s point of view.

The experts have more than 20 years of experience in ICT Governance, Security and Governance Solutions, ICT Management in the Government Sector, Risk Management in the Banking sector and Cybersecurity areas. The experts agreed to take part in the 2-hour videoconference Focus Group moderated by the first author. The experience of these experts provided a conceptual understanding of applicable governance, organisational constraints and BCM self-assessment need. Other sets of required developments were gathered to improve the answer to the identified research problem, especially those related to strategic guidelines and process streamlining.

The conduction of the Focus Group with the experts followed the script. We presented the FAMMOCN prototype and collected data. The survey was provided by using the videoconference tool capabilities, and its data was used in the analysis. The transcription of the session was coded and used for data analysis and interpretation.

Second iteration – semi-structured interview

Essentially, the evaluation activity step, defined by Peffers et al. (2007), covered three stages including:

Providing, in the email invitation, the FAMMOCN supporting documentation and the Self-Assessment System to be used in advance;

In a videoconference session, review the documentation and fill some example metrics in the FAMMOCN Measurement System application tool;

A 90 min semi-structured interview with an integrated survey, included in the videoconference session.

According to the results of the first iteration, the Implementation Guide and the FAMMOCN Self-Assessment System were developed. These documents and a short leaflet presenting FAMMOCN were attached to the email invite. The email was sent weeks in advance and had the questions and the survey to be used in the semi-structured interview.

Simulation and interview

The purpose of the semi-structured interview was to obtain data and grounded evidence to evaluate the artefact’s utility, quality, and efficacy in solving the problem defined. The evidence was obtained by simulating the use of the artefact and exploring and analysing the professional’s contributions, perspectives and interpretations, supported in the semi-structured interview.

The guidelines provided by Tremblay et al. (2010) and Kvale (2007) for designing interviews were followed. The organisations were selected to represent different business sectors and sizes.

The conduction of the semi-structured interview was supported by a non-recorded videoconference and followed an interview script. First, the research was introduced and clarified potential issues about the documentation sent by email. Second, a survey for organisation and participant characterisation was completed. Third, the Self-Assessment System sent by email was analysed and reviewed the answers given. Fourth, the Measurement System was reviewed and exemplified by asking the professional to fill some metrics in the Measurement System application tool. These two last tasks enable a partial assessment of the quality, utility and effectiveness of the artefact. In pre-defined moments, some questions were introduced, identical to the questions used in the demonstration step, to provide comparability and consistency. Therefore, the research question was divided into four underlying questions and the attributes mentioned in section 3.2 were evaluated during the interview.

The FAMMOCN was evaluated by ten medium-sized and large-sized organisations. They are national and multinational organisations in different and relevant business sectors to ensure the scope of FAMMOCN’s applicability. The organisations are in the central and local public government sector, banking sector, transport and logistics sector, retail, industrial sector and the ICT business and consulting sector. Each organisation appointed one or two professionals to participate in the study. There are scheduling constraints with smaller organisations because of the scarcity of resources and their professional availability.

The professionals who have validated the FAMMOCN are managers, senior managers and top managers in their organisations, with over 20 years of experience in the ICT area. It is relevant to highlight that most of the professionals have implemented and coordinated BC plans and solutions.

Results and discussion

The demonstration and evaluation steps of the research achieved the objectives and were successfully carried out. However, it is complex to find organisations available to participate in the studies, especially when they may reveal weaknesses or gaps in the BC area. Organisations do not intend to disclose their preparedness to deal with incidents. One measure found to mitigate this risk was not recording the interviews during the evaluation phase. Another measure was to ensure the anonymity of the organisations and professionals who participated in the study. With these guarantees, some limitations in the scheduling of interviews were overcome.

In this context, the evaluation step was carried out using a semi-structured interview with a simulation of the use of the FAMMOCN Measurement System, replacing the use of a case study. However, how the evaluation step was designed made it possible to obtain equally relevant results. By sending the documentation and the FAMMOCN Self-assessment System for use and filling in advance, the organisation tested and used FAMMOCN according to its availability.

The organisations used in the study differ from each other, with distinct dimensions and business sectors. In this context, FAMMOCN proved to be clear, flexible and scalable, consistent and complete and aligned with other Standards, used by organisations and professionals in the BC area.

The results of the demonstration and evaluation steps show that FAMMOCN allows to:

Support the organisation in implementing a BCM;

Streamline its organisational processes in the identification of priorities in performance, human resources, financial and ICT, on the necessary documentation and other elements that influence business processes;

Strategically guide the implementation of a BCP and other plans;

Design value-added plans to ensure the response, recovery, resume and restoration of business processes, supported by ICT;

Identify and implement a pre-defined level of operationality according to the organisation’s maturity and capacity.

FAMMOCN provides the necessary information that guides the organisation to the effective use of resources and in the selection of strategies that best adapt to its organisational capacity. FAMMOCN supported the Top Management commitment and awareness, necessary for starting and establishing a BCMS, appropriate to the organisation.

The organisation can measure in detail the success of its action, and identify gaps, deficiencies, or needs to achieve the objectives of each activity. This can streamline the availability of support or resources needed at the right time. It allows guiding the organisation in obtaining an improvement in organisational processes, which results in an adequate response, recovery, resume and restoration of business processes, in the face of a disruptive incident or event.

Focus Group data analysis and results

We examined Focus Group data for the meanings and implications of the research question (Tremblay et al., 2010). We analysed the transcriptions for common themes and variations. It provided rich descriptions of the participant’s reactions to the design features of FAMMOCN. The analysis explored the answers and the reactions as inputs for a strategic re-design of the prototype.

In Table 6, there is a representative example of the analysis performed, with interactions and added value for iterations to design. The results presented in Table 6 translate the expert perspectives for improvement and integration into the process iteration. Threats that we need to address, although essentially outside the FAMMOCN scope. The column “Treatment” describes what was re-design as an iteration result.

Table 6

Focus Groups data analysis of a sub-question.

Improvement	Treatment
In “crisis mode”, there are 10–20 controls that assure the continuity wanted.	Reviewed the priority of metrics for the essential activities. Reviewed level one of maturity for essential continuity actions.
A quick assessment that results in a score and allows benchmarking is relevant to justify the BC.	Design and integration of the Self-Assessment system.
Threats	Treatment
A high amount of metrics may not speed up the implementation of the framework, especially in small organisations.	Revised the priority and level of maturity in metrics. Balanced the number of metrics per activity.
The more parameterisation on the metrics, the more complicated their implementation will be.	The metrics and measurement system are configured only on each FAMMOCN release.
Training people is relevant for framework maintenance.	The Implementation Guide will include relevant information for maintenance. Reviewed the source of evidence and responsibility for measurement information of each metric.

The survey applied in the discussion of FAMMOCN evaluates the attributes presented in Table 2. The analysis information of the survey is presented in Table 7. The “Grade” column represents the percentage of participants that confirms the attribute description. The “Key contributions and perspectives” column presents the main reactions of the participants to the attribute.

Table 7

Set of assessed attributes.

Attribute	Grade	Key contributions and perspectives
Complexity	100%	Define the organisation’s priorities, for example, in an implementation guide.
Clarity	100%	Even someone who does not master the BC area can understand it.
Completeness	100%	Alternative flows allow the organisation to create a BCP, even if it does not intend to do a full implementation, with verification and improvement.
Consistency	100%	It is transversal and involves processes in several areas.
Cohesion	100%	The metrics objective guides what should be addressed.
Integrability	100%	Considering the FAMMOCN Model, it is simple to see whether the organisation is less mature and has to approach some components.
Alignment	100%	A scoring system allows benchmarking between organisations to obtain recognition for compliance and enhance the adoption of the framework.
Scalability	100%	Suggested a closed universe of metrics, only including other metrics in new versions.
Dynamic	60%	An organisation may decide to give up the measurement of some metrics at the expense of others with a higher weighted value.
Feasibility	100%	Smaller organisations may have difficulty in complying with higher levels of maturity due to a lack of available human resources or the additional work underlying the use of the framework.
Adaptability	100%	The metrics are direct and can be modular. This allows an adaptation to my reality and what I need.
Maintainability	100%	The framework should be integrated into a recognised Standard to be maintained by a community.
Management Support	80%	It allows following the BCP implementation and the evolution of the BC maturity level reached.
Governance Support	80%	- An implementation guide will allow the organisation to define its path to continuity. - Each metric should have a question of self-assessment that summarised its purpose, using a score.

The contributions achieved from attribute “scalability” presented in Table 7 resulted in delivering the FAMMOCN with metrics applicable to all organisations. This is obtained by avoiding metrics for technologies, business sectors or other specific characteristics not commonly applied to all organisations.

The contributions achieved from attribute “complexity” and management and governance support resulted in the development of the Implementation Guide, and the Self-Assessment System development. Altogether, they address the issues of providing an understanding of how to use the FAMMOCN and a quick self-assessment that allows internal and external benchmarking. The Self-Assessment System collects information about the organisational profile to allow benchmarking. The number of metrics per activity was revised to balance complexity and effectiveness. We revised the maturity level of each metric, complying with CMMI guidance but ensuring a BC solution for all organisation’s dimensions.

The contributions of the “dynamic” attribute were considered in an internal research team debate on if it was advisable to hide the weighting of the Measurement and the Self-Assessment System. We decided to maintain the weight visible to enhance transparency. The number of metrics per activity was balanced between added value and effectiveness.

This section presented improvements introduced in the FAMMOCN through the knowledge of the Focus Group experts. We intended to present some reactions and statements resulting from the Focus Group session, which demonstrated that the framework can solve the identified problem. The metrics capabilities of the FAMMOCN were recognised by the experts to be relevant to any organisation. Some reactions are listed here:

Metrics that allow a quick assessment are important to support the justification, to the top management, for the investment in BC;

The framework is important for any organisation. Knowing the weight of each metric is relevant because objectively, there are things of different importance;

We get a perspective of where we are in BC preparedness;

The amount of metrics is phased and some arise from the measurement of the system itself;

The framework supports and streamlines organisational processes, and especially for top management, it will give a score overview that helps decision-making.

After the FAMMOCN prototype development and the Focus Group session, we adapted the artefact to integrate contributions and perspectives to be used in organisations. We developed the Implementation Guide and the Self-Assessment System according to the Focus Group results. The researchers developed application tools for the Measurement and Self-Assessment Systems. After these developments, we were prepared for the next iteration.

Interviews data analysis and results

Sub-section 4.4.2 presented the simulation and the interview. This sub-section presents a summary of the reactions to the simulation and the answers and contributions given during the interview. The reactions were obtained after the simulation with the Measurement System and the filling of the Self-Assessment System applications tools.

The research question was decomposed into four underlying questions, as presented in Table 8, to enhance the focus of the interview. The key contributions and perspectives of the participants are also presented.

Table 8

Interview questions about the research question.

Underlying research question	% of validating answers	Key contributions and perspectives
According to your experience and knowledge, do you believe FAMMOCN can provide added value in supporting and streamlining organisational processes for the implementation of a BCP?	100%	- Defines priorities on what to address first. -Guides to the relevant processes and to focus on a set of important intentions.
Can FAMMOCN assemble a comprehensive set of strategic guidelines for implementing a BCP through metrics?	100%	-A metric is worth a thousand words.
Do you believe FAMMOCN can guide an organisation in formulating a response, recovery, resume and restoration of ICT-supported business processes at a pre-defined level of operationality?	100%	-Gives guidance on what should be done.
Given what was presented at this session, do you believe FAMMOCN can adapt according to the BC maturity of an organisation?	100%	-It allows for measuring lower levels of maturity and in later phases, it can target higher levels.

From Table 8, the perspectives mentioned the guidance provided to implement a BCMS. Another common perspective from using FAMMOCN is to know where the organisation is in terms of the implemented arrangements and defined processes within the scope of BC. The organisations also mentioned that FAMMOCN is adaptable to their context and helps to find gaps in the BCMS.

In Table 9, there are questions about the utility of FAMMOCN, since the research must produce an artefact created to address a problem (Peffers et al., 2007).

Table 9

Interview questions related to the utility of FAMMOCN.

Utility related question	% of validating answers	Key contributions and perspectives
Do you believe that the eventual effort of the organisation with the implementation of FAMMOCN is mitigated by the streamlining of organisational processes in the BC scope, achieved through its use?	100%	-If the measurement effort can be shared by many. -Constraints are mitigated by the definition of those responsible. -By managing efforts on the priority issues.
Do you believe that the FAMMOCN Implementation Guide has supported and/or streamlined its implementation in the organisation?	100%	-Complete in what needs implementation. -Simple and straightforward. -Yes, the BCP implementation is simplified.
How do you characterise the level of complexity underlying the use of FAMMOCN, concerning the frameworks that you coordinated, implemented, coordinated/implemented, or that you are more familiar with?	100%	-The use is simple, without great costs. -Focused on implementation and close to the organisation’s reality.
Does it believe that the FAMMOCN Self-Assessment System provides a guiding and strategic approach to the implementation of a BCP?	100%	-When filling in the metrics guidelines emerge.

Table 9 reveals the utility of FAMMOCN, for example, by allowing the measurement effort to be distributed by many. Therefore, most of the utility of the framework arises after the mitigation of measurement effort by distribution and assignment of responsibility for measurement. Another relevant perspective is that FAMMOCN helps the organisation in schematising how the processes are established and what the gaps are. Since the evaluation of the utility provided for solving the problem (Hevner et al., 2004) isn’t justified by the information presented in Table 9, there is a need to continue to evaluate the quality and efficacy of FAMMOCN.

Thus, quality was partly confirmed by the demonstration step by the attributes presented in Table 7. Therefore, after the organisations use the artefact, the questions were asked openly to confirm the demonstration results. Table 10 presents some perspectives of the organisations.

Table 10

Interview questions related to the quality of FAMMOCN.

Quality related question	% of validating answers	Key contributions and perspectives
From your perspective, what advantages will the use of FAMMOCN have in an organisation?	100%	-Guidance for implementation. -Visualise and identify areas to improve. -Measurement responsibility is shared by many. -Focus the organisational effort. -Raise awareness of the Top Management. -Risk awareness. Improve responsiveness. -Modularity, ease and intuitive.
What difficulties/limitations/constraints did you feel or expect underlying the use of FAMMOCN?	100%	-First phase implementation. -Sufficient human resources. -Top Management commitment and awareness. -Basic training and appropriate culture of employees. -Organisational resistance to change. -Organisational culture and awareness gap to implement BCM beyond the strict domain of ICT disaster recovery.
What characteristics of FAMMOCN do you consider most relevant?	100%	-Measurement of the implementation. -Presents the needs and the weakest points to address. -Evaluation process and descriptive metrics. -Metric-oriented, objective and modular/flexible allows adaptation to reality, needs and maturity.

In Tables 8, 9 and 10, the organisations confirm the functionality, completeness, consistency, accuracy, performance, reliability, usability, and fit with the organisation defined by Hevner et al. (2004).

The last set of questions of the interview is presented in Table 11. The questions had the focus on the measurement systems since they are the endpoint artefacts that can be evaluated for efficacy.

Table 11

Interview questions related to the efficacy of FAMMOCN.

Efficacy related question	% of validating answers	Key contributions and perspectives
Do you believe that the use of the FAMMOCN Self-Assessment System may be relevant for the organisation to adopt a BCMS or for the implementation of a BCP?	100%	Organisations may realise that aren’t prepared and need to do something.
Do you believe FAMMOCN, with its Measurement System, can be a driver of the adoption of a BCMS in the organisation or for the implementation of a BCP?	100%	Metrics give numbers, which are easier to interpret, on what needs to be done.
The FAMMOCN Measurement and Self-Assessment Systems contain metrics or questions at higher levels of multidisciplinary maturity in BC, which are assumed not to fully apply to some dimensions or types of organisations. Do you believe this assumption does not reduce the effectiveness of using FAMMOCN?	100%	In organisations with a low level of maturity, simple and concise proposals ease results.

The perspectives included in Table 11 help to identify the efficacy of FAMMOCN in adopting a BCMS or being itself the BCMS used by the organisation.

In the demonstration step, there were some attributes, presented in Table 7, which had a lesser grade. The design iteration tried to resolve the identified issues, producing an improved artefact. Table 12 presents these attributes and the reactions of the participants to each of them. It should be noted that the iterations in the design produced the Implementation Guide, the FAMMOCN Self-assessment System, and two application tools that enable the use of the FAMMOCN Measurement System and the Self-assessment System. Other improvements were integrated, such as the revision of the number of metrics, or including metrics on change management.

Table 12

Attributes improvement in FAMMOCN according to the demonstration step.

Attribute	Grade	Key contributions and perspectives
Governance Support	100%	-Defined structure and processes, a direction and a sequence of steps. -Tackle the weak points through measurement.
Management Support	100%	-Allows an overview, relatively simply, about what the organisation needs to do. -Rich and important metrics for management. -Descriptive, explanatory. -Complete. Guides on how to implement the BCP.
Dynamic	100%	-Change management is embedded.

As shown in Table 12, all attributes were positively evaluated by all participants of the organisations interviewed. The nature of the research venue may dictate whether such iteration is feasible (Peffers et al., 2007). At the end of the evaluation activity, the researchers stopped iterations to design because of the minor level of improvements to the effectiveness of the artefact. The study will continue to the communication step and leave further improvement for future work.

Conclusions

Considering the identified gaps in the mentioned Standards and the requirements of the research question, we proposed a framework based on the RSL results. This basis allowed us to develop a framework guided by the objectives for the solution, which include the demonstration and evaluation activities.

Bearing in mind the DSR methodology process for FAMMOCN, the objective of this paper was to communicate the evaluation of the framework for the multidisciplinary assessment of organisational maturity on business continuity. Its four components are the parts of the artefact developed according to the DSR methodology and supported by an SLR and Standards analysis. It was demonstrated and evaluated by BC experts and ICT professionals using Focus Group and semi-structured interviews.

We consider that we answered the research question affirmatively. The results confirm that the framework is not complex and is feasible and maintainable, supporting the organisation in formulating a response, restart, recovery and restoration of business processes supported by ICT, at a pre-defined level of operationality. We also confirmed that it is consistent, complete and enhances management by streamlining organisational processes. Moreover, it supports governance and is aligned with Standards, providing the strategic guidelines to implement a BCP. Yet it is scalable and adaptable to several types of organisations in distinct business sectors and BC pervasion stages. The Model is clear and cohesive, transmitting the path the organisation must walk to achieve a (better) BCP and BC preparedness. The Measurement and Self-Assessment systems contribute to applying the Model with the support of the Implementation Guide.

As mentioned in the discussion, there are constraints in adopting a framework that can consume time and resources, highlighting the difficulties of justifying BC investment to the Top Management when no disrupted events occur or the probability of occurrence is blurred. Being this out of the control of the framework, even with the uneasiness of this study to solve the problem, there is the need to raise awareness of the thematic and disseminate the framework’s adoption.

Therefore, the last step of the DSR defines the communication of research and especially for this moment of research, by communicating its effectiveness to researchers and practitioners. This was one objective of this paper and will continue with other publications that disseminate the knowledge resulting from this study.

For future work, we are willing to integrate the FAMMOCN’s Measurement and Self-assessment Systems into an online platform. The objective is to make available to a pilot set of organisations and refine the metrics and other elements of FAMMOCN. These systems will allow defined alerts and improve the interaction between the parties involved in the measurement and the faster visualisation of results.

Declarations

Author contribution statement

N. Russo: conceived and designed the experiments; performed the experiments; analyzed and interpreted the data; contributed reagents, materials, analysis tools or data; wrote the paper.

H. São Mamede: conceived and designed the experiments; analyzed and interpreted the data; contributed reagents, materials, analysis tools or data.

L. Reis: conceived and designed the experiments; analyzed and interpreted the data; contributed reagents, materials, analysis tools or data.

C. Silveira: conceived and designed the experiments; analyzed and interpreted the data; contributed reagents, materials, analysis tools or data.

Funding statement

This work was supported by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia, within project LA/P/0063/2020.

Data availability statement

No data was used for the research described in the article.

Declaration of interest’s statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.