The COMET Sleep Research Platform.

INTRODUCTION: The Comparative Outcomes Management with Electronic Data Technology (COMET) platform is extensible and designed for facilitating multicenter electronic clinical research.
BACKGROUND: Our research goals were the following: (1) to conduct a comparative effectiveness trial (CET) for two obstructive sleep apnea treatments-positive airway pressure versus oral appliance therapy; and (2) to establish a new electronic network infrastructure that would support this study and other clinical research studies. DISCUSSION: The COMET platform was created to satisfy the needs of CET with a focus on creating a platform that provides comprehensive toolsets, multisite collaboration, and end-to-end data management. The platform also provides medical researchers the ability to visualize and interpret data using business intelligence (BI) tools.
CONCLUSION: COMET is a research platform that is scalable and extensible, and which, in a future version, can accommodate big data sets and enable efficient and effective research across multiple studies and medical specialties. The COMET platform components were designed for an eventual move to a cloud computing infrastructure that enhances sustainability, overall cost effectiveness, and return on investment.

Introduction

Health care is currently experiencing a major shift in the way data are collected and utilized. With advances in technology, the opportunity for clinicians, allied health professionals, researchers, and patients to collect health-related information has grown exponentially. The key is to confirm that these data are converted to meaningful outcomes. To properly identify which outcomes are the most significant, greater importance is being placed on obtaining input from numerous individuals with diverse areas of expertise and perspectives. Similarly, we felt that input from a broad base of stakeholders would be the best way to objectively determine which features or requirements were most important for our primary function: designing an informatics platform to support clinical research.

This manuscript comprehensively describes the Comparative Outcomes Management with Electronic Data Technology (COMET) platform, including its three layers and two subsystems. Multiple examples are provided with data from a comparative effectiveness trial (CET); however, it should be stressed that the platform is designed to be repurposed within other research disciplines. An overview of concepts are provided that can be adapted by project managers for other intentions, with sufficient details for technical implementation by informatics experts.

Background Analysis for Platform Development

In a requirements analysis performed with input from over 40 stakeholders, including principal investigators, research staff, biostatisticians, clinicians, and informaticians, similar components were found to be desired across most clinical trials. A total of 18 essential components (listed in Table 1) were identified to address the technical, clinical, and research needs for an informatics platform and became the conceptual foundation for the COMET research platform.

Table 1.

List of 18 COMET Components

1	Single-Point Access
2	Ontology(s)
3	Query Tool(s)
4	Query(s), Result(s) and Iteration(s)
5	Content Request Initiation Process(es)
6	Federated Query and Data Mapping(s), Extract(s)
7	Administration Data Model(s) and COMET Registration(s)
8	Authorized Access, Permission Role(s) Matrix
9	HIPAA Compliance (IRB) and Codebook(s)
10	Authorized Use, Permission Role(s) Matrix
11	Content: Sleep and Sleep-related, Clinical and Research Content
12	Clinical Evaluation Questionnaire (CEQ) (ASQ)
13	Integrated Clinical Database(s) (I-CLINICAL)
14	Integrated Research Database(s) (I-RESEARCH)
15	Research Project Database(s), Portal, and Other
16	Content Management System(s) (CMS) (DMS)
17	BioBank Registry(s) and Physical Specimen(s)
18	Content Request(s), Transfer(s) and Fulfillment Process(es)

Note: HIPAA—Health Insurance Portability and Accountability Act; IRB—Institutional Review Board; ASQ—Alliance Sleep Questionnaire; DMS—Document Management System.

The COMET sleep research platform development was funded with a grant provided by the Agency for Healthcare Research and Quality (AHRQ) with a primary goal of creating a new platform for the effective use of electronic data in comparative effectiveness research (CER).1

The secondary goal of the AHRQ project was to complete a CER trial to test the efficacy of the newly developed platform. Thus, CET, a randomized multicenter research study was implemented to compare the effectiveness of two treatments for overweight, hypertensive obstructive sleep apnea patients: positive airway pressure (PAP) versus oral appliance (OA) therapy. Four sites were involved in CET including Stanford University, Harvard University, the University of Pennsylvania, and the University of Wisconsin–Madison. Medical specialties represented included psychiatry, neurology, pulmonology, cardiology, and sleep.

During the early development phase for the platform, a new team of stakeholders—including principal investigators, research staff, clinicians, nurses, project managers, and developers—independently identified requirements that were necessary to perform CET. This was a crucial test to confirm that the conceptual development roadmap using the 18 components enabled all necessary requirements. Upon review, it was determined that implementation of the 18 COMET components would support all the desired CET requirements, and that agile software methodology development (see “Agile software development” in the COMET Platform Layers section; also see “Agile Methodology” in Appendix D: Glossary of Terms) should begin immediately following this framework.

Currently, the COMET sleep research platform is a standardized platform for collecting, organizing, storing, analyzing, presenting, and sharing research data from multicenter clinical research trials. This platform was designed with a focus on stability, standardization, and scalability so that both smaller and larger projects can be accommodated. Close attention to cost effectiveness, ease of use for researchers, reusability by other research projects, and governance were also prioritized. This platform is further enhanced by applying business intelligence (BI)2–5 to the COMET project, which are approaches first tested in the business environment that can be applied to transform raw data sets into clinically useful information.

This system has the potential to improve health care by providing a more efficient method for integrating clinical research projects across multiple sites and multiple research disciplines. Rather than repurposing existing research platforms to fit new research needs, or designing whole new research platforms each time a study is funded, we designed COMET to provide an infrastructure that was flexible enough to accommodate diverse multicenter clinical research studies. Referencing a common expandable toolset creates a foundation for rapidly building and deploying varying research studies, thus reducing overall costs and increasing return on investment (ROI) by providing access to quality electronic data in a timely manner for analytic purposes. With regard to the future of hosted systems, the intent for the COMET platform is to facilitate a move of the platform or component parts of the platform to cloud computing services in future versions of the platform.

Developing the COMET Platform

Several technology platforms were evaluated to identify the best fit for the COMET platform based on input from numerous stakeholders on what requirements are most necessary to perform research studies. Ultimately, the team chose a broad family of servers and other technologies that are interoperable to allow selection of only the components required for a particular research project. Not all COMET components need be selected for every trial.

The Microsoft family of servers and development products were selected for the COMET infrastructure in order to minimize software development costs, to allow for the integration of products, and to have the future ability to move to cloud computing. Cloud computing is the practice of using multiple remote servers hosted on the Internet to store and process data instead of local servers or a personal computer. To facilitate the latter design goal, the COMET platform comprises three technology layers that map to the cloud computing service model6,7 (1) Infrastructure Layer, (2) Server Software Layer, and (3) Application Software Layer (Figure 1). The characteristics of the COMET design are outlined in the following section in addition to the technology solutions that were ultimately selected.

Figure 1.

COMET Platform Layers

The Infrastructure Layer

The infrastructure layer provides the foundation (hardware, network, and storage) for the platform, similarly to how the foundation of a house is built using a concrete slab and footings. Desired features of the platform include reliability, security, scalability, extensibility, and sustainability—features that are comparable to features of many informatics infrastructures. These features produce value by delivering a dependable, industrial-strength infrastructure that optimizes computing performance and has the capacity to accommodate future growth. Currently, the COMET platform is located at a secure data center at Stanford University, which enhances our network security. The system includes three Dell rack-mount host computers with a dozen virtual machines (VMs). For a more detailed view of the COMET platform, see Appendix B.

Server Software Layer

The server software layer includes all the computer programs that are installed on the infrastructure layer. The elements used in the infrastructure layer have impacts on the overall computer performance metrics, including software application response time. The COMET infrastructure is running Microsoft Windows Server 2008 R2 and 2012 as server operating systems (OS) on which all of the server software applications run as listed in Table 2.

Table 2.

Server Software Applications: Versions, Features, and Functions

Server Applications: Features and Functions
Microsoft SharePoint Server 2010, Enterprise Version  Used with Microsoft Internet Information Services (IIS) and standard web interfaces
• Facilitates collaboration
• Enables presentation of data, reports, and queries
• Provides document management with content version control
• Allows compliant governance provisions—access and use permissions with granular control
• Accesses robust metadata and taxonomy categorization with search capability
• Supports automated workflows
Microsoft Internet Information Services (IIS) (included with Windows Server 2008 R2)  Used with SharePoint Server
• Provides front-end for content in standard web formats
• Accessible by various internet browsers
• Accommodates requirements for presentation on mobile devices
Ipswitch WS_FTP
• Provides File Transfer Protocol (FTP) services
• Allows secure file transfers between servers and administrative staff
Oracle MySQL Database 5.5  Used with Alliance Sleep Questionnaire (ASQ)
• Allows demonstration of query capability of federated data
Microsoft SQL Server 2008 R2 and 2012 BI Edition Versions
• Primary database storage engine
• Used to accumulate and store data from web forms and device transfers
• Enables business intelligence (BI) features
Microsoft SQL Server Analysis Services
• Utilized to create a data warehouse for multidimensional analyses (cubes)
Windows Task Manager (included with Windows Server 2008 R2)
• Used to run recurrent applications and procedures
Active Directory (AD) (included with Windows Server 2008 R2)
• Directory service that assigns and enforces policies on the network
• Contains and authorizes all users and computers in the network domain

By selecting a solution that is based on the technologies from one vendor, Microsoft, this provided a level of integration, interoperability, and time to delivery that we believe is worth the cost in relation to less expensive technology offerings and open source solutions that often do not provide the level of support required. In addition to interoperability of components, some of the key components such as MS SQL Server and MS SharePoint come in various editions that roughly map to different levels of feature sets, performance, and cost (Table 3). By identifying optional interoperable components in various editions, researchers have fine grained control to select the technology that fits the project in terms of feature set, scalability, cost, and duration of use. Selection of data presentation features is one example of the granular control researchers have in making technology choices to match their needs.

Table 3.

Interoperable Software Components: Editions, Feature Sets, and Costs

Software Component	Edition	Feature Set	Cost
MS SQL Server	Express	Limited	Free
	Standard	Basic	Low
	BI	Business intelligence	Medium
	Enterprise	Full featured	High
MS SharePoint	Foundation	Limited	Free
	Online	Full featured	Low
	Server	Full featured	High

An example of a low-cost, self-service BI solution may include a single server housing a database server (SQL Server) and a web server (IIS). Research team members use client-based tools (Excel) to visualize data with multidimensional analysis and pivot tables.

A higher cost, team-based BI solution may add two servers including a directory service for user authentication (Active Directory) and a collaboration and presentation subsystem (SharePoint). In addition to hosting Excel PowerPivot tables, this configuration provides for more data visualization features (dashboards, scorecards, database reports) in a secure, permissioned environment. Furthermore, team members only need to enter their personalized username and password; then they can use any browser on any OS to access and interact with the data from any internet connected location. This eliminates the need for the installation of additional software on the team member’s local computer.

Application Software Layer

The application software layer is dependent upon the server software layer to execute tasks with the infrastructure layer hardware. The application software layer enables developers to design computer programs that perform tasks specified by the end user.

Agile software development

The agile software development methodology8 is used for the development of applications in this layer. The agile method is an iterative approach by a collaborative team of developers, knowledge engineers, and business analysts to introduce successive improvements through collaboration that ultimately produces a sustainable model that decreases costs and increases quality improvements while maintaining usability. This methodology allows the project team to focus the majority of the development effort on the solution rather than on theoretical proposals or overly detailed software documentation. The development team members are listed in Table 4.

Table 4.

COMET Sleep Research Development Team

Team Role	Members
Stakeholders	Principal Investigator, Project Manager
Information Technology (IT)	Director of IT, Developers
Content	Content Specialist, Knowledge Engineer
Ontology	Ontologist
Business Management	Business Analyst, Research Manager

The agile methodology is especially useful in developing data visualizations such as pivot tables, dashboards, scorecards, and database reports where data must be presented in a simple, self-evident fashion while at the same time being information dense. It is extremely difficult to create a seemingly simple, insightful graphic representation of complex data without an iterative approach such as agile that involves input from a diverse team.

Automating data processing such as receipt, messaging, and processing of device data also requires an agile development process. Frequently device interchange files are not well documented and are sometimes ambiguous. Errors in hand annotations must also be detected, documented in a clear manner for correction, and ultimately received, interpreted, cataloged, and imported into the study database. Numerous iterations in an agile process involving clinicians, content specialist, database administrators, and programmers must be completed to create comprehensive rules that are properly coded for accurate and automated operation. The agile creation of a comprehensive library of rules, messages, and data structures of standard medical device interchange files are also a valuable contribution of the COMET platform.

Selection of development languages

Development languages were selected for rapid development, optimization, and ease of maintenance. Technologies such as Active Server Pages (ASP), Hyper Text Markup Language (HTML), and Java Script (Jscript) are used for browser-based web forms. Microsoft Access 2010, Microsoft SQL Server Management Studio 2008 R2, and Structured Query Language (SQL) are used for data manipulation within the SQL Server and MySQL Databases. Related statements to a given task or process are combined into stored procedures for ease of maintenance and integration with the Windows Task Manager and COMET report scheduling engines. Microsoft Visual Studio is utilized for the creation of code for data file imports utilizing the Microsoft C# (that is, C “Sharp”) language and Visual Basic Scripting (VBScript) languages. The change control approach for software development is used so code is accurately versioned and documented. The Microsoft Business Intelligence Development Studio 2008 R2 and 2012 is used to create the multidimensional cubes from the SQL Server star schema designs for fast compilation of results into relevant outcomes utilizing predefined dimensions for data slicing.

Application Software Layer Subsystems

At the highest level, CET data management requires two subsystems for the Application Software Layer: (1) a data pipeline subsystem for content collection and processing of data into data set results, and (2) a collaboration and presentation subsystem to provide for a consolidated location for investigators to access, view, and query data ultimately leading to reporting and analysis (Figure 2).

Figure 2.

CET Requirements for the Application Software Layer of the COMET Platform

Data Pipeline Subsystem of the Application Software Layer

The content collection processes for CET had significant requirements that were important to consider early in the development process. These core requirements to accommodate multiple data inputs (federated data, web forms, and device data) and the solutions fulfilled for CET are discussed in detail in Appendix C.

Any data system that regularly accumulates and transforms validated data to desired output formats demands a data pipeline subsystem that implements extract, transform, and load (ETL) features with some level of automation. Automated services are central to the COMET platform and enable recurring tasks to be regularly and dependably repeated with detailed tracking logs to minimize the need for human intervention during this processing (Figure 3).

Figure 3.

Data Processing: COMET Data Pipeline Subsystem

The automated COMET ETL processes include reliable scheduling of data acquisition, quality control, and cleansing, as well as ongoing data verification during the clinical trial and research period. The automation features support archived reports documenting data exceptions, exclusions, missing data, and errors while providing mechanisms to manage unhandled or unrecognized data, minimizing the need for manual intervention.

COMET Platform Components

There are three platform components that support these automated and repeatable processes: (1) Windows Task Manager and WS_FTP; (2) Extract, Transform, and Load; and (3) The COMET Report Engine.

Windows Task Manager and WS_FTP

Windows Task Manager is used to schedule the execution of code to upload the various files collected from both clinical centers and scoring centers into the COMET Data Coordinating Center (DCC) platform. Differential scheduling needs including execution at fixed times, varying dates, or varying intervals can be configured. On a scheduled basis, the folders are checked for the upload of file sets at the four research sites and WS_FTP is then used to host the transfer of files to the DCC at Stanford for further processing. Additionally, all the valid file sets are archived upon transfer, noting both the original upload location in addition to the archival location to accurately log the file upload history. The log also notes all validation processes including passes and failures.

The COMET platform is extensible, and is able to accommodate diverse content, including file sets with different components containing an array of files with various formats. It supports raw data file types including the following: Standard CSV/American Standard Code for Information Interchange (ASCII), ASCII/Nonstandard data files, and Non-ASCII proprietary data files. The varying file types uploaded can include alphanumeric data files, instrument data files, and more complex file types such as image and video files.

Extract, Transform, and Load (ETL)

Extraction requires that the files collected from Windows Task Manager scheduled jobs and web forms are consolidated into the COMET SQL Server database at Stanford. Scheduling of automated tasks is flexible (e.g., daily, weekly, or monthly) to support: (1) ongoing administration of studies; (2) regularly scheduled data refresh cycles to approximate real-time status updates (e.g., recruitment progress); and (3) systematic scheduling, accumulation, transformation, and presentation of data in a timely format. Transformation includes a documented series of reproducible steps that are taken to insure the data are validated for correct values, duplicates are removed, missing values are flagged and corrected, and scoring algorithms and derived variables are calculated.

Another critical step during the transformation is to harmonize disparate data. For COMET, a physical data model (PDM) is utilized where detailed study-specific data dictionaries associate metadata to content. Metadata include information related to the structure, formatting, creation, naming conventions, purpose, and the location of all elements in the data dictionary. These data can be mapped to data from other systems given the separate data model is well documented. The processed data is then loaded into table structures designed to better accommodate the type of data uploaded and ultimately facilitate visualizations and data set extractions.

COMET Report Engine

The COMET report engine is programmed to run at scheduled intervals (hourly, daily, weekly) throughout the study, transforming data with scoring and processing algorithms, while simultaneously archiving progress over time. Health Insurance Portability and Accountability Act (HIPAA) and protected health information (PHI) compliance procedures occur at this stage using random renaming functions to de-identify unique patient identifiers. SQL Server Agent facilitates scheduled weekly runs of processing algorithms and report generation. The report engine design allows point-in-time views of study data (beginning with recruitment) to monitor relevant statistics for study management.

Collaboration and Presentation Subsystem of the Application Software Layer

The COMET Collaboration and Presentation Subsystem is accessible through the SharePoint infrastructure, which provides permissioned access to information and a common area for communication and collaboration. Collaboration features, dashboards and scorecards, and reports and analysis components of the subsystem are described and demonstrated below.

Collaboration Features

The collaboration features include document libraries, calendars, contacts, task lists, workflows, and permissioned work spaces. Document libraries allow for easy storage and retrieval of files related to the ongoing development of the COMET platform as well as those documents related to the clinical research and CET. Calendars manage conference calls and the associated materials discussed during the calls. Contacts facilitate communication between team members. Task lists track progress to date and deadlines for uncompleted tasks by research site and task owners. Workflows allow for the automation of tasks designed to save staff time. These collaboration features permit check-in and check-out of documents with version control enabling multiple individuals to provide input to a given document. All features are permissioned for accessibility based on a user’s role in the study.

Dashboards and scorecards

A SharePoint website—for presentation of relevant, real-time data during the study recruitment period—allows the clinicians, clinical coordinators, research staff, and centralized scoring center staff to access information related to study management. All of the sites are permissioned for accessibility based on a user’s need for access and study role. For example, a three-column dashboard created for research staff involved in the processing of 24-hour ambulatory blood pressure monitoring (ABPM) data provides the status for this particular outcome. The first column provides more static summary information including the study workflow and key team members (with links to contact information). The second column provides access to live key performance indicators (KPI), e.g., ABPM files transferred by visit; live data reports, e.g., the number of files transferred and scored by clinical center; and a document library that provides links to study protocols, manuals, templates, references, and other pertinent files. The third column includes links to collaboration features such as calendars and task lists. This graphical user interface (GUI) was designed to be intuitive (more general information to the left, more detailed information to the right) and to present a balance of visual information (e.g., graphs and illustrations) and textual information.

Reports and analysis

While BI and multidimensional analysis (MDA) are used primarily for competitive advantage for businesses in industries such as retailing or banking, BI can also allow an effective review and more efficient analysis of vast amounts of clinical and research data in health care. The use of BI and MDA within COMET enables easier, more insightful querying of big data sets and integration of data across disparate research systems. With MDA, individual researchers can rapidly and effectively answer questions of interest using real-time data. By providing user-driven experiences, MDA decreases staff hours necessary to run custom queries or write and rewrite customized reports. The elements of BI in the COMET platform include (1) The COMET Data Warehouse; and (2) Microsoft SharePoint, Access, and Excel.

The COMET Data Warehouse

The COMET data warehouse comprises a series of processed and transformed outcome variables collected during a study with a prespecified set of specific aims. These outcomes are organized into sets of star schema design data sets containing specified study outcomes in measures and dimensions for slicing. These data sets are then utilized to create multidimensional cubes that can be used to better visualize data using BI solutions. BI can present the measures of interest for a given study or group of studies and can quickly and efficiently slice data using relevant dimensions. For CET, one of the multidimensional cubes focused on the primary and secondary cardiovascular outcomes including variables derived from 24-hour ABPM and vascular ultrasound devices. Multidimensional cubes can also be de-identified.

Microsoft SharePoint, Access, and Excel

While the study is ongoing, progress reports can be generated and displayed using Share-Point, Access, and Excel. Excel Power Pivot and Excel Power View can be used as the BI front-end tools to provide a rich user-driven experience of the data accumulated on a robust research platform that enables easy access to these tools. Figure 4 shows one of the dashboards accessible by permissioned users to select, view, and interact with a series of Excel Power Pivot Reports.

Figure 4.

Power Pivot Dashboard: COMET Collaboration and Presentation Subsystem

Figure 5 is a screenshot of one of the Power Pivot visualizations used during CET to monitor the number of web forms entered by study staff. It depicts three filters that allow the user to view the number of web forms entered after selecting various combinations of study visit, patient randomization status, and site. Data in tables and graphs update immediately as different filter options are selected. Final study reports can also be constructed using these BI tools.

Figure 5.

Power Pivot Report: COMET Collaboration and Presentation Subsystem

Discussion

Frequently, informatics tools for clinical research are created at the time of the award of a grant and commencement of clinical trials. Tools created may fulfill one or more purposes for the trial, but may be limited for future applications. Many times, when the grant is over, development of one or more tools ceases and may or may not remain available due to the costs to keep the infrastructure and staff funded. Performing a comprehensive requirements analysis prior to development with a diverse group of stakeholders will increase the generalizability and sustainability of informatics tools developed for a specific research protocol.

Implementation

In a requirements analysis phase performed prior to the commencement of COMET, we strived to identify key features that would most likely lead to a sustainable research infrastructure. Based on this input, development of the COMET platform focused on designing components that provide a robust, standardized, extensible, and scalable clinical research platform. The robust features of the platform include its use of industry standard OSs and servers with thorough third-party documentation and ample avenues for technical support. Utilizing products from a single vendor reduces overall development costs due to the established integration of various server products; databases; and applications with features for governance, and data-access and use permissions. Standardization of table structures, data imports, and use of a data dictionary with associated metadata provides the means to share data across different institutions. The normalization of key reference fields from medical devices provides a generalizable mapping structure to easily accommodate multiple devices. Extensible applications developed using agile processes are built to be data driven, allowing the flexibility to support diverse content as study needs arise.

The team chose a broad family of servers and other technologies, both open source and commercial software, which are interoperable to allow selection of only the components required for a particular research project. Not all COMET components need be selected for every trial. As COMET addresses a superset of clinical research needs, use of a commercial software provider is warranted because of the breadth and depth of features required (e.g., extensibility) that could not be economically provided by a small academic team. This approach helps ensure a platform that is sustainable into the future and provides the latest cutting edge tools on an integrated platform. Additionally, comprehensive training, support, and development resources are accessible worldwide.

We chose the Microsoft family of servers and development products to minimize software development costs, to allow for the integration of multiple products, and to reduce the time to delivery; and also to have the ability to move to a cloud computing service model in the future. By identifying optional interoperable components in various editions (ranging from freeware to Enterprise), researchers have a fine-grained control for selecting the technology that fits the project in terms of feature set, scalability, cost, and duration of use. The researcher can also choose to apply selected components to portions of the project or utilize the full end-to-end data management solution offered by the COMET platform. The reusable nature of the COMET Platform applications and application software layer (see Appendix B: COMET Infrastructure Detail View) will save time and money, increase data quality, and be easier to use for research than one-off partial solutions. Note that informatics solutions cannot address all potential barriers to data sharing endeavors, as there are institutional, regulatory, and governance issues that must be addressed in concert with informatics issues.

Governance

Several administrative requirements were also identified during the requirements analysis. For a multicenter clinical trial, governance issues such as approval by an Institutional Review Board (IRB) and monitoring by a Data and Safety Monitoring Board (DSMB) were important to address. We found it beneficial for the parent site to take the lead on the necessary IRB submissions and to obtain the initial IRB approval because this permits the creation of templates to aid the remaining sites with their individual IRB submissions. A primary function of the DSMB is to monitor patient safety and study metrics. Constructing queries to track and report on important metrics such as participant enrollments and randomizations in addition to participants who withdrew or dropped out (patient driven; pre- and post-randomization),were excluded or disqualified (staff driven; pre- and post-randomization) from the study provides the necessary parameters for reporting.

Strengths of the COMET Platform

The COMET platform is unique in its approach to analysis of research data with the implementation of MDA and BI for visualization and rapid access to data and results. The platform is scalable in its ability to accommodate data sets, by both visit and accumulation. Rather than build a new research platform each time a study is funded, COMET was designed to provide a general purpose infrastructure that is flexible enough to support diverse multicenter clinical research study management. Inclusion of a common expandable toolset creates a foundation for rapidly building and deploying varying research studies, thus reducing overall costs and increasing ROI. This system, while developed within a large academic center, could ultimately provide the greatest value to smaller research centers lacking access to the infrastructure and information technology (IT) staff required to implement a collaborative solution.

Limitations of the COMET Platform

As in all technology offerings, there is a team of informatics experts and business analysts behind the design of the COMET platform. While the COMET platform is generalizable to accommodate diverse study aims and extensible to span several medical disciplines, implementation for new research studies will still require the involvement of informaticians and business analysts. While the platform provides a unique tool for medical research, several more iterative deployment cycles are required to account for most, if not all, permutations of technical, clinical, and research requirements. With further attention given to documentation and automation, a future version of the platform could accommodate development by individuals outside of our existing team. In addition to the internal use by study investigators, the COMET platform has implemented a public website (http://comet-sp.stanford.edu) with information on the COMET platform and potential uses by other organizations and research projects. This external public website can be used in the future for the dissemination of published research results and direct correspondence with other interested investigators.

Conclusion

The COMET platform has the capacity to unite clinical and research needs with IT to create an end-to-end data management solution. Future versions of the platform may accept additional types of user input and enhanced mechanisms for instrument data collection. It is envisioned that dynamic generation of clinical input mechanisms and their corresponding database tables will allow for the rapid inclusion of new and varied types of content.

The platform was designed to be flexible enough to accommodate requirements for many different studies. The platform’s architecture can be scaled up to accommodate big data sets with greater complexity, automation, and dependence on Enterprise-level solutions or scaled down to provide specific features at much lower financial and manpower costs using free or low-cost components for basic research. The integration of technology early in the study start-up period greatly increases study efficiency and reduces the time and effort spent on manual processing. Furthermore, BI and MDA provide for an extensible model for analysis that can harmonize data from data warehouses across medical disciplines and multiple CER studies.

With the inclusion of MDA and BI, the platform could eventually be utilized for predictive analysis that can facilitate early detection and the implementation of prescribed protocols for disease prevention. Predictive analytics, a technology feature that finds patterns and trends within large volumes of data that may predict outcomes, is included as part of the Microsoft Business Intelligence products contained within the SQL Server Analysis engine. With the proper input from clinicians, knowledge engineers, and biostatisticians, predictive analysis has great potential for use in health care systems to predict disease outcomes, causes, and yet-unknown medical correlations that may exist in the data.

The ultimate model for sustainability may be to plan for a future move to cloud computing services, and the COMET platform or component parts of the platform can be moved to cloud computing services in future versions. The COMET platform uses a segmented, componentized architecture where the technology layers logically map to the cloud computing service model (NIST Definition)6 including Infrastructure as a Service (IAAS), Platform as a Service (PAAS), and Software as a Service (SAAS) (Figure 6). The benefit of moving the COMET architecture to the cloud architecture is the reduction of operational costs, particularly in the area of infrastructure and platform.

Figure 6.

Future Cloud Computing Service Model

A robust community of users, developers, and supporters must evolve together with a collaborative workspace for ongoing contributions, concurrently with the technical and infrastructure iterative development of the COMET platform. A COMET “Hub” has been identified as such a place with support and funding currently being pursued.

This endeavor has the support of interested academic researchers from multiple institutions and industry leaders such as Microsoft. The website for the COMET Hub will launch in the winter of 2014 and will eventually provide hands-on access to the COMET platform and applications in a subscription-based cloud environment that allows collaborators to use key features, evaluate suitability for their own projects, contribute to the growth of the platform, and ultimately use the platform for their own medical research projects.

In conclusion, COMET is a research platform that is a generalizable and extensible infrastructure that, in a future version, will be able to accommodate big data sets and enable efficient and effective clinical research across multiple studies and medical specialties. The COMET platform components were designed for an eventual move to a cloud computing environment that significantly enhances overall cost effectiveness, and may be a potential key to sustainability for this and other platforms.

Acronyms	Terms
ABPM	24-hour Ambulatory Blood Pressure Monitoring
AD	Active Directory
AHRQ	Agency for Healthcare Research and Quality
ASCII	American Standard Code for Information Interchange
ASP	Active Server Pages
ASQ	Alliance Sleep Questionnaire
BI	Business Intelligence
CET	Comparative Effectiveness Trial
CMS	Content Management System
COMET	Comparative Outcomes Management with Electronic Data Technology
CSV	Comma Separated Value
DCC	Data Coordinating Center
DMS	Document Management System
DSMB	Data and Safety Monitoring Board
EDF	European Data Format
ESS	Epworth Sleepiness Scale
ETL	Extract, Transform, and Load
FTP	File Transfer Protocol
GUI	Graphical User Interface
HIPAA	Health Insurance Portability and Accountability Act
HTML	Hyper Text Markup Language
IAAS	Infrastructure as a Service
IIS	Internet Information Services
IRB	Institutional Review Board
Jscript	Microsoft dialect of Java Script (ECMAScript)
KPI	Key Performance Indicator
MDA	Multidimensional Analysis
NIST	National Institute of Standards and Technology
OA	Oral Appliance
OS	Operating System
PAAS	Platform as a Service
PAP	Positive Airway Pressure
PDF	Portable Document Format
PDM	Physical Data Model
PHI	Protected Health Information
POMS	Profile of Mood States
ROI	Return on Investment
SAAS	Software as a Service
SAQLI	Calgary Sleep Apnea Quality of Life Index
SQL	Structured Query Language
VBScript	Visual Basic Scripting
VM	Virtual Machine