Comparison of RN licensure examination: China and the United States.

Having the two largest nursing workforces across the globe, both China and the United States have implemented licensure examinations to standardize the qualifications of registered nurse. China established the National Nursing Licensure Examination (NNLE) in 1995. Like the National Council Licensure Examination for Registered Nurses (NCLEX-RN) in the United States, the NNLE has become an important link in the nurse registration process and contributed a lot in quality assurance and development of nursing profession. It may be necessary for the NNLE to learn from NCLEX-RN in several aspects, including increasing the frequency of exam, providing examinees with individualized services, and continuing reforming exam content and format. By better aligning the content and format of the examination with the current nursing paradigm and clinical requirements, nurses may enter the healthcare field better prepared to care for patients. Considering the magnitude of their nursing workforce, both China and the United States have great potential to serve as a role model for other developing countries as they look towards establishing national nursing education programs. As new policies around standardized nurse education in China take form, we urge that the NNLE be included alongside curricular reforms. As these reforms are implemented, continued research will be needed to evaluate the effectiveness of curriculum and licensure exam regulations upon nurse preparedness and quality of nursing care in China.

Introduction

Influential reforms in the education of health professionals in the western world began taking shape during the early 20th century, carrying far-reaching impacts that continue to carry international impact on modern medical and nursing education today. Early in 1903, the concept of registered nurse was introduced in the United States, which stipulated that all personnel who provided direct nursing care to patients must complete nursing professional training courses, pass registered nurse examination, and obtain a registered nurse license [1]. As a landmark report on professional nursing education with global implications, the Goldmark report was published in 1923 in the United States, 13 years after the Flexner report first established an academic model for medical education [2]. Lagging slightly behind medical education reforms, the Goldmark report called for university-based nursing schools, placing nursing education in an academic sphere similar to medicine. To guard the safety of the public, entry into the practice of nursing is strictly regulated. All registered nurse candidates are required to meet a set of requirements, including passing the National Council Licensure Examination for Registered Nurses (NCLEX-RN), which has been implemented since 1987 [3].

Halfway across the globe, China has been influenced significantly by the nursing reforms taking shape in North America. Historically, western policy reforms as well as missionary nurses played important roles in China's early development of modern nursing education. Indeed, by 1923, missionary nurses accounted for over 30% of the nursing workforce in China. The first college-level nursing program was established in 1920 at the Peking Union Medical College with funding from the Rockefeller Foundation. Ten years later, China's nationalist government funded its first secondary-level nursing school [4]. There were 216 secondary nursing programs established in China by 1949, when the communist government came into power. During the national educational reforms initiated by the new government, nursing education beyond the secondary level was abolished, and nursing education only regained its academic context in the 1980s. The effects of this 30-year exclusion of nursing from the realm of higher academia are seen today, as China experiences a shortage of nursing faculty with proper teaching qualifications [5].

Over the past three decades, China has been steadily building its nursing workforce. Despite the lack of research published on China's nursing education and licensure policies, China's global prominence as a producer of healthcare professionals is evident. With one of the largest nursing education systems across the globe, China is home to over 3.2 million nurses [6]. The average cost of nursing education in China, which was US $3000 per graduate, is the lowest across the globe. In comparison, North America expends an estimated US $101,000 for each of its 74,000 nursing graduates each year. North America spends more than any other global region, over 30 times the amount spent by China [2]. Although the magnitude of China's nursing workforce lags in comparison to its enormous medical workforce, Chinese nurses account for about 13.7% of the global nursing workforce in 2015 [7].

Despite its high volume of nurses, China is nonetheless experiencing a nursing shortage, as the demand for healthcare professionals increases across the country, which carries nearly twenty percent of the world's population [8]. As China seeks to meet its growing demand for nurses, it is vital that the country does not sacrifice the quality of its healthcare professionals for the sake of quantity. In an effort to build not only a larger but more competent and qualified workforce, China has joined the international movement to implement a national licensure examination for nurses. In 1995, China established the National Nursing Licensure Examination (NNLE). Prior to the establishment of the licensure examination for nurses, nurse registration and management in China had been unregulated, lacking any national standards. Due to the lack of a rigorous licensing examination system, the quality of nurses varied widely across the country.

Although the NNLE has now been implemented in China for over 20 years, there is a lack of research published in English focused on the NNLE and its policies. The purpose of this paper is to review the history of nurse licensure policy in China, describe when and how the format and content of the NNLE have historically been reformed, compare the RN licensure examination system between China and the United States, and offer recommendations to inform future policy changes.

Data and methods

To search literature on the NNLE, database search using China National Knowledge Infrastructure, Wanfang Data, and PubMed MEDLINE was performed using keywords for the China National Nursing Licensure Examination for Chinese and English language literature published since 1993, when the examination was first enacted. Alternative names for the examination (China Licensure Examination for Nurses and National Nurse Qualification Examination) were also included. Due to the lack of English language literature published on the examination itself, the search criteria were expanded to broadly include nursing education in China. Official documents issued by China's government sectors (e.g. Ministry of Health) were also reviewed. The majority of previous literature on the NCLEX-RN was retrieved from National Council of State Boards of Nursing (NCSBN) website.

Literature on the NNLE were included if they were published or issued in Chinese or English after 1992 when the NNLE was enacted. Literature were excluded if they did not meet above-mentioned criteria. All recruited literature were reread in full text. To facilitate analysis, recruited literature were summarized according to author or issuance entity, year of publication or issuance, country, language, research design or type of literature, and themes (Table 1).

Table 1

Summary of recruited literature.

Authors/Issuance entity	Year	Country	Language	Research design/type	Themes
Wong et al. [9]	2012	China	English	Review of literature and policy documents	Development of nursing education in China
Zhang and Wang [10]	1994	China	Chinese	Descriptive analysis	Pilot of China's nursing licensure examination
Tang and Jiang [11]	2015	China	Chinese	Comparative study, review of literature and policy documents	Comparison on the test guidelines for RN licensure examination in China and the United States
MOH of China, Ministry of Human Resources and Social Security [12]	2010	China	Chinese	Policy document	Measures for the National Nursing Licensure Examination
Zhao and Zhang [13]	1997	China	Chinese	Descriptive analysis	Content, format, and management of the NNLE
NCSBN [14]	2018	United States	English	Policy document	Information for registrations and scheduling of NCLEX appointments
NCSBN [15]	2018	United States	English	Policy document	Requirements for eligibility to take the NCLEX and to get a license/registration
California Board of Registered Nursing [16]	2018	United States	English	Policy document	Steps to become a California registered nurse
Health Human Resources Development Center [17]	2018	China	Chinese	Policy document	Information for taking the NNLE
The NNLE Committee [18]	2017	China	Chinese	Policy document	Cut Scores for the NNLE
Chen [19]	2016	China	Chinese	Descriptive analysis, review of policy documents	Design and future development of the NNLE
Fan and Xiao [20]	2017	China	Chinese	Review of literature	Teaching of internal nursing science based on the NNLE
Wang [21]	2017	China	Chinese	Treatment-control study	Nursing education reform and nursing students' performance in the NNLE
Yang [22]	2017	China	Chinese	General introduction	Nursing education reform based on the NNLE
Zhang et al. [23]	2011	China	Chinese	Review of policy documents	History, role, and future development of the NNLE
MOH of China [24]	2008	China	Chinese	Policy document	Regulation on nurse practicing and registration
Su [25]	2018	China	Chinese	Review of policy documents and literature	Reform of the NNLE and development of nursing education
Zhuang [26]	2014	China	Chinese	Review of policy documents	Nursing care of infectious patients in the NNLE
Yang and Yang [27]	2015	China	Chinese	Review of policy documents and literature	Nurse licensure and continuous management
Yi et al. [28]	2011	China	Chinese	Review of policy documents and literature	Development tendency of the NNLE and coping strategies

Results

History of the NNLE

The Ministry of Health issued the Regulation on Nurse Administration in 1993, declaring that nursing graduates from health schools and universities must pass the NNLE in order to become licensed as a registered nurse [9]. The first licensure examinations for nurses were piloted in 1994 in Yunnan Province of western China, Henan Province of central China, and Beijing of eastern China[10]. The examination has been administered annually since 1995 [11]. In 2008, the NNLE was redesigned under the Nurse Regulation as the new act outlined the basic components of the NNLE and an updated test guideline was developed. Following the implementation of the NNLE, China's Ministry of Health disseminated the Measures for the Licensure Examination for Nurses (hereinafter referred to as “the examination measures”), putting forward more detailed provisions of the licensure examination and promoting its most recent developments. With 22 items, the examination measures specify the nature, examination time, subjects and other basic content of the NNLE, clearly stating the educational requirements for the examination [12].

The content of the NNLE primarily derives from a national questionnaire survey, in which over 2000 Chinese RNs participated. The survey results had established a list of more than 100 nursing tasks that highlighted the knowledge, skills, and attitude required by a practicing RN. Since its national implementation in 1995, the NNLE's content and format have evolved. From 1995 to 2010, the examination was divided into five subjects: basic nursing, internal medicine nursing, surgical nursing, obstetrics and gynecology nursing, and pediatric nursing. Psychology, ethics, and regulations were gradually incorporated as well. The examination time was eventually reduced from two days to a single day of assessment. In 2011, significant changes were made to the test contents, as the focus of the test shifted from being science-based to clinically-based. This was intended to more closely align the exam with clinical practice, to better assess examinees' knowledge and ability. In addition to basic nursing knowledge, communication skills and nursing ethics were included in the assessment. Questions concerning nursing administration, education, and traditional Chinese medicine were also added to the test. During the same year, the number of test subjects decreased from four to two, as the former categories of basic knowledge, knowledge of relevant specialties, specialized knowledge, and professional practice skills were reorganized into basic science knowledge and nursing skills. An examinee must pass both sections to obtain qualification as a registered nurse.

Traditionally, the NNLE has been a paper-based test. Studies were carried out to evaluate the possibility of computerized testing, which carries many potential benefits. Computerized testing can support more precise measurements, increase the frequency of administered tests, and expand test materials from words to include photos, videos, and virtual simulation. Certain virtual simulation methods were also evaluated for their potential inclusion in the NNLE. In 2015, the National Health and Family Planning Commission (former Ministry of Health) decided to pilot computerized examinations the following year, in 2016. In May 2016, computer-based tests (CBT) were successfully implemented in three regions, while other regions continue to use the traditional paper-based test. The test score equivalents, converted score reports, and qualified standard have been changed accordingly with the application of CBT. The examination results demonstrate improved quality and security with CBT, compared to paper-based tests. At present, the overall management and implementation of the NNLE has been gradually catching up to meet internationally recognized measurement standards. Following the success of CBT in 2016, the National Health and Family Planning Commission decided to apply computerized examination throughout China in 2017.

Comparison with the NCLEX-RN

Concept and definition of licensure examination

The NNLE is an examination to determine whether an RN applicant has the knowledge and practical ability to provide nursing service. Focusing on the fundamental knowledge and skills, the difficulty of the NNLE matches the required level of nurses who have engaged in nursing practice for one year after receiving secondary nursing education [13]. Only those who pass the examination can get a primary nursing qualification certificate and apply for the registration of providing nursing services [12]. Similarly, the NCLEX-RN is an examination that measures knowledge, skills, and abilities that are needed to practice safely and effectively as a newly licensed, entry-level registered nurse [14].

Prerequisites for taking the RN licensure examination

According to the examination measures, the eligibility requirements for taking the NNLE include completing a nursing or midwifery program which last at least three years, graduating from a nursing or midwifery specialty in a secondary school or higher education institution, completing a clinical clerkship in a teaching or comprehensive hospital for at least eight months, and obtaining a diploma, associate degree or bachelor degree.

In the United States, the prerequisites for taking the NCLEX-RN are determined by the board of nursing (BON) or regulatory body (RB) [15]. For example, the California Board of Registered Nursing stipulate that completion of an education program meeting all California requirements is a must for all applicants. Those who lack any educational requirements must successfully complete an approved course prior to taking the examination. An Associate Degree in nursing, Bachelor of Science in nursing, and entry level Master programs in nursing, which respectively take 2–3 years, 4 years, and 1–2 years, are the three types of pre-licensure nursing programs to become a registered nurse in California. Furthermore, California Licensed Vocational Nurses and military corpsmen are permitted to take the NCLEX-RN if they meet certain requirements [16].

Time and frequency of RN licensure examination

In China, the exam is held simultaneously throughout the whole country, and offered only once per year. In 2018, the exam was held from May 5th to May 7th. During this period of time, an examinee can take just one of six rounds of the examination [17]. However, in the US, each qualified candidate can take the NCLEX-RN eight times in one year by making individual appointments with the NCSBC. This allows those who failed to pass the examination multiple opportunities to retake the test, and thus enter the profession once they are qualified. Nevertheless, candidates are not allowed retake their exam sooner than 45 days after administration of the exam.

Exam content and format

The NNLE now has two sections: basic science knowledge and nursing skills. Each section lasts 100 min, consisting of 120 questions. All items are selected from an item database. The content and format of the four kinds of questions included in the NNLE are presented in Table 2 below. Questions based on case studies are intended to assess examinee's ability to analyze and integrate clinical information. Since 2017, the NNLE has been provided in the CBT format at test centers located across China.

Table 2

Content and format of China's National Nursing Licensure Examination.

Question Type	Content	Format
A1	Definitions of technical terms	Multiple choice
A2	Clinical case study	Multiple choice
A3	Clinical case study	Multiple choice
A4	Clinical case study	Multiple choice

In the United States, the framework for RN examination is based on patient needs, which are categorized into safe and effective care environment, health promotion and maintenance, psychosocial integrity, and physiological integrity. Items of varying levels of difficulty are included in the NCLEX-RN examination, with most of them involving application of knowledge. Nurse licensure candidates have been taking the NCLEX-RN examination on computers at domestic or international test centers since the format changed to computerized adaptive testing (CAT) in April 1994. Compared with traditional pencil-and-paper examinations, the CAT is able to produce reliable exam results using fewer items by targeting items based on a candidate's ability. As the NCLEX-RN examination varies in length, the number of test items ranges from 75 to 265, including 15 pretest items that are not scored. Most of the items are multiple-choice, although there are also alternate formats, including multiple response, fill-in-the-blank calculation, ordered response, and/or hot spots. Multimedia such as charts, tables, graphics, sound and video may be used for all item types [14].

Exam management

Under the leadership of the National Health Commission, the Health Human Resource Development Center (HHRDC) is responsible for developing and reviewing test questions, preparing test guidelines, formulating a scoring standard, and daily management of the NNLE. A standardized system is implemented to assign and verify test questions, and to score examinations. The HHRDC commissions authorities from several disciplines to establish an expert committee for the NNLE, which is responsible for making adjustment for major issues regarding test guidelines, contents, and cut-off score. The HHRDC is responsible for recommending an annual cut-off score of the test. Information from the following three sources is important in determining the cut-off score: test-paper based research, test equating process, and studies on nurse supply and demand. Based on these sources and the HHRDC's recommendations, the NNLE committee makes the final decision on the cut-off score. For example, in July 2014, the committee set the following cut-off scores: 77 for basic science knowledge and 79 for nursing skills. To pass the examination, one has to meet both of these criteria [18]. Starting in 2016, NNLE scores have been reported as converted scores based on standard scores. The 2016 cut-off score for the two subjects, for example, was 300. The HHRDC also offers test data analyses at country, provincial, and school levels to all parties of interest. These reports inform nursing schools about their graduates' performance and areas of improvement in the NNLE, compared to national standards. Nursing schools are thus informed of any disparities between their educational plans and national standards, as they strive towards bridging those gaps. For those who pass the NNLE, a paper-based certificate is mailed to them, except in three provinces where they can download an electronic certificate through the internet [17].

The NCSBN, an independent and not-for-profit organization whose members include jurisdictional boards of nursing from around the world, is in charge of developing the NCLEX-RN examination. A candidate needs to register for the NCLEX-RN examination and schedule their exam with Pearson VUE. Every NCLEX-RN is scored twice for the purpose of quality control: first by the computer at the test center, and then the result is rechecked after the transmission of the examination record to Pearson VUE. The NCSBN Board of Directors sets the passing standards and reevaluates these standards once every three years. The standard criterion is based on a minimum level of ability to safely and effectively practice nursing at an entry level. Official examination results are released to candidates only by the BONs or RBs. There are three different scenarios that determine whether a candidate passes or fails the examination: the 95% confidence interval rule, maximum-length exam, and run-out-of-time rule. Those who fail the examination will be sent a Candidate Performance Report (CPR) which provides a summary of their relative strengths and weaknesses based on the test plan. Failing candidates can use the CPR to guide their study before retaking the examination [14]. Furthermore, all nursing programs recognized by their BON/RB may be informed about their candidates' performance by subscribing to NCLEX Program Reports which are produced semi-annually and annually.

Impact of the licensure examination on nursing workforce and education in China

From the implementation of the NNLE in 1995 until the end of 2017, the number of registered nurses in China rose from 1.1 million to 3.8 million, with an annual increase of about 122,000 (Fig. 1). This rapid increase in the number of registered nurses not only reflects positively upon the NNLE, but has also relieved the shortage of human resources in the nursing field. From 2011 to 2015, approximately 0.6–0.7 million nursing graduates took the test annually. In 2000, about 80,000 applicants were licensed. The number of licensed applicants rose to 350,000 by 2011[19]. By establishing a national standard for nurses, the NNLE promotes the quality of the new nurses entering the workforce. By the end of 2013, among working registered nurses, 0.1% had Master's degrees, 12.5% were college graduates, and 47.3% graduated from junior college schools. Secondary school graduates accounted for 38.7% of working nurses, and high school graduates accounted for 1.4%. In 2015, more than 50% of NNLE examinees had graduated from junior college or above [19].

Fig. 1

Growth of nursing workforce in China, 1995–2017.

The HHRDC considers the NNLE not only a tool to assess examinees' ability but also a bridge between nursing practice and education. The HHRDC also attached great importance to soliciting ideas from nursing experts and healthcare administrators in developing the NNLE. Furthermore, some studies are conducted every year, leading to minor modification of the examination guideline. For example, as humanistic clinical skills becomes increasingly important in clinical nursing practice, the NNLE places more emphasis on assessing communication skills and legal/ethic competencies by increasing the number of items related to them. As a result, many nursing schools began to offer relevant courses which were not included in their curriculum [19]. Generally, the NNLE has significantly influenced China's nursing education in terms of teaching, curriculum, pedagogy, student assessment, and clerkship[20,21,22]. Since the NNLE continuously updates itself according to up-to-date nursing concept and technology, the vision of nursing education in China has also been reformed correspondingly. Furthermore, the strict access system for the examination restrains unregulated schools and nursing specialty and contributes to the development of nursing education to a higher and more standardized level [23].

RN license renewal system

Those who pass the NNLE are granted a Nurse Qualification Certificate, with which they are eligible to apply for a Nurse Practicing Certificate and be registered as a nurse at provincial health authorities. Registration date and allowed site of practice are indicated in the Nurse Practicing Certificate and the registration should be renewed every 5 years [24].

Stakeholders' satisfaction with the NNLE

The NNLE is regarded as an important measure to promote the competencies of the nursing workforce, improve nursing care quality, and ensure nursing care safety. NNLE passing rates have been widely used to indicate a given nursing program's quality of education [25]. Nevertheless, concerns remain regarding incongruence between test guidelines and test content [26], insufficiency of some exam contents (e.g. holistic nursing, mental health, and health promotion) that are important to nursing practice [27], the gap between preparing for taking the NNLE and completing clinical clerkships [28].

Discussion

Having the two largest nursing workforces in the world [29], both China and the United States have relied on licensure examinations in the selection of qualified registered nurses. In these two countries, the RN licensure exams have become an important link in the nurse registration process and contributed a lot in quality assurance and development of nursing profession.

Since the implementation of the NNLE in China, the examination has been keeping on changing in its content, format, and management with the purpose of better serving RN selection. Compared with NCLEX-RN, the NNLE may still need to be further improved in several aspects, such as increasing the frequency of exam, providing examinees with individualized services, and enriching exam content and format. For a long time, the NNLE has relied on multiple choice questions no matter being a traditional paper-based or computer-based test. Although this format has its strengths, such as objectivity and ease of scoring, there are deficiencies in other ways. For example, answers may be overly simplified, failing to test examinees' thought processes. Results may also be skewed by guessing. To address these issues, it may be necessary for the NNLE to change the format to computerized adaptive testing. Furthermore, multimedia methods, such as sound, video and graphics, should be integrated to develop a stronger and larger question bank and to create a clinical environment that more closely simulates reality [30].

Although the implementation of the NNLE marks an important movement towards improving national standard of nursing professional across China, there is still much work to be done. Over the past decade, more attention has been paid to China's nursing education programs, particularly in the context of the current nursing shortage and educational reforms. As Wang et al. report, the curricula of nursing education programs are inconsistent across China, without any national standards of learning objectives. Most nursing programs mirror medical curriculum, oftentimes using nearly identical course materials [8]. While nursing education has historically been grounded in the biomedical model, China has been experiencing a shift towards a biopsychosocial approach [4]. This paradigm shift is moving nursing education, similar to medical education, from being disease-focused to patient or prevention-focused.

While some educational programs are including holistic nursing and patient-centered care within their curricula, this change has yet to be implemented nationally. Strikingly, there is no mention of the NNLE as a potential avenue through which this paradigm shift can be further realized on a national scale. Exam content and format ought to be reformed alongside nursing program curricula. Even within discussions concerning the efforts to reform China's nursing education system, any significant mention of the NNLE remains missing. By employing the licensure examination as a tool to effectively regulate China's nurses on a national level, the impact of these budding nursing curriculum reforms can be more clearly evaluated. As the NNLE has the potential to play a key role in the standardization of nurse quality across the country, there must be greater alignment between nursing curricula across the country and the exam content.

According to their clinical supervisors, nursing students in China “display too much theoretical knowledge and lack ‘real’ nursing skills” [8]. As others have recommended, Chinese nursing programs need to improve the clinical training of their students. High patient loads and lack of available supervisors have been reported during clinical placements for students [8]. As the nursing paradigm has shifted to focus upon patient care, clinical nursing skills ought to be more rigorously assessed through the NNLE. Nurses need to do a variety of practical operations in clinical work. Therefore, the importance of nursing skill tests in the examination cannot be overemphasized [29].

Conclusions

Considering the magnitude of their nursing workforce, both China and the United States have great potential to serve as a role model for other developing countries as they look towards establishing national nursing education programs. Within China's nursing education system, curriculum reforms have been underway, in accordance with shifts in its nursing paradigm towards patient-centered care. We strongly recommend that the National Nursing Licensure Examination be included in China's discussions on nursing reforms as the exam carries national weight as a standardization tool. As of now, the licensure exam is the only national standard for registered nurses in China, as nursing curricula are not uniform across the country. By better aligning the content and format of the NNLE with the current nursing paradigm, Chinese nurses may enter the healthcare field better prepared to care for patients. Along with educational reforms that improve clinical training for nurses, we recommend that the NNLE better assess clinical skills. It may also be necessary for the NNLE to increase the frequency of exam, provide examinees with individualized services, and enrich exam content and format.

As new policies around standardized nurse education in China take form, we urge that the NNLE be included alongside curricular reforms. As these reforms are implemented, continued research will be needed to evaluate the effectiveness of curriculum and licensure exam regulations upon nurse preparedness and quality of nursing care in China.

Conflicts of interest

None.

Funding

This work was supported by Health Professional Education Foundation of Thailand and China Medical Board (12-121).

Ethical approval

Not applicable.