Locating and Critically Analyzing Primary Research Articles

JONAVolume 47, Number 4, pp 212-218Copyright B2017 Wolters Kluwer Health, Inc. All rights reserved. THE JOURNAL OF NURSING ADMINISTRATION Nursing Informatics Competency Assessment for the Nurse Leader The Delphi Study Sarah Collins, PhD, RN Po-Yin Yen, PhD, RN Andrew Phillips, PhD, RN Mary K. Kennedy, MS, RN-BC OBJECTIVE: The aim of this study was to identify nursing informatics compete ncies perceived as relevant and required by nurse leaders.

BACKGROUND: To participate as a full partner in healthcare leadership among rapidly advancing health information technologies (HITs), nurse leaders must attain knowledge of informatics competencies re- lated to their clinical leadership roles and respon- sibilities. Despite this increased need to engage in HIT-related decision making, a gap remains in vali- dated informatics competencies specific to the needs of nurse leaders.

METHODS: An environmental scan and 3-round sur- vey using Delphi methods used with nurse leaders for competency identification were used.

RESULTS: Between 26 and 41 participants responded to each Delphi round. Most nurse leaders acquired HIT knowledge through on-the-job training. We identified 74 competencies from an initial list of 108 competencies.

CONCLUSION: This work can advance nursing prac- tice to move beyond Bon-the-job informatics training [ to a more competency-based model of nursing infor- matics education and practice. In today _s high-technology environment, leaders in the healthcare field encounter decisions related to health information technology (HIT). The roadmap and recommendations identified by the AMIA Elec- tronic Health Record 2020 Task Force include simpli- fying and speeding up documentation, improving data exchange and interoperability, reducing data entry, focusing on patient outcomes, improving usability and safety, fostering innovation through the use of application programming interfaces and data stan- dards, and promoting integration of electronic health records (EHRs) to include all areas of care. 1The achieve- ment of these recommendations to resolve complex EHR issues that bridge the cli nical, policy, and technical domains will require effective collaboration between vendors, informaticians, and clinical leaders at health- care organizations. To parti cipate as a full partner in these collaborations, clinical leaders must attain knowl- edge of informatics compete ncies related to their clin- ical leadership roles and res ponsibilities. Nurse leaders serve as the voice for more than 3 million nurses, the largest segment of the US healthcare workforce, across hospital, ambulatory, community, home, and long-term-care settings. 2According to the American Organization of Nurse Executives, the roles that rep- resent most nurse leaders are directors (32%), man- agers (26%), and chief nursi ng officers (CNO)/chief nursing executives (CNEs) (17%). 3We believe that similar to the diffusion of evidence-based nursing practice nurse leaders _ability to make informed strategic and operational decisions related to HIT adoption, implementation, and innovation is critical, necessitating the attainment of informatics competencies relevant to their work. 4,5 Despite this increased need to engage in HIT-related decision making, a gap remains in 212 JONA Vol. 47, No. 4 April 2017 Author Affiliations: Senior Clinical and Nurse Informaticist and Instructor in Medicine (Dr Collins) , Partners Healthcare Systems, Brigham and Women _s Hospital, Harvard Medical School, Boston, Massachusetts; Clinical Assistant Professor (Dr Yen), Department ofBiomedical Informatics, The Ohio State University, Columbus;Assistant Professor (Dr Phillips), MGH Institut e of Health Professions, School of Nursing, Massachusetts General Hospital, Partners HealthSystem, Boston; and Informati cs Specialist and Consultant (Ms Kennedy), AEGIS Informatics LLC, Barrington, Rhode Island. The authors declare no conflicts of interest.Correspondence: Dr Collins, Partners Healthcare System, 93 Worcester St, Floor 2, Wellesley, MA 02481 ( [email protected] ). DOI: 10.1097/NNA.0000000000000467 Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved. validated informatics competencies specific to the needs of nurse leaders and reflective of the rapidly changing information technology (IT) environment. 6,7 Background Clancy 8reported on 28 nursing schools identifying significant nursing informatics deficits in faculty com- petencies and curriculum. These deficits may inhibit nursing_s advancements toward future Real-Time Healthcare System (RTHS) defined as providing 4 rights:Bthe right information, at the right time, in the right formatIto the right audience.[ 9Information life-cycle management is a key technology instrumen- tal to the success of the RTHS and is a fundamental component of nursing informatics. 10 Specifically, the RTHS is a next-generation,transformative model for healthcare operations, management, and information technology in whichBchief information officers (CIOs) should activate an engagement plan with the CNO including these steps: Educate the CNO about the major technology breakthroughs that build the RTHS manage- ment paradigm. Review the evolving land- scape of applications, real-time analytics, and infrastructure.

Together, build specificvalue cases that clarify RTHS benefitsVstarting with mature wins, like finding smart pumps, but moving ahead to nurse and patient experience and patient throughput.

Emphasize those cases with a positive impact on nurses"daily work lives, patient satisfaction, and quality of care.

Seek out innovative nurse managers and key influencers of staff attitude to improve cultural readiness. As part of this readiness, create or re- invent the role of chief nursing informatics officer, whose focus goes well beyond support of the EHR.[ 11(p1) These recommendations, aligned with our team_s recent work on nursing informatics governance struc- tures, are especially noteworthy for nursing practice in that they forecast future competencies for many nurse leader roles and will move well beyond the use of EHRs and management of EHR implementation projects, toward a continued focus on improving the patient experience, as well as the nursing care environment through innovative and applied technologies. 12 In fact, Hessels et al 13 reported evidence linking lower 7-day readmissions and shorter length of stay to advanced EHR adoption, while noting that a supportive nursing environment is independentlyand positively linked to better delivery of nursing care and patient satisfaction, inferring a significant role for informatics competen- cies to appropriately integrate technology into nursing practice. Too often IT decisions are left to others as nursing leaders lack the competencies necessary to actively participate in theIT decision-making process. 6 As technologies rapidly expand and new models of healthcare delivery emerge, a new set of competencies are required by today_s nursing professional and are critical for nursing practice leaders to continue to ad- vance and innovate.

To address the gap of a lack of nursing infor- matics competencies for nurse leaders, we conducted a 2-year, multimethod study to define, develop, and validate a nursing informatics competencies self- assessment instrument specific to the needs of nurse leaders to evaluate their levels of nursing informatics competencies and target learning and professional development opportunities. We followed the 8-step scale development process by DeVellis. 14 These steps are summarized in Table 1. For each step, we identi- fied the process step and method(s) utilized: (1) clarify the intended concepts to measure, (2) generate an item pool, (3) determine the format for measurement, and (4) have the initial item pool reviewed by experts, (5) consider inclusion of validated items, (6) administer items to a development sample, (7) evaluate the items Table 1.Eight-Step Scale Development Process and the Corresponding Methodologies Used in Developing the Nursing Informatics Competency AssessmentVNurse Leader Survey Instrument Development Phase Instrument Development Process Steps Method(s) Utilized Competency identification 1. Clarify the intended concepts to measureEnvironmental scan of indexed research databases using key terms to identify initial list of competencies for evaluation 2. Generate an item pool 3. Determine the format for measurement 4. Have the initial item pool reviewed by expertsDelphi study in 3 rounds for content and face validity using a survey instrument and CVI Factor analysis 5. Consider inclusion of validated items Multivoting and survey instrument 6. Administer items to a development sample 7. Evaluate the itemsExploratory factor analysis for scale optimization and factor identification 8. Optimize scale length JONA Vol. 47, No. 4 April 2017213 Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved. (item performance, factor analysis,!), and (8) opti- mize scale length.

We further separated the 8 steps into 2 phases:

competency identification (steps 1-4) and factor anal- ysis (steps 5-8). In order to provide details of the process, we published the study into 2 articles. In this article, we summarize the 1st phase of the scale development process: competency identification. Con- currently, we published the 2nd phase of the pro- cess, factor analysis, into a 2nd publication, entitled BNursing Informatics Competency Assessment for the Nurse Leader: Instrument Refinement, Valida- tion, and Psychometric Analysis,[ 15 which details our development of a final validated instrument to mea- sure informatics competencies specific to the Nurse Leader: Nursing Informatics Competency Assessment for the Nurse Leader.

The aim of this article is to identify the nursing informatics competencies perceived as relevant and required by today_s nurse leader and/or manager. Methods We conducted an environmental scan to identify existing work related to nurse leaders_informatics competencies to inform development of our survey deployed using the Delphi method for nurse leader competency identification.

Environmental Scan We performed an environmental scan during fall 2012, to identify existing work related to nurse leaders_infor- matics competencies (steps 1-3). We searched PubMed and CINAHL for literature with validated competency statements and self-assessment scales and used Google and Google Scholar search engines to find related edu- cational resources and reports. The following search terms were used:Binformatics[and/orBcompetency[ and/orBhealth information technology[and/orBHIT[ and/orBnursing[and/orBself-assessment scale.[ Delphi Survey Method to Establish Content/Face Va l i d i t y In step 4, we aimed to understand the nursing infor- matics competencies perceived as relevant and required by today_s nurse leader and/or manager. We conducted a Delphi survey to seek opinions of nursing leadership about HIT competencies critical for the nurse leader and at the same time establish the content/face validity of the survey instrument. We selected sets of compe- tencies based on results of our environmental scan and closely replicated the methods used by Westra and Delaney. 16 The Delphi survey was developed using REDCap software and included 3 rounds. The survey was distributed using snowball sampling to nurseleaders and managers in the Organization of Nurse Leaders (ONL) (Massachusetts and Rhode Island) via e-mail, leveraging the ONL governance structure (ie, board and committee members) as targeted partic- ipants were also asked to forward the survey within their network of nurse leaders and managers. All study procedures were approved by Partners Healthcare System_s institutional review board.

In rounds 1 and 2, we asked participants to vote on each competency as relevant or not relevant, using a binary response, yes or no. The respondents were also able to include comments as free text, such as suggested changes to the phrasing of the competency in an effort to provide unbiased feedback. This also provided a source of qualitative data for further evaluation of results and clarification of included competencies. For the final survey round, a 4-point Likert scale (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = very relevant) was used so that a more sophisticated analysis and reduction ofthe competencies could be performed. During rounds 2 and 3, the results from the previous round, including all free-text comments, were anonymized and provided to the participants.

A content validity index (CVI) score was calcu- lated to evaluate the ranked relevance of the compe- tencies by participants. 17 The threshold to retain a competency was a CVI greater than 0.80, derived from the Likert scale votes ofBquite important[orBvery important,[and was consistent with the work of Westra and Delaney. 16 We used Kruskal-Wallis anal- ysis of variance to identify differences among groups (ie, different types of organizations and roles). We also evaluated criteria with a borderline CVI threshold of less than 0.80 using qualitative data but did not exclude any items with a CVI of greater than 0.80. Results Environmental Scan In 2002, Staggers et al 18 used the Delphi method to establish the 1st reliable and valid informatics compe- tencies for nurses at 4 levelsof practice (entry level to advanced informatics specialists). Since that research, knowledge related to nursing informatics competen- cies has grown as technology has moved closer to the bedside, and chair side, of clinical practice.

We s t r a a n d D e l a n e y 16 defined competencies for nurse and health leaders in 2008 with a focus on infor- matics knowledge and skills. Gassert 19 recognized the potential of clinical informatics to transform clinical practice with the right tools and skills. Also in 2008, Hart 20 performed a review of the literature in an effort to define those competencies that would advance evidence-based practice and the use of HIT. That research revealed a history of research that focused 214JONA Vol. 47, No. 4 April 2017 Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved. on attitudes and use of e-resources but not necessarily the competencies and skills needed by nurses to work in a technology-rich work environment.

Hart 21 later went on to focus on job-specific nurs- ing informatics competencies for nurse managers, noting that this focus was too narrow because infor- matics competencies were not incorporated into spe- cific jobs for nurses. In 2013, Simpson 7emphasized the importance of job-specific nursing informatics competencies required by the CNE and how the lack of key competencies was a barrier to full engagement in HIT decision making. Some of his key findings went on to reveal that their role is often tactical (ie, func- tional) and that the CNE is often in aBrecommender[ role. The study also revealed an informatics compe- tency deficit related to societal and technical trends as they relate to nursing. 7The need for job-specific com- petencies continues to be recognized as a need for the future of nursing. 2Based on these findings, we selected Westra and Delaney_s 16 competencies with those more recently identified by Simpson 7as the initial set of 108 competencies in the item pool for the Delphi survey.

Delphi Study to Establish Content/Face Validity The Delphi study was performed to establish content/ face validity of items that would later be used in the development of a validatedself-assessment instru- ment. Three rounds of surveys were conducted: (1) June to July 2013, (2) September to October 2013, and (3) December 2013 to January 2014 (Figure 1). A high-level summary of the sample population and initial findings are available online (http://www.himss.org/ ni-impact-survey; HIMSS 2015 Nursing Informatics Symposium). 22 There were 34 participants in round 1, 26 participants in round 2, and 41 participants in round 3 (Table 2). Most participants were execu- tives, followed by directors and then managers and then those who were holding master_sordoctoral degrees. The largest number of respondents came from community hospitals and then from academic medical centers. When asked about their HIT training, mostrespondents replied that their knowledge came from Bon-the-job[training or self-learning.

The Delphi survey began with 108 competency items. The list was reduced to 98 items when 10 items were excluded after round 1 and further reduced to 92 items when 6 additional items were excluded after round 2. During rounds 1 and 2, we rephrased 29 competencies based on feedback from the respon- dents. After the last round, a total of 74 competencies were retained when 18 items were excluded because of having a CVI of less than 0.80 (Figure 1).

The 74 retained competencies mapped into 15 broad informatics categories, based on modified Westra and Delaney_s 16 competency categories (Table 3).

The top 15 competencies, in detail, were ranked by priority (Table 4). The 1st 2 items received 100% of the votes asBquite[orBvery relevant[: (1) the ability to ensure that nursing values/requirements are represented in HIT selection and evaluation and (2) inclusion of nursing information within HIT systems. The subsequent items focused on collabo- ration, communication, and advocacy. Discussion This initial study identified 74 competencies with a CVI of greater than 0.80 specific to nurse leaders and nurse managers from an initial list of 108 competen- cies. The large number of competencies identified as Bimportant[speaks to the urgent need for informat- ics education and the increasing complexity of the nursing leadership role. We observed a changing emphasis on competencies focused on outcomes, quality, and cost compared with studies conducted in past years 16; these new areas of focus could be attributed to an ever increasing volume of patient data that require new tools for nurse managers to use to effectively evaluate care. Skills in managing large outcomes, quality, and cost data sets are needed to ensure nursing isBat the table[and is able to advocate for nursing-specific needs. The current on-the-job nature of informatics education identified in the survey responses is inadequate to meet this need. Too often, HIT leadership and strategic decisions are based within IT versus at the provider leadership level, although this trend may be changing. 12 Nursing must be in a position to lead and participate in HIT selection, development, and optimization. Our data showed that this need is not limited to large health systems but is consistent across all organization types and nursing leadership roles. There is a clear educational gap around nursing informatics exemplified by the large number of com- petencies identified in ourdata and the majority of on- the-job training noted. A validated self-assessment tool, the next step in our research, will provide nursing with Figure 1. Delphi study: results after each round.

Practice Implications for the Nurse Executive It is clear that nursing informatics and HIT knowl- edge should no longer be delegated to a specialist, but should be an expected core competency of profes- sional practice. This research engaged expert nurse leaders to identify areas of nursing informatics that are important in their practice as leaders. Identifica- tion of these areas can bring greater specificity, in discussion with the information technology commu- nity, about domains that are ripe for development.

This work can also provide a foundation for engage- ment plans that are developed and revised to support the healthcare deliverysystemoftomorrow.

The competencies identified here have application for the design of residency programs, continuing edu- cation programs, and curriculum development for nurse executives, leaders, and nurse managers. The level of Table 2.Delphi Round Sample Sizes and Demographics Delphi Round SamplesRound 1 Round 2 Round 3 Total n = 34 Total n = 26 Total n = 41 Participant characteristics Highest education level Doctoral 7 (20.6%) 4 (16%) 11 (26.8%) Master 24 (70.6%) 21 (84%) 23 (56.1%) Bachelor 2 (5.9%) 0 (0%) 5 (12.2%) Associate 1 (2.9%) 0 (0%) 2 (4.9%) Years of experience 925 14 (41.2%) 15 (60%) 19 (46.3%) 16-25 11 (32.4%) 4 (16%) 8 (19.5%) 11-15 3 (8.8%) 2 (8%) 8 (19.5%) 6-10 5 (14.7%) 3 (12%) 6 (14.6%) 3-5 1 (2.9%) 1 (4%) 0 (0%) 1-2 0 (0%) 0 (0%) 0 (0%) G1 0 (0%) 0 (0%) 0 (0%) Role Executive 18 (52.9%) 10 (40%) 13 (31.7%) Director 8 (23.5%) 7 (28%) 13 (31.7%) Manager 6 (17.6%) 6 (24%) 11 (26.8%) Staff nurse 0 (0%) 1 (4%) 0 (0%) Researcher 0 (0%) 0 (0%) 1 (2.4%) Other 2 (5.9%) 1 (4%) 4 (9.8%) Self-rated HIT knowledge Above average 13 (39.4%) 11 (44%) 23 (59%) Average 16 (48.5%) 14 (56%) 15 (41%) Below average 4 (12.1%) 0 (0%) 0 (0%) HIT education received Formal education 5 (14.7%) 7 (28%) 12 (29.3%) On-the-job training 33 (97.1%) 22 (88%) 40 (97.6%) Self-learner 16 (47.1%) 11 (44%) 22 (53.7%) Other 0 (0%) 0 (0%) 2 (4.9%) Participants_organization characteristics Type of organizationCommunity Hospital 17 (50%) 10 (40%) 16 (39%) Critical-access hospital 2 (5.9%) 0 (0%) 2 (4.9%) Academic medical center 10 (29.4%) 9 (36%) 11 (26.8%) Integrated delivery network 2 (5.9%) 4 (16%) 4 (9.8%) Academic/university 0 (0%) 1 (4%) 4 (9.8%) Other 3 (8.8%) 2 (8%) 6 (14.6%) MagnetistatusYes 6 (17.6%) 6 (24%) 6 (14.6%) No 28 (82.4%) 19 (76%) 35 (85.4%) Percent for each item may not sum to 100% because of multiselect answer choice options. Total counts for each item may not sum to total sample because of missing responses. Ta b l e 3 .Categories of Competencies Retained in Round 3 Results (CVI90.80) CategoryNo. of Competencies Retained Management concepts 9 Requirements and system selection 9 Ethical/legal concepts 8 Information systems concepts 7 Advanced software applications 6 Executive leadership 5 Financial 5 Implementation/management 5 Patient-related applications 5 Data issues 4 Technical knowledge 4 Collaboration 2 Electronic communications 2 HIT selection 2 Standardization 1 Total 74 216JONA Vol. 47, No. 4 April 2017 Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved. detail within the description of each competency pro- vides additional opportunities to close education and skill gaps as we look to support ongoing professional education, optimize HIT to improve patient care out- comes, enhance the patient experience, and optimize nursing workflow. In addition to psychometric valida- tion of these competencies, future work should explore opportunities to develop interprofessional programs and partnerships with organizational CIOs to identify and specify nursing requirements for tomorrow_sreal- time healthcare system (ie,value cases) and evaluation of EHR impact on outcomes and patient satisfaction.

Limitations Our findings are limited to the experiences of the survey responders who may have a specific interest in infor- matics or an already identified need. To limit bias, we used a comprehensive compilation of existing com- petencies identified from our environmental scan. In addition, no new competencies were identified by res- pondents during the Delphi process. We are unable toreport a response rate because of our snowball sam- pling methodology. Conclusions This research begins to identify the competencies nurse leaders have identified as relevant to their practice in today_s rapidly evolving, technology-rich healthcare delivery system. The nursing profession is quickly moving beyond EHR implementation work as a major focus toward the optimal use of EHR tools and data. We e xpe ct that as information technology continues to mature nursing practice will move beyondBon-the-job infor- matics training[to a more competency-based model of nursing informatics education and practice. This com- petency identification research can serve as a pragmatic foundation to advance nursing informatics practice.

Acknowledgments The research team thanks the executive leadership, com- mittees, and membership of ONL (Massachusetts, Rhode Island, New Hampshire, Connecticut) for their support.

References 1. Payne TH, Corley S, Cullen TA, et al. Report of the AMIA EHR- 2020 Task Force on the status and future direction of EHRs.

J Am Med Inform Assoc. 2015;22(5):1102-1110.

2 . Institute of Medicine.The Future of Nursing: Leading Change, Ad- vancing Health. Washington, DC: The National Academies Press; 2011.3. American Organization of Nurse Executives. AONE Salary and Compensation Study for Nurse Leaders, Executive Summary [Internet]. 2013 [cited December 29, 2015]. p10. http:// wwwaoneorg/resources/nurse-leaders-compensation-summarypdf.

Accessed January 20, 2017.

Table 4.Top 15 Competency Priority Ranking Final Competencies to Be Retained by PriorityNot Relevant, %Somewhat Relevant, %Quite Relevant, %Ve r y Relevant, % CVI 1. Ability to ensure that nursing values/requirements are represented in HIT selection and evaluation0 0 35 65 1 2. Inclusion of nursing information within HIT system 0 0 22 78 1 3. Budgeting using technology 0 2 39 59 0.98 4. Data-based planning and decision making through the utilization and synthesis of HIT system data0 3 54 44 0.98 5. Ability to collaborate with other departments regarding project management and resource allocation for HIT system implementation0 3 47 50 0.97 6. Ability to collaborate with chief medical officer peers related to HIT and needs of nurses and physicians0 3 37 61 0.98 7. Ability to collaborate with interprofessional team in HIT selection process0 3 38 59 0.97 8. Ability to advocate for the development (or purchase) and use of integrated, cost-effective HIT systems within the organization0 3 35 62 0.97 9. Communicating a system and nursing vision about the benefits of HIT0 3 30 68 0.98 10. Ability to involve frontline staff in the evaluation of HIT systems related to their practice0 3 32 65 0.97 11. Ability to involve frontline staff in the development of HIT system requirements0 3 24 73 0.97 12. Ability to involve frontline staff in appropriate aspects of HIT design, implementation, and testing related to their practice0 3 24 73 0.97 13. Ability to see HIT as a top priority and strategic decision 0 3 31 67 0.98 14. Recognition of value of clinicians_involvement in all appropriate phases of HIT0 5 36 61 0.97 15. Quality assurance using technologyV5 38 58 0.96 JONA Vol. 47, No. 4 April 2017217 Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved. 4. Crow G. Diffusion of innovation: the leaders_role in creating the organizational context for evidence-based practice.Nurs Adm Q.

2006;30(3):236-242.

5. Shirey MR. Evidence-based practice: how nurse leaders can facilitate innovation.Nurs Adm Q. 2006;30(3):252-265.

6. Remus S. Advancing the digital health discourse for nurse leaders.Stud Health Technol Inform. 2016;225:412-416.

7. Simpson RL. Chief nurse executives need contemporary informatics competencies.Nurs Econ. 2013;31(6):277-287; quiz 288.

8. Clancy T.Integrating AACN Essentials, QSEN KSAs, and TIGER Competencies for Nursing Informatics.Orlando,FL:

National Nursing Informatics Deep Dive Program; 2015.

9. KHare S. Gartner Symposium/ITxpo 2014: Key Tenets of a Real- Time Healthcare System (RTHS) [Internet]. MicroPact.com.

2014 [cited June 18, 2016]. http://www.micropact.com/blog/ detail/key-tenets-of-a-real-time-healthcare-system-rths/.

Accessed January 20, 2017.

10. American Nurses Association.Nursing Informatics: Scope and Standards of Practice. 2nd ed. Silver Spring, MD: nursesbooks.

org; 2015.

11. Shaffer V, Runyon B. Healthcare Provider CIOs Must Boost CNO Collaboration to Materialize the Real-Time Health System [Internet].Gartner, Inc. 2016. https://www.gartner.com/ doc/3241419/healthcare-provider-cios-boost-cno. Accessed January 20, 2017.

12. Collins SA, Alexander D, Moss J. Nursing domain of CI governance: recommendations for health IT adoption and optimization.JAmMedInformAssoc. 2015;22(3):697-706.

13. Hessels A, Flynn L, Cimiotti J, Bakken S, Gershon R. Im- pact of Heath Information Technology on the Quality of Patient Care [Internet].Online Journal Nursing Informatics.

2015;19 [cited May 5, 2016]. http://www.himss.org/impact-heath-information-technology-quality-patient-care. Accessed January 20, 2017.

14. DeVellis RF.Scale Development: Theory and Applications.

Thousand Oaks, CA: Sage Publications; 2011.

15. Yen P, Phillips A, Kennedy M, Collins S. Nursing Informatics Competency Assessment for the Nurse Leader: Instrument Refinement, Validation, and Psychometric Analysis.J Nurs Adm. 2017, in press.

16. Westra BL, Delaney CW. Informatics competencies for nurs- ing and healthcare leaders. In:AMIA Annual Symposium Pro- ceedings of the American Medical Informatics Association Annual Fall Symposium, 8Y12 November 2008. Washington DC; 2008:

804-808.

17. Polit DF, Beck CT. The content validity index: are you sure you know what_s being reported? Critique and recommendations.

ResNursHealth. 2006;29(5):489-497.

18. Staggers N, Gassert CA, Curran C. A Delphi study to deter- mine informatics competencies for nurses at four levels of practice.Nurs Res. 2002;51(6):383-390.

19. Gassert CA. Technology and informatics competencies.Nurs Clin North Am. 2008;43(4):507-521.

20. Hart MD. Informatics competency and development within the US nursing population workforce: a systematic literature review.Comput Inform Nurs. 2008;26(6):320-329.

21. Hart MD. A Delphi study to determine baseline informatics competencies for nurse managers.Comput Inform Nurs. 2010; 28(6):364-370.

22. Collins S, Kennedy M, Phillips A, Yen P. Nursing informat- ics competencies for nurse leaders/managers: a Delphi study.

HIMSS 2015 Nursing Informatics Symposium. Chicago, IL:

Healthcare Information and Management Systems Society (HIMSS); 2015:1. http://s3.amazonaws.com/rdcms-himss/files/ production/public/2015Conference/handouts/PSNI13.pdf.

Accessed March 4, 2017.