Professional Portfolio

Running Head: PART 6 EVALUATION PLAN

Evaluation Plan (Portfolio)

Anne Marie Wouapet

Walden University

NURS 6431 Section 01 System Design, Planning and Evaluation

February 5, 2017

Evaluation Plan

An evaluation plan forms an integral part of a project proposal that provides important information to improve a project during implementation. Evaluation plan becomes an important tool when examining the usability and effectiveness of new technology in a health care facility. The paper involves the PICO question that aims at evaluating the effectiveness of the use of the EHR in clinical documentation. The purpose of this paper is explaining the evaluation plan, performing the literature review, assessment methodology, assessment tool, discussion of ethical issues in the implementation of the new system. More so, there is summarizing of the criteria used in defining the success of the plan, outline regarding the scope limitations, and the opportunities coming from the evaluation plan.

PICO Question

The PICO question is as follows: Does implementing a new unified acute and ambulatory Electronic Health Record (EHR) system within the hospital, compared to when they are not used improve the quality of healthcare for the patients through documentation? P (problem) is the quality of patient care through documentation, I (intervention or indicator) is implementing the new EHR system, C (comparison) is no implementation of the new system having other consequences, and O (outcome) is the improved quality of patient care.

An Explanation of the Goals of the Evaluation Plan

The evaluation plan seeks to answer the PICO question stated in this document. When implementing a new technology, the aim is to solve a given problem or seal a gap identified in the area of application. Documentation is a key role for nursing. Poor documentation leads to medication errors. Medication errors should not be ignored in any health care setting. They can cause harm that can also escalate to costing the patient’s life as well as costing the facility thousands to millions of dollars in a year. Research has shown that medication errors tend to decline by as much as 85% when health care services are automated, or appropriate technology is put in place (Seibert, Maddox, Flynn & Williams, 2014). Electronic Medical Records system is one of the most widely used systems to eliminate these errors. In some cases, Clinical Decision Support systems are integrated with EHRs to provide a more reliable source of information needed for real-time decision making (Gschwandtner, Kaiser & Miksch, 2011). The adoption of these systems in the ambulatory care setting is gaining popularity in most facilities. However, it remains unclear if the implementation of such systems would improve quality of care through documentation.

Literature Review

The first peer-reviewed article titled Advanced Practice Nurses’ Meaningful Use of Electronic Health Records focuses on the increasing role of EHRs in the US healthcare system. It analyses the processes and mechanisms in place to assess the use of EHRs in modern healthcare procedures (Narcisse, Kippenbrock, Odell & Buron, 2013). It includes systematic analysis reviewing the effectiveness of procedures using Electronic Health Records (Rozas & Klein, 2010). The article concludes by recognizing the fact that the implementation of Electronic Health Records is an important step in ensuring that disease management occurs.

Penoyer et al., (2014), looked at the use of EHR systems in the hospital system, as well as issues that may affect its adoption in healthcare institutions. The article concludes that it is necessary for organizations to review EHR processes and ensure that structures are put in place for better functioning in healthcare processes. It furthermore interprets a survey carried out by healthcare professionals to assess the effectiveness of the study. Dolezel and Moczygemba (2015), analyzed the implementation of EHR systems in physician practices. It incorporates analytical data to assess the efficiency of the adaptation of these systems. The article concludes by explaining the fact that though the transition to EHR may be challenging, it is a necessary process in the current healthcare setting.

McAlearney, Hefner, Sieck, Rizer and Huerta (2014), assessed the implementation of Electronic Health Record with interviews carried out on several participants. It includes research into the successful management of ambulatory EHR implementation. It uses evidence from surveys carried out, and it gives more insight into practices that can improve the use of these systems in healthcare centers. Hessels et al., (2015) through their article, analyzed the relationship between the adoption of Electronic Health Records and the rate of satisfied patients. Their analysis highlights the success of the implementation of this technology on improving healthcare practices. It concludes that the adoption of Electronic Health Records in hospitals was associated with a shortened length of stay in the various hospitals. Kirkendall, Goldenhar, Simon, Wheeler and Andrew Spooner (2013), analyzed the effectiveness of the transformation from traditional data recording methods to the use of EHR in medical practices, through detailed research. The study revealed the importance of a smooth transition to the use of modern technology, especially through practices such as training. It reveals that though many healthcare professionals embrace the use of EHR, not many are knowledgeable concerning its application in healthcare. See Appendix for more details.

Evaluation Methodology (Including Research Design)

The evaluation methodology to be used to answer the PICO question is cognitively-based, which requires expertise in medicine as well as the human-computer interaction. Participants from both teams will collaborate on the pragmatic and theoretical aspect of the evaluation process. The evaluation process will be divided into four sections: profiling testers, designing scenarios, creating an evaluation form and designing questionnaire for the evaluator (Centers for Disease Control, n.d.). The purpose of profiling testers is to identify and select potential users of the EHR system. Four panels of users will be designed based on several requirements from information technology experts and health care professionals.

Additionally, a research design is an overall strategy chosen for integration of various components of study in a logical and coherent way, hence ensuring effectiveness in addressing the PICO question. The design comprises the blueprint regarding collecting, measurement as well as the data analysis. The design involved is the peer-reviewed and the predetermined process derived from the identification of research or PICO question, the study protocol, analysis, and the interpretation of the results. The sources of information are the peer review groups that help in identifying research materials through electronic databases, pursuing references, experts’ advice, and hand-searching journals. The selected literature is based on medical history information and predictions. The data is mainly collected from the experimental data, by using the data extraction forms. Any qualitative data is usually coded based on medical documentation reviews for validity, and robustness (Timmins, 2013).

Evaluation Tool

The chosen tool for evaluation is the “Electronic Health Record End User Survey” (Agency for Healthcare Research and Quality, n.d.). The tool is a questionnaire that focuses on the usability of an EHR. The questionnaire is designed for the clinical staff in the ambulatory setting to evaluate the usability of an electronic health record in ambulatory care. The aim of the assessment tool is to measure the appropriateness of ambulatory care after the implementation of clinical documentation. The device involves various types of a survey that incorporate many stakeholders who ensure that the hospital adopts new technology relating to the improvement of health care within the hospital. The tool is associated with a survey tool for assessing the EHR implementation based on development initiatives guide. The EHR End User Survey measures the effectiveness realized in the hospital setup through documentation as compared to using the old system of documentation (Agency for Healthcare Research and Quality, n.d.). Based on the developed PICO question that aims at evaluating the benefits that subsume the overtaken documentation. The evaluation tool captures various hospital domains including the end users feedback regarding training and competency, usefulness, usability, infrastructure, and the user support (Kaphingst et al., 2012). The tool involves the validation efforts based on needs assessment, the pilot study and the analysis of the nurse respondents. The End User Survey tool based on the EHR provides questionnaire type of review where the clinical staff answer the asked questions focusing on the current state assessment and usability within the hospital. The remote documentation applicable to the new unified ambulatory system makes it easier and efficient since it increases the number of patients handled at the same time. The tool when applied to measure the comparison will provide information based on the questions developed by the clinical team and given to selected stakeholders who give independent information on the success or failure realized (Agency for Healthcare Research and Quality, n.d.). Additionally, the tool looks impressive due to several reasons ranging from the clarity of questions to simplicity of the questions. Such a structured questionnaire permit respondents time to evaluate their responses carefully without interference from the researcher. Moreover, the tool has allowed for both subjective and objective questions. The questionnaire is framed in with questions in such a way that it can gather quantitative data which could have been difficult to collect otherwise. Also, most of the questions are directly related to my PICO question which guides the research.

Ethical Issues that may Arise and How to Handle them

Ethical conflict over the appropriateness of use of personal health records has been an issue of concern. Unauthorized and inappropriate disclosures of this information have occurred as long as detailed records have been kept. The privacy and confidentiality of data, as well as the security of health information systems, are protected by policies set by the relevant agencies. Health care facilities are required to protect data from unauthorized use and disclosure or destruction. As health information systems and databases penetrate in the health care sector, the chances of unethical disclosure of personal information will continue to increase accordingly.

Nursing informaticists are charged with protecting and safeguarding patients' information specially the data stored in the electronic health record systems (Berner, 2008). The federal legislation also legally protects patient's rights to privacy and confidentiality regarding health information. Hospitals and health care professionals are expected to exercise care when collecting and disclosing personal as well as health care information. Nursing informaticists have an ethical obligation to protect information obtained from the nurse-client relationship. When a patient entrusts their health records to the health care facility, they expect it to be kept confidential. The Nursing Informaticist is responsible for providing a health information system and support that meet privacy and confidential guidelines to ensure possible seal case of misuse of disclosure of personal information.

Moreover, the clients must be informed of their rights concerning their personal information and have consent when the information is being collected, used and disclosed to another party (Goldstein, 2010). However, the health care professionals have the right to share with other health care team. The client should be explained that the information will be shared and inform them who will receive the information on the other end. Nevertheless, nurses have ethical obligation to disclose in cases which involve risk of significant harm to the safety of the client.

Strategies for using Evaluations to Address Ethical Issues

Evaluation of Electronic Health Records can be used to address the issues of disclosure, loss or misuse of personal information. The evaluation tool used in the evaluation process and also utilized in the evaluation methodology should integrate sections to predict possible cases of violation of privacy and confidentiality. The questionnaire is one of the widely-used evaluation; the tool allows the researcher to use structured questions to get input from both the patients/clients and the nurses. There is a limit to what can be shared and what should remain private and confidential. However, drawing the line between the two remains a subject of discussion. Privacy can be addressed by ensuring that the system implemented does not eliminate the client's right to make decisions about how their personal information is shared (Goodman, et al., 2011). Another important aspect to include in the evaluation process is a criterion to determine if the health care professionals and clients understand privacy rights outlined in federal and state statutes, court decisions, and professional code of ethics.

During the evaluation, the performance and ethical issues are looked at the developed policies to identify the compliance or otherwise regarding the system. The system, when seen to be much vulnerable in the exposure of health information, is usually subjected to improvements to maintain the initial goal of enhancing confidentiality and privacy. Counterchecking the health records of the system and that in the record book also helps in evaluating any manipulations that may have been made in case the confidentiality was minimal. The records should be at par to conclude that the system is promoting privacy and confidentiality. Another important strategy involves the segmentation of the patient information to ensure that majority of information that is much sensitive becomes difficult to access (Rothstein, 2010).

Criteria Used to Define the Success of the Plan and Disseminating Findings

The structured questionnaire will be helpful in the evaluation project since my PICO question involves measurement of different constructs of the respondents’ feelings, suggestions, opinions, and other related aspects. Most of these constructs are perceived to be unobservable individual characteristics that cause variation in behavior. The items in the questionnaire will be phrased in such a way that they pose one evaluation characteristic per item so that the respondents will take minimal time while responding to the questions. The aim is to ascribe quantitative value to qualitative data so that the items can be amenable to statistical analysis. More so, the end user survey tool promotes the use of advance technology (electronic health records) hence becoming useful for the evaluation as it saves time and related resources for the hospital (Timmins, 2013).

More so, the survey becomes the standard method relating to the collection of data. The survey tool is flexible, comfortable during implementation, and offering unlimited data range hence providing reliable results for the new mode of documentation within the hospital. Another reason for selecting the tool is because it measures the change-over-time effects of the clinical documentation. It helps in monitoring pre-system conditions against the post-system conditions hence gathering extensive feedback (Penfold et al., 2011).

The outcome goals formulated in the earlier stages of the project will form the basis for determining the success of the evaluation tool. The deliverables for the project are the evaluation goals. Thus, they should match the goals of the end-users which should in term match those of the EHR implementation in acute and ambulatory care (Seto, Foisy, Arkison, Klassen & Williams, 2012). Additionally, the evaluation will be based on clinical outcome measures, clinical processes, patient care quality, provider adoption of EHR, formulate evaluation and attitude measures tool (Friedman & Wyatt, 2010). These items will be factored in the matrix to facilitate the success of the evaluation.

Limitations to the Scope of the Plan and Resulting Opportunities

To develop an effective evaluation plan, the researcher should understand the stakeholders involved in the study, how the implementation of electronic health record (EHR) will affect their environment, and determine the criteria for defining success. The method applies a questionnaire while focusing on the EHR usability. The plan is designed for the clinical staff to evaluate the effectiveness of the EHR in the ambulatory setting. Despite the appropriateness of the established and subsequent implementation of the clinical documentation, the plan suffers some various limitations. Bloomrosen et al., (2011), reported that the response rate for questionnaires can be as low as 20%. The survey becomes inadequate in understanding particular information since it does not provide the opportunity to see the change of the emotions, feelings, and the behaviors of the users during the evaluation (Bloomrosen et al., 2011). More so, the EHR end user survey does not measure the truthfulness of the respondent when measuring the effectiveness of the newly implemented system within the hospital. It becomes difficult in knowing how much the clinical staff has thought while providing the response in the questionnaire hence lacking reliability. The survey plan also suffers the limitation of being evaluator-imposed. It means that the evaluator tends to make decisions on what to include in the review hence may end up missing some critical information regarding the clinical documentation by the system. The clinical staff may be forgetful while answering the questions relating to the usability of the ambulatory system hence providing incomplete feedback about the system (Romano & Stafford, 2011).

Future Opportunities Building upon the Evaluation Plan.

Based on the limitations of the evaluation plan discussed above, there are also some future opportunities for building upon the EHR End User Survey. One opportunity would involve automating the survey such that the user will be able to access the evaluation plan while still in the hospital. It will help the clinical staff not to forget any crucial information when the assessment is being done. More so, the survey tool would be made valid by ensuring the clinical staff providing responses have a prior chance of personal study of the usability of the system. He or she electronically verifies some information relating to the unified system hence ending up providing reliable feedback concerning the usability of the system. Another possible opportunity to build the survey evaluation plan is providing an opportunity to train the clinical staff on handling the new documentation strategies before fully adopting the system. It will help them efficiently learn how to use it and therefore provide complete, and timely feedback during the evaluation (Jimmieson & Griffin, 1998).

Summary

To sum it up, a cognitive-based assessment methodology will be valuable for EHR evaluation. The evaluation method focuses on usability, and it only tests with the potential users of the system. A cognitive approach is necessary when designing the panel of users, evaluation forms, scenarios, and questionnaires. Regarding the EHR, it will be required to study patients and other users exposed to such technological tool as well as studying patient-doctor relation over a period (Stroud & Gansauer, n.d.). It will ensure credibility and reliable results. An evaluation tool forms part of the essential basis for a sound research. The tool selected should be tested for reliability and validity. Additionally, during the assessment process, the questionnaires should be designed in a manner that the questions eliminate possible bias. The articles conclusively agree to the fact that EHRs improve outcomes in healthcare institutions and furthermore improve the quality of care that the patients receive (Scutt, 2008). It is also mentioned that it is important to develop ambulatory EHR, to ensure efficient practices. Therefore, it is necessary to ensure proactive implementation in healthcare facilities for improved healthcare processes. Privacy and confidentiality of the EHR (electronic health records) have a significant impact to the nursing informaticists. The transfer of health data to the system being implemented should be made gradually and with a lot of accuracies. Identifying both opportunities, and limitations becomes paramount while evaluating a new system within the hospital setup or any organization. It helps in creating possible strategies that will contribute to avoid unintended consequences that may lead to failure of the system.

Appendix

References

Agency for Healthcare Research and Quality. (n.d.). Health IT survey compendium. Retrieved from http://healthit.ahrq.gov/portal/server.pt/community/health_it_tools_and_resources/919/health_it_survey_compendium/27874

Berner, E. S. (2008). Ethical and legal issues in the use of health information technology to improve patient safety. HEC Forum, 20(3), 243–258. doi:10.1007/s10730-008-9074-5

Bloomrosen, M., Starren, J., Lorenzi, N. M., Ash, J. S., Patel, V. L., & Shortliffe, E. H. (2011). Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. Journal of the American Medical Informatics Association, 18(1), 82-90. DOI: https://doi.org/10.1136/jamia.2010.007567

Centers for Disease Control. (n.d.). Evaluation Planning: What is it and how do you do it?

Retrieved from: http://www.cdc.gov/healthcommunication/research/evaluationplanning.pdf

Dolezel, D., & Moczygemba, J. (2015). Implementing EHRs: An exploratory study to examine current practices in migrating physician practice. Perspectives in Health Information Management, 12(Winter). Retrieved from: http://perspectives.ahima.org/implementing-ehrs-an-exploratory-study-to-examine-current-practices-in-migrating-physician-practice/

Friedman, C. P., & Wyatt, J. C. (2010). Evaluation methods in biomedical informatics (2nd ed.). New York, NY: Springer Science+Business Media, Inc.

Goldstein, M. M. (2010). Health information technology and the idea of informed consent. Journal of Law, Medicine & Ethics, 38(1), 27–35. DOI: 10.1111/j.1748-720X.2010.00463.x

Goodman, K. W., Berner, E. S., Dente, M. A., Kaplan, B., Koppel, R., Rucker, D., et al. (2011). Challenges in ethics, safety, best practices, and oversight regarding HIT vendors, their customers, and patients: A report of an AMIA special task force. Journal of the American Medical Informatics Association, 18(1), 77–81. DOI: https://doi.org/10.1136/jamia.2010.008946

Gschwandtner, T., Kaiser, K., & Miksch, S. (2011). Information requisition is the core of guideline-based medical care: Which information is needed for whom? Journal of Evaluation in Clinical Practice, 17(4), 713–721. DOI: 10.1111/j.1365-2753.2010.01527.x

Hessels, A., Flynn, L., Cimiotti, J. P., Bakken, S., & Gershon, R. (2015). Impact of Heath Information Technology on the Quality of Patient Care. On-line journal of nursing informatics, 19. Retrieved from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5001503/

Jimmieson, N. L., & Griffin, M. A. (1998). Linking client and employee perceptions of the organization: A study of client satisfaction with health care services. Journal of Occupational and Organizational Psychology, 71(1), 81–96. DOI: 10.1111/j.2044-8325.1998.tb00664.x

Kaphingst, K. A., Kreuter, M. W., Casey, C., Leme, L., Thompson, T., Cheng, M. R., ... & Culbert, A. (2012). Health Literacy INDEX: development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials. Journal of health communication, 17(sup3), 203-221. DOI: http://dx.doi.org/10.1080/10810730.2012.712612

Kirkendall, E., Goldenhar, L., Simon, J., Wheeler, D., & Andrew Spooner, S. (2013). Transitioning from a computerized provider order entry and paper documentation system to an electronic health record: Expectations and experiences of hospital staff. International Journal of Medical Informatics, 82(11), 1037-1045. Doi: http://dx.doi.org/10.1016/j.ijmedinf.2013.08.005

McAlearney, A., Hefner, J., Sieck, C., Rizer, M., & Huerta, T. (2014). Evidence-based management of ambulatory electronic health record system implementation: An assessment of conceptual support and qualitative evidence. International Journal of Medical Informatics, 83(7), 484-494. DOI: http://dx.doi.org/10.1016/j.ijmedinf.2014.04.002

Narcisse, M., Kippenbrock, T., Odell, E., & Buron, B. (2013). Advanced Practice Nurses' Meaningful use of electronic health records. Applied Nursing Research, 26(3), 127-132. DOI: http://dx.doi.org/10.1016/j.apnr.2013.02.003

Penfold, R. B., Kullgren, J. T., Miroshnik, I., Galbraith, A. A., Hinrichsen, V. L., & Lieu, T. A. (2011). Reliability of a patient survey assessing cost-related changes in health care use among high deductible health plan enrollees. BMC Health Services Research, 11(1), 133–143. DOI: 10.1186/1472-6963-11-133

Penoyer, D., Cortelyou-Ward, K., Noblin, A., Bullard, T., Talbert, S., & Wilson, J. et al. (2014). Use of Electronic Health Record Documentation by Healthcare Workers in an Acute Care Hospital System. Journal of Healthcare Management, 59(2), 130-144. Retrieved from: http://web.b.ebscohost.com.ezp.waldenulibrary.org/ehost/pdfviewer/pdfviewer?sid=157cf576-cbb4-4db3-926d-30a76f07b39c%40sessionmgr104&vid=1&hid=128

  Romano, M. J., & Stafford, R. S. (2011). Electronic health records and clinical decision support systems: Impact on national ambulatory care quality. Archives of Internal Medicine, 171(10), 897–903. doi:10.1001/archinternmed.2010.527

Rozas, L. W., & Klein, W. C. (2010). The value and purpose of the traditional qualitative literature review. Journal of Evidence-Based Social Work, 7(5), 387–399. DOI: http://dx.doi.org/10.1080/15433710903344116

Scutt, D. (2008). How to review literature. Radiologic Technology, 79(4), 306–308. Retrieved from: http://web.b.ebscohost.com.ezp.waldenulibrary.org/ehost/pdfviewer/pdfviewer?sid=e19434f1-fcaa-48e1-b856-5cf6bb14cdd1%40sessionmgr104&vid=1&hid=130.

Seibert, H. H., Maddox, R. R., Flynn, E. A., & Williams, C. K. (2014). Effect of barcode technology with electronic medication administration record on medication accuracy rates. American Journal of Health-System Pharmacy, 71(3), 209-218. DOI: 10.1177/0193945914566641

Seto, I., Foisy, M., Arkison, B., Klassen, T., & Williams, K. (2012). The evaluation of an evidence-based clinical answer format for pediatricians. BMC Pediatrics, 12, 34–41. DOI: 10.1186/1471-2431-12-34

Stroud, S., & Gansauer, L. (n.d.). Nursing evidence-based nursing practice tool kit: Practice, evidence, and translation process. Spartanburg Regional Health Care System. Retrieved from: https://www.spartanburgregional.com/app/files/public/537/Nursing-evidence-based-practice-tool-kit.pdf

Timmins, F. (2013). Nursing Research Generating and Assessing Evidence for Nursing Practice. Nurse Education in Practice, 13(6), e29. DOI: http://dx.doi.org/10.1016/j.nepr.2013.04.001.