Select one current, qualitative or mixed methods scholarly nursing article related to your PICOT question and determine its strengths, limitations, and potential application. Complete the Johns Hopkin
Johns Hopkins Nursing Evidence-Based Practice
Appendix E
Research Evidence Appraisal Tool
Evidence level and quality rating: | |
Article title: | Number: |
Author(s): | Publication date: |
Journal: | |
Setting: | Sample (composition and size): |
Does this evidence address my EBP question? Yes No-Do not proceed with appraisal of this evidence | |
Is this study: QuaNtitative (collection, analysis, and reporting of numerical data) Go to Section I: QuaNtitative QuaLitative (collection, analysis, and reporting of narrative data) Go to Section II: QuaLitative Mixed methods (results reported both numerically and narratively) Go to Section III: Mixed Methods |
Section I: QuaNtitative | |||
Level of Evidence (Study Design) | |||
Is this a report of a single research study? A |
|
| |
1. Was there manipulation of an independent variable? |
|
| |
2. Was there a control group? |
|
| |
3. Were study participants randomly assigned to the intervention and control groups? |
|
| |
If Yes to questions 1, 2, and 3, this is a randomized controlled trial (RCT) or experimental study. | LEVEL I | ||
If Yes to questions 1 and 2 and No to question 3 or Yes to question 1 and No to questions 2 and 3, this is quasi-experimental. | LEVEL II | ||
If No to questions 1, 2, and 3, this is nonexperimental. | LEVEL III | ||
Study Findings That Help Answer the EBP Question | |||
Skip to the Appraisal of QuaNtitative Research Studies section |
Section I: QuaNtitative (continued) | ||
Is this a summary of multiple sources of research evidence? |
|
|
1. Does it employ a comprehensive search strategy and rigorous appraisal method? If this study includes research, nonresearch, and experiential evidence, it is an integrative review (see Appendix F). |
|
|
2. For systematic reviews and systematic reviews with meta-analysis B | ||
| LEVEL I | |
| LEVEL II | |
| LEVEL III | |
A systematic review employs a search strategy and a rigorous appraisal method, but does not generate an effect size. A meta-analysis, or systematic review with meta-analysis, combines and analyzes results from studies to generate a new statistic: the effect size. | ||
Study Findings That Help Answer the EBP Question | ||
Skip to the Appraisal of Systematic Review (With or Without a Meta-Analysis) section |
Appraisal of QuaNtitative Research Studies | |||
Does the researcher identify what is known and not known about the problem and how the study will address any gaps in knowledge? |
|
| |
Was the purpose of the study clearly presented? |
|
| |
Was the literature review current (most sources within the past five years or a seminal study)? |
|
| |
Was sample size sufficient based on study design and rationale? |
|
| |
If there is a control group:
|
|
| N/A |
|
|
| N/A |
|
|
| N/A |
Are data collection methods described clearly? |
|
| |
Were the instruments reliable (Cronbach’s [alpha] > 0.70)? |
|
| N/A |
Was instrument validity discussed? |
|
| N/A |
If surveys or questionnaires were used, was the response |
|
| N/A |
Were the results presented clearly? |
|
| |
If tables were presented, was the narrative consistent with the table content? |
|
| N/A |
Were study limitations identified and addressed? |
|
| |
Were conclusions based on results? |
|
| |
Complete the Quality Rating for QuaNtitative Studies section |
Appraisal of Systematic Review (With or Without Meta-Analysis) | ||
Were the variables of interest clearly identified? |
|
|
Was the search comprehensive and reproducible?
|
|
|
|
|
|
|
|
|
Was there a flow diagram that included the number of studies eliminated at each level of review? |
|
|
Were details of included studies presented (design, sample, methods, results, outcomes, strengths, and limitations)? |
|
|
Were methods for appraising the strength of evidence (level and quality) described? |
|
|
Were conclusions based on results? |
|
|
|
|
|
|
|
|
Did the systematic review include a section addressing limitations and how they were addressed? |
|
|
Complete the Quality Rating for QuaNtitative Studies section (below) |
Quality Rating for QuaNtitative Studies |
Circle the appropriate quality rating below: A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate control; definitive conclusions; consistent recommendations based on comprehensive literature review that includes thorough reference to scientific evidence. B Good quality: Reasonably consistent results; sufficient sample size for the study design; some control, and fairly definitive conclusions; reasonably consistent recommendations based on fairly comprehensive literature review that includes some reference to scientific evidence. C Low quality or major flaws: Little evidence with inconsistent results; insufficient sample size for the study design; conclusions cannot be drawn. |
Section II: QuaLitative | ||
Level of Evidence (Study Design) | ||
A Is this a report of a single research study? |
|
|
Study Findings That Help Answer the EBP Question
| ||
Complete the Appraisal of Single QuaLitative Research Study section (below) |
Appraisal of a Single QuaLitative Research Study | ||
Was there a clearly identifiable and articulated:
|
|
|
|
|
|
|
|
|
|
|
|
Were study sample participants representative? |
|
|
Did they have knowledge of or experience with the research area? |
|
|
Were participant characteristics described? |
|
|
Was sampling adequate, as evidenced by achieving saturation of data? |
|
|
Data analysis:
|
|
|
|
|
|
Do findings support the narrative data (quotes)? |
|
|
Do findings flow from research question to data collected to analysis undertaken? |
|
|
Are conclusions clearly explained? |
|
|
Skip to the Quality Rating for QuaLitative Studies section |
For summaries of multiple quaLitative research studies (meta-synthesis), was a comprehensive search strategy and rigorous appraisal method used? B |
|
|
Study Findings That Help Answer the EBP Question | ||
Complete the Appraisal of Meta-Synthesis Studies section (below) |
Appraisal of Meta-Synthesis Studies | ||
Were the search strategy and criteria for selecting primary studies clearly defined? |
|
|
Were findings appropriate and convincing? |
|
|
Was a description of methods used to:
|
|
|
|
|
|
Did synthesis reflect: |
|
|
|
|
|
|
|
|
|
|
|
Was sufficient data presented to support the interpretations? |
|
|
Complete the Quality Rating for QuaLititative Studies section (below) |
Quality Rating for QuaLitative Studies |
Circle the appropriate quality rating below: No commonly agreed-on principles exist for judging the quality of quaLitative studies. It is a subjective process based on the extent to which study data contributes to synthesis and how much information is known about the researchers’ efforts to meet the appraisal criteria. For meta-synthesis, there is preliminary agreement that quality assessments should be made before synthesis to screen out poor-quality studies1. A/B High/Good quality is used for single studies and meta-syntheses2. The report discusses efforts to enhance or evaluate the quality of the data and the overall inquiry in sufficient detail; and it describes the specific techniques used to enhance the quality of the inquiry. Evidence of some or all of the following is found in the report:
C Lower-quality studies contribute little to the overall review of findings and have few, if any, of the features listed for High/Good quality. |
1 https://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/6_4_ASSESSMENT_OF_QUALITATIVE_RESEARCH.htm
2 Adapted from Polit & Beck (2017).
Section III: Mixed Methods | ||
Level of Evidence (Study Design) | ||
You will need to appraise both the quaNtitative and quaLitative parts of the study independently, before appraising the study in its entirety. | ||
| Level | Quality |
Insert here the level of evidence and overall quality for this part: | | |
| Level | Quality |
Insert here the level of evidence and overall quality for this part: | | |
| ||
| ||
Study Findings That Help Answer the EBP Question | ||
Complete the Appraisal of Mixed Methods Studies section (below) |
Appraisal of Mixed Methods Studies3 | |||
Was the mixed-methods research design relevant to address the quaNtitative and quaLitative research questions (or objectives)? |
|
|
|
Was the research design relevant to address the quaNtitative and quaLitative aspects of the mixed-methods question (or objective)? |
|
|
|
For convergent parallel designs, was the integration of quaNtitative and quaLitative data (or results) relevant to address the research question or objective? |
|
|
|
For convergent parallel designs, were the limitations associated with the integration (for example, the divergence of quaLitative and quaNtitative data or results) sufficiently addressed? |
|
|
|
Complete the Quality Rating for Mixed-Method Studies section (below) |
3 National Collaborating Centre for Methods and Tools. (2015). Appraising Qualitative, Quantitative, and Mixed Methods Studies included in Mixed Studies Reviews: The MMAT. Hamilton, ON: McMaster University. (Updated 20 July, 2015) Retrieved from http://www.nccmt.ca/ resources/search/232
Quality Rating for Mixed-Methods Studies |
Circle the appropriate quality rating below A High quality: Contains high-quality quaNtitative and quaLitative study components; highly relevant study design; relevant integration of data or results; and careful consideration of the limitations of the chosen approach. B Good quality: Contains good-quality quaNtitative and quaLitative study components; relevant study design; moderately relevant integration of data or results; and some discussion of limitations of integration. C Low quality or major flaws: Contains low quality quaNtitative and quaLitative study components; study design not relevant to research questions or objectives; poorly integrated data or results; and no consideration of limits of integration. |
4