Nursing - Evidence-Based Practice: Qualitative and Quantitative research report.

6 IASSIST Quarterly Fall 2009 by Kristi Thompson 1 Torturing Nurses With Data: Building a Successful Quantitative Research Module Introduction/Abstract This paper discusses two iterations of an effort to create a quantitative research module for a Master's in Nursing research methods course at the University of Windsor. The first version involved a single three-hour class incorporating both lecture and hands-on practice, followed by an assignment to independently locate and analyze a dataset. The second version was both more extensive and more structured, with a three-hour lecture, an assigned reading, a three-hour practice session, and an analysis assignment using a pre- selected data set. This paper compares feedback from the two iterations of the module and explains what worked and what did not work.

Background In-depth support for data began at the University of Windsor in 2006 with the opening of the Academic Data Centre. The Data Centre was conceived of as a service operating within the library that would offer multifaceted support for quantitative research. It includes my position of data librarian as well as that of a data centre manager who runs a drop-in consulting service. As this was a new service without an established customer base we were given an open mandate to promote ourselves, find customers and serve them in any way that seemed appropriate. One of the main goals of this new service was to advance data and quantitative literacy on campus, but exactly how we were to accomplish this goal was unclear.

Getting a toehold in the classroom – any classroom – seemed like a good way to start, so we combed through the course catalog looking for classes that seemed like they might incorporate use of data, and then sent out custom emails to selected professors offering our assistance. Somewhat to our surprise and relief, several of them took us up on our offer. And so we found ourselves doing various things – one professor needed help putting a teaching dataset together, others asked us to do presentations on available sources. One asked us to explain to her class the difference between qualitative and quantitative data, another wanted us to talk about survey construction. And one particularly adventurous professor of nursing invited us to design a unit to incorporate quantitative research into her research methods course. The Challenge The data centre manager and I spent some time working with the professor to determine what our goals were to be for this unit. The students taking the Masters of Nursing program at Windsor are mostly practicing nurses of varying ages who want to upgrade their credentials, and the majority of them do not intend to go on to further research. The program has a required statistics course which is separate from the methods course that is fairly math-oriented. Many of the students had not yet taken the statistics course, and for these this unit was to be their first exposure to data analysis.

There were about 15 students in the class.

Many students have difficulty grasping or retaining the details of statistical theory, but still need to grapple with the research results produced by the application of that theory. We determined that what we wanted to do for these students was to get them to think about quantitative research from a practical, real-world perspective. We thought their main need was to be able to understand and evaluate the quantitative research that they would come across in practice. "Not only does practical knowledge about survey methods and secondary analysis teach students how research is actually conducted, it informs critical assessment of arguments based on the interpretation of survey data." 2 We also hoped to influence those who were going on to further research to consider more quantitative projects. First Version We were given a single 3-hour block of class time to work with. This limitation forced us to carefully consider exactly what elements a quantitative unit needed to include. We ended up designing a program with three components; a lecture, a lab and an assignment. The focus of both the lecture and the lab was to prepare the students for the assignment. This assignment was in outline very straightforward: we told the nursing students to each come up with a research question that they could investigate by doing some sort of data analysis. They had to obtain the data, analyse it, and then write a research paper, complete with literature search and discussion. Given the limited preparation the students would have, we set aside large blocks of time for walk-in help in the Academic Data IASSIST Quarterly Fall 2009 7 Centre when both the centre director and I would be available to answer questions.

The Lecture Our class period started with a lecture where I talked about some basic concepts of data and quantitative research – how data is collected, what types of data are collected, differences between population data, survey data and experimental data, and some of the types of research questions each type of data can be used to answer. One lesson I had already learned when given limited time to talk to a class of students with no real data background is to not spend the time talking about the interfaces to different archives. I’ve moved away from mentioning the details of specific sources as much as possible – I’ve found it usually isn’t retained, and whatever is retained, they could get as well from a class handout or web page.

Another of my main goals for a data information literacy session did not crystallize until after that first unit we did with the nurses; to cut down on the number of students who come to me hoping to find the Impossible Dataset.

Several of the students came in looking for datasets that I knew without needing to conduct a search were highly unlikely to exist: datasets that violated confidentiality rules, surveys that could be used to compare small geographic areas, surveys of very specific populations. For example, a number were hoping to find surveys of nurses in particular cities or working in particular specializations, while another was looking for information that could probably only be obtained by linking individuals and their hospitalization records.

To cut down on these types of questions, and on the number of disappointed students who need to be told repeatedly to find another topic, I’ve found that the approach that works best is to spend some time discussing the realities of data collection, what sort of people and groups collect it, and why. The idea is to give researchers a conceptual framework for thinking about data sources – what is collected, what is released. Researchers who have some understanding of the data collection and dissemination process have more realistic expectations. However, as I had not yet fully developed this line of thinking before that first unit, I instead found myself making these explanations to many of the students during the walk-in help sessions.

The Lab The lab component was handled by the data centre manager, Dan Edelstein, with me on hand to provide extra help to individuals who needed it. Dan demonstrated how to do a basic set of analyses using SPSS, answering simple questions using a test dataset, and then walked them through doing the same analyses themselves on a different dataset. He stuck with showing them how to do descriptive statistics and frequencies, cross-tabulations, t-tests, ANOVAs and linear regression. The students who’d already taken the stats course actually didn’t have much of an advantage here, as they seemed to have learned more about how to calculate statistics than how to read them.

When I first presented this paper at IASSIST 2008, some of the session attendees questioned the idea that we could teach a group of students who lacked previous experience to “use SPSS” in the space of a single lab session. This is a valid question; SPSS is a package that can appear almost bewilderingly complex, especially to someone who is without much statistical background and is therefore unfamiliar with many of the terms used in analysis. The answer is quite simple: we did not teach them to use SPSS.

We showed them how to do a very limited set of things using SPSS, and gave them a handout that laid out the steps for each procedure so they could be followed almost mechanically. Our focus was on having them produce output which could then be interpreted.

An equally cogent question would be how we expected to teach them enough statistics to interpret that output in the course of that same lab session. Here the answer is a little more complex. What we did not do is teach them any statistical theory, nor did we really expect them to understand any. So given that we did not teach them either SPSS or statistics, what exactly were we teaching them, and what was its value?

Understanding how statistics are computed is not the same as being able to use statistical methods for research, and it was the application of statistics to research that we wanted to focus on. "The goal of quantitative research is to answer research questions or test hypotheses." 3 In this case that meant we told them to simply ignore most of the SPSS output and zero in on the numbers that would allow them to make a decision – to accept or reject a hypothesis. We wanted to abstract away from the details of the different statistical procedures and how they worked, and instead focus on what the procedures were doing, and what they all had in common: numbers that showed an effect and told them whether the effect was likely to be due to chance.

This approach is discussed in an earlier article I co-wrote with Dan Edelstein, where we said "our aim is not to teach statistics in itself, but to provide users with the practical knowledge needed to carry out their research... Teaching statistical theory is the professors’ role." 4 The Assignment The assignment was the test of our approach. The professor had merely stipulated that the students had to write a research paper on a topic relevant to nursing. Although elements of this program were based on previous work we had done with classes and individuals, Dan and I had never attempted anything quite as ambitious as a three- hour crash-course in quantitative analysis for students with little or no background. This left us uncertain as to exactly what level of work to expect on the assignment, so 8 IASSIST Quarterly Fall 2009 as a result we left it as general as possible. We hoped that telling the students to choose their own research question, leaving the choice of data and method of analysis open, and offering extensive one-on-one walk-in support would allow them to find and work at a level within their abilities. Our expectations were not particularly high – we just wanted them to write papers using some form of data analysis to coherently and appropriately support an argument, demonstrating that they had acquired some understanding of the role of statistical methodology in the research process.

The students took our injunction to find a research question that interested them and ran with it. Many of them drew on their experience and training to come up with projects that were personally or professionally relevant to them One community practitioner looked at characteristics of groups that don’t access preventative care, while another investigated links between local service provision and hospital readmission rates. A military nurse who expected to work on medical programs in an international setting looked at international data on health interventions and mortality. Many projects ended up being surprisingly sophisticated, and we were kept very busy during the month allotted to their assignment supporting our data neophytes as they used the simple analysis techniques they had been shown to conduct complex and interesting research. In short, our low expectations were not merely met or exceeded, but utterly blown out of the water. The professor had a similarly enthusiastic response, emailing us: "I just finished marking their papers for the quantitative assignment and they were superb. I have not had such an amazing outcome in papers such as these…" 5 The Second Version Having judged the first iteration of our quantitative module a success, Dr. Snowdon invited us to implement an expanded version to her class the following year. Instead of the single three-hour block of class time we’d been given before, she offered us up to three three-hour class sessions so that we would not be under such marked time constraints. This time, due to some quirk of scheduling, the class was more than twice as large as before, and even fewer of the students had taken statistics or used statistical software. We realized that the size of this class was going to be a real issue, as we’d had difficulty finding time to give the 15 students we’d had the previous time all the individual support they needed.

We knew that our first version had worked well, but we were not exactly sure why it had been as successful as it had been. We had not thought to include a formal feedback mechanism, but we had spoken with many students, and the main request we received in this anecdotal feedback was for more class time on SPSS, as many of them found the analysis the most stressful part. As we had more time IASSIST Quarterly Fall 2009 9 to work with, we decided to cover much the same material we had previously, but more slowly and with more time for questions and practice. So we split the lecture and covered the introductory and data finding material in one class, and devoted the second to practice with statistics and SPSS.

The third class we reserved for questions and one-on-one help.

The largest change we made was to the assignment. We thought that the free-form assignment we had used the previous time, allowing the students to "find and work at a level within their abilities," had given us an idea of what level of work to expect. So given the larger class size, we decided to this time give the students a selection of prepared data sets and accompanying research questions to choose from. Independently locating data was made an optional element of another assignment later in the term.

We also decided that this time we would have the class fill out feedback forms so we could better judge what did and did not work.

The numeric feedback we received on these forms was mixed. The students mostly reported that they found the content relevant and useful, but were divided on whether the unit would encourage or discourage them from conducting further quantitative research.

Many of them did feel that the unit had a positive effect on their understanding of quantitative research articles.

However, Dr Snowdon this time did not rave about the quality of the assignments, and the students working on the assignments seemed less enthusiastic and engaged than the previous group had been. The majority rated the assignment "too difficult", even though we had designed it to be at a lower level of complexity than the work that many of the students had voluntarily taken on the previous term.

The comments on the feedback forms helped clarify these results, and helped us to discern that the changes we had made to the program had had some unanticipated effects.

Particularly telling were comments like these: 6 • “(The quantitative unit) was statistics, not quantitative research” • “It was more like a stats class without having the theory” • “It seemed like a STATS project instead of a research project” We also got a number of requests, both in the formal evaluation and informally, for even more time spent on the SPSS and statistics components. In other words, spending three hours instead of one on the SPSS and statistics material seemed to paradoxically increase the students' anxiety even more, causing them to ask us to spend still more time teaching it. Discussion The two versions of this unit that we carried out formed an unintended, informal experiment on the use of a free-choice assignment in teaching students to conduct quantitative research. And the results of this experiment indicate that changing the assignment was a mistake. The comment “it seemed like a STATS project instead of a research project” sums up what went wrong quite well. Our original goal had been to get these students to think about quantitative research from a practical, real world perspective – to focus on how statistical findings are used to answer a question rather than on the details of how they are computed.

But giving the students pre-selected research questions and prepared datasets left them with little to think about except the details of how the statistics were computed.

Not allowing them to choose their own research question squelched the interest and enthusiasm that had made the first iteration of the program exciting for both them and us; instead of being curious and excited about the results they were finding, they obsessed over doing the procedures correctly. In short, the unit turned into an exercise in torturing nurses with data, which is probably not the best way to encourage students to go on to further quantitative research!

Notes 1 Contact: Kristi Thompson, Data Librarian, Leddy Library, University of Windsor, Windsor, Ontario, N9B 3P4. Phone:1 519 253 3000 x3858 Email: kathomps@ uwindsor.ca This paper was presented at the IASSIST 2008 conference at Stanford in the session "Numeracy, Quantitative Reasoning and Teaching About Data".

2. Corti, Louise (2004). Survey Data in Teaching Project (SDiT): Enhancing Critical Thinking and Data Literacy.

IASSIST Quarterly Vol. 28 No. 2-3 p 39-54.

3 Vogelsang, Joan (1999). Quantitative Research Versus Quality Assurance, Quality Improvement, Total Quality Management,, and Continuous Quality Improvement.

Journal of Perianesthesia Nursing Vol.14 iss.2 p 78-81.

4. Edelstein, Daniel M. and Kristi Thompson (2004). A Reference Model for Providing Statistical Consulting Services in an Academic Library Setting. IASSIST Quarterly Vol. 28 No. 2-3 p 35-38.

5. Anne Snowdon, e-mail message to the author, December 13, 2006 6. Comments taken from feedback forms filled out by students in Nursing 63-583, Research Methods in Nursing, December 2007. Copyright of IASSIST Quarterly is the property of IASSIST Quarterly and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.

However, users may print, download, or email articles for individual use.