EDUC 872 Book Review Assignment Instructions For this assignment, you will write a 5-page book review in current APA format that focuses on the required course textbook, Make It Stick: The Science of

Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

MAKE IT STICK Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

make it stick The Science of Successful Learning Peter C. Brown Henry L. Roediger III Mark A. McDaniel THE BELKNAP PRESS of HARVARD UNIVERSITY PRESS Cambridge, Massachusetts London, En gland 2014 Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Copyright © 2014 by Peter C. Brown, Henry L. Roediger III, Mark A. McDaniel All righ\bs reserve\f Printed in the United States of America Library of Congress Cataloging- in- Publication Data Brown, Peter C.

Make it stick : the science of successful learning / Peter C. Brown, Henry L. Roediger, Mark A. McDaniel. pages cm Includes bibliographical references and index. ISBN 978- 0- 674- 72901- 8 1. Learning— Research. 2. Cognition— Research. 3. Study skills.

I. Title.

LB1060.B768 2014 370.15'23—dc23 2013038420 Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Memory is the mother of all wisdom.Aeschylus Prometheus Bound Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Contents Preface ix 1 Learning Is Misunderstood 1 2 To Learn, Retrieve 23 3 Mix Up Your Practice 46 4 Embrace Diffi culties 67 5 Avoid Illusions of Knowing 102 6 Get Beyond Learning Styles 131 7 Increase Your Abilities 162 8 Make It Stick 200 Notes 257 Suggested Reading 285 Ac know ledg ments 289 Index 295 Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

ix People generally are going about learn- ing in the wrong ways. Empirical research into how we learn and remember shows that much of what we take for gospel about how to learn turns out to be largely wasted effort. Even college and medical students— whose main job is learning— rely on study techniques that are far from optimal. At the same time, this fi eld of research, which goes back 125 years but has been particularly fruitful in recent years, has yielded a body of insights that constitute a growing science of learning: highly effective, evidence- based strategies to replace less effective but widely accepted practices that are rooted in theory, lore, and intuition. But there’s a catch: the most effective learning strate- gies are not intuitive. Two of us, Henry Roediger and Mark McDaniel, are cog- nitive scientists who have dedicated our careers to the study of learning and memory. Peter Brown is a storyteller. We have Preface Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Preface ê x teamed up to explain how learning and memory work, and we do this less by reciting the research than by telling stories of people who have found their way to mastery of complex knowledge and skills. Through these examples we illuminate the principles of learning that the research shows are highly effective. This book arose in part from a collaboration among eleven cognitive psychologists. In 2002, the James S.

McDonnell Foundation of St. Louis, Missouri, in an effort to better bridge the gap between basic knowledge on learn- ing in cognitive psychology and its application in education, awarded a research grant “Applying Cognitive Psychology to Enhance Educational Practice” to Roediger and McDaniel and nine others, with Roediger as the principal investigator.

The team collaborated for ten years on research to translate cognitive science into educational science, and in many re- spects this book is a direct result of that work. The research- ers and many of their studies are cited in the book, the notes, and our acknowledgments. Roediger’s and McDaniel’s work is also supported by several other funders, and McDaniel is the co-director of Washington University’s Center for Inte- grative Research in Learning and Memory.Most books deal with topics serially— they cover one topic, move on to the next, and so on. We follow this strategy in the sense that each chapter addresses new topics, but we also ap- ply two of the primary learning principles in the book: spaced repetition of key ideas, and the interleaving of different but related topics. If learners spread out their study of a topic, returning to it periodically over time, they remember it better.

Similarly, if they interleave the study of different topics, they learn each better than if they had studied them one at a time in sequence. Thus we unabashedly cover key ideas more than once, repeating principles in different contexts across the book.

Created from liberty on 2021-11-21 07:19:56.

Preface ê xi The reader will remember them better and use them more ef- fectively as a result.This is a book about what people can do for themselves right now in order to learn better and remember longer. The responsibility for learning rests with every individual. Teach- ers and coaches, too, can be more effective right now by help- ing students understand these principles and by designing them into the learning experience. This is not a book about how education policy or the school system ought to be reformed.

Clearly, though, there are policy implications. For example, college professors at the forefront of applying these strategies in the classroom have experimented with their potential for narrowing the achievement gap in the sciences, and the results of those studies are eye opening. We write for students and teachers, of course, and for all readers for whom effective learning is a high priority: for train- ers in business, industry, and the military; for leaders of profes- sional associations offering in- service training to their mem- bers; and for coaches. We also write for lifelong learners nearing middle age or older who want to hone their skills so as to stay in the game. While much remains to be known about learning and its neural underpinnings, a large body of research has yielded principles and practical strategies that can be put to work im- mediately, at no cost, and to great effect.

Created from liberty on 2021-11-21 07:19:56.

1 Early in his career as a pi lot, Matt Brown was fl ying a twin- engine Cessna northeast out of Har- lingen, Texas, when he noticed a drop in oil pressure in his right engine. He was alone, fl ying through the night at eleven thousand feet, making a hotshot freight run to a plant in Ken- tucky that had shut down its manufacturing line awaiting product parts for assembly. He reduced altitude and kept an eye on the oil gauge, hop- ing to fl y as far as a planned fuel stop in Louisiana, where he could ser vice the plane, but the pressure kept falling. Matt has been messing around with piston engines since he was old enough to hold a wrench, and he knew he had a problem.

He ran a mental checklist, fi guring his options. If he let the oil pressure get too low he risked the engine’s seizing up. How much further could he fl y before shutting it down? What would happen when he did? He’d lose lift on the right side, 1 Learning Is Misunderstood Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 2 but could he stay aloft? He reviewed the tolerances he’d memorized for the Cessna 401. Loaded, the best you could do on one engine was slow your descent. But he had a light load, and he’d burned through most of his fuel. So he shut down the ailing right engine, feathered the prop to reduce drag, in- creased power on the left, fl ew with opposite rudder, and limped another ten miles toward his intended stop. There, he made his approach in a wide left- hand turn, for the simple but critical reason that without power on his right side it was only from a left- hand turn that he still had the lift needed to level out for a touchdown.

While we don’t need to understand each of the actions Matt took, he certainly needed to, and his ability to work himself out of a jam illustrates what we mean in this book when we talk about learning: we mean acquiring knowledge and skills and having them readily available from memory so you can make sense of future problems and opportunities. There are some immutable aspects of learning that we can probably all agree on: First, to be useful, learning requires memory, so what we’ve learned is still there later when we need it. Second, we need to keep learning and remembering all our lives. We can’t advance through middle school without some mastery of language arts, math, science, and social studies.

Getting ahead at work takes mastery of job skills and diffi cult colleagues. In retirement, we pick up new interests. In our dotage, we move into simpler housing while we’re still able to adapt. If you’re good at learning, you have an advantage in life. Third, learning is an acquired skill, and the most effective strategies are often counterintuitive.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 3 Claims We Make in This Book You may not agree with the last point, but we hope to per- suade you of it. Here, more or less unadorned in list form, are some of the principal claims we make in support of our argu- ment. We set them forth more fully in the chapters that follow.Learning is deeper and more durable when it’s effortful.

Learning that’s easy is like writing in sand, here today and gone tomorrow. We are poor judges of when we are learning well and when we’re not. When the going is harder and slower and it doesn’t feel productive, we are drawn to strategies that feel more fruitful, unaware that the gains from these strategies are often temporary. Rereading text and massed practice of a skill or new knowl- edge are by far the preferred study strategies of learners of all stripes, but they’re also among the least productive. By massed practice we mean the single- minded, rapid- fi re repetition of something you’re trying to burn into memory, the “practice- practice- practice” of conventional wisdom. Cramming for ex- ams is an example. Rereading and massed practice give rise to feelings of fl uency that are taken to be signs of mastery, but for true mastery or durability these strategies are largely a waste of time. Retrieval practice— recalling facts or concepts or events from memory— is a more effective learning strategy than re- view by rereading. Flashcards are a simple example. Retrieval strengthens the memory and interrupts forgetting. A single, simple quiz after reading a text or hearing a lecture produces better learning and remembering than rereading the text or reviewing lecture notes. While the brain is not a muscle that gets stronger with exercise, the neural pathways that make up a body of learning do get stronger, when the memory is Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 4 retrieved and the learning is practiced. Periodic practice ar- rests forgetting, strengthens retrieval routes, and is essential for hanging onto the knowledge you want to gain.When you space out practice at a task and get a little rusty between sessions, or you interleave the practice of two or more subjects, retrieval is harder and feels less productive, but the effort produces longer lasting learning and enables more versatile application of it in later settings. Trying to solve a problem before being taught the solution leads to better learning, even when errors are made in the attempt. The pop u lar notion that you learn better when you receive instruction in a form consistent with your preferred learning style, for example as an auditory or visual learner, is not sup- ported by the empirical research. People do have multiple forms of intelligence to bring to bear on learning, and you learn better when you “go wide,” drawing on all of your apti- tudes and resourcefulness, than when you limit instruction or experience to the style you fi nd most amenable. When you’re adept at extracting the underlying principles or “rules” that differentiate types of problems, you’re more successful at picking the right solutions in unfamiliar situa tions.

This skill is better acquired through interleaved and varied practice than massed practice. For instance, interleaving prac- tice at computing the volumes of different kinds of geometric solids makes you more skilled at picking the right solution when a later test presents a random solid. Interleaving the identifi cation of bird types or the works of oil paint ers im- proves your ability both to learn the unifying attributes within a type and to differentiate between types, improving your skill at categorizing new specimens you encounter later. We’re all susceptible to illusions that can hijack our judg- ment of what we know and can do. Testing helps calibrate Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 5 our judgments of what we’ve learned. A pi lot who is respond- ing to a failure of hydraulic systems in a fl ight simulator dis- covers quickly whether he’s on top of the corrective proce- dures or not. In virtually all areas of learning, you build better mastery when you use testing as a tool to identify and bring up your areas of weakness.All new learning requires a foundation of prior knowledge.

You need to know how to land a twin engine plane on two engines before you can learn to land it on one. To learn trigo- nometry, you need to remember your algebra and geometry. To learn cabinetmaking, you need to have mastered the proper- ties of wood and composite materials, how to join boards, cut rabbets, rout edges, and miter corners. In a cartoon by the Far Side cartoonist Gary Larson, a bug- eyed school kid asks his teacher, “Mr. Osborne, can I be ex- cused? My brain is full!” If you’re just engaging in mechanical repetition, it’s true, you quickly hit the limit of what you can keep in mind. However, if you practice elaboration, there’s no known limit to how much you can learn. Elaboration is the pro cess of giving new material meaning by expressing it in your own words and connecting it with what you already know. The more you can explain about the way your new learning relates to your prior knowledge, the stronger your grasp of the new learning will be, and the more connections you create that will help you remember it later. Warm air can hold more moisture than cold air; to know that this is true in your own experience, you can think of the drip of water from the back of an air conditioner or the way a stifl ing summer day turns cooler out the back side of a sudden thunderstorm.

Evaporation has a cooling effect: you know this because a humid day at your uncle’s in Atlanta feels hotter than a dry one at your cousin’s in Phoenix, where your sweat disap- pears even before your skin feels damp. When you study the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 6 principles of heat transfer, you understand conduction from warming your hands around a hot cup of cocoa; radiation from the way the sun pools in the den on a wintry day; con- vection from the life- saving blast of A/C as your uncle squires you slowly through his favorite back alley haunts of Atlanta. Putting new knowledge into a larger context helps learn- ing. For example, the more of the unfolding story of history you know, the more of it you can learn. And the more ways you give that story meaning, say by connecting it to your un- derstanding of human ambition and the untidiness of fate, the better the story stays with you. Likewise, if you’re trying to learn an abstraction, like the principle of angular momentum, it’s easier when you ground it in something concrete that you already know, like the way a fi gure skater’s rotation speeds up as she draws her arms to her chest. People who learn to extract the key ideas from new mate- rial and or ga nize them into a mental model and connect that model to prior knowledge show an advantage in learning com- plex mastery. A mental model is a mental repre sen ta tion of some external reality. 1 Think of a baseball batter waiting for a pitch. He has less than an instant to decipher whether it’s a curveball, a changeup, or something else. How does he do it?

There are a few subtle signals that help: the way the pitcher winds up, the way he throws, the spin of the ball’s seams. A great batter winnows out all the extraneous perceptual dis- tractions, seeing only these variations in pitches, and through practice he forms distinct mental models based on a different set of cues for each kind of pitch. He connects these models to what he knows about batting stance, strike zone, and swing- ing so as to stay on top of the ball. These he connects to men- tal models of player positions: if he’s got guys on fi rst and second, maybe he’ll sacrifi ce to move the runners ahead. If he’s got men on fi rst and third and there is one out, he’s got to Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 7 keep from hitting into a double play while still hitting to score the runner. His mental models of player positions connect to his models of the opposition (are they playing deep or shal- low?) and to the signals fl ying around from the dugout to the base coaches to him. In a great at- bat, all these pieces come together seamlessly: the batter connects with the ball and drives it through a hole in the outfi eld, buying the time to get on fi rst and advance his men. Because he has culled out all but the most important elements for identifying and responding to each kind of pitch, constructed mental models out of that learning, and connected those models to his mastery of the other essential elements of this complex game, an expert player has a better chance of scoring runs than a less experienced one who cannot make sense of the vast and changeable infor- mation he faces every time he steps up to the plate. Many people believe that their intellectual ability is hard- wired from birth, and that failure to meet a learning challenge is an indictment of their native ability. But every time you learn something new, you change the brain— the residue of your experiences is stored. It’s true that we start life with the gift of our genes, but it’s also true that we become capable through the learning and development of mental models that enable us to reason, solve, and create. In other words, the elements that shape your intellectual abilities lie to a surprising extent within your own control. Understanding that this is so en- ables you to see failure as a badge of effort and a source of useful information— the need to dig deeper or to try a differ- ent strategy. The need to understand that when learning is hard, you’re doing important work. To understand that striv- ing and setbacks, as in any action video game or new BMX bike stunt, are essential if you are to surpass your current level of per for mance toward true expertise. Making mistakes and correcting them builds the bridges to advanced learning.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 8 Empirical Evidence versus Theory, Lore, and Intuition Much of how we structure training and schooling is based on learning theories that have been handed down to us, and these are shaped by our own sense of what works, a sensibil- ity drawn from our personal experiences as teachers, coaches, students, and mere humans at large on the earth. How we teach and study is largely a mix of theory, lore, and intuition.

But over the last forty years and more, cognitive psychologists have been working to build a body of evidence to clarify what works and to discover the strategies that get results.

Cognitive psychology is the basic science of understanding how the mind works, conducting empirical research into how people perceive, remember, and think. Many others have their hands in the puzzle of learning as well. Developmental and educational psychologists are concerned with theories of human development and how they can be used to shape the tools of education— such as testing regimes, instructional or- ganizers (for example topic outlines and schematic illustra- tions), and resources for special groups like those in remedial and gifted education. Neuroscientists, using new imaging tech- niques and other tools, are advancing our understanding of brain mechanisms that underlie learning, but we’re still a very long way from knowing what neuroscience will tell us about how to improve education. How is one to know whose advice to take on how best to go about learning? It’s wise to be skeptical. Advice is easy to fi nd, only a few mouse- clicks away. Yet not all advice is grounded in research— far from it. Nor does all that passes as research meet the stan- dards of science, such as having appropriate control condi- tions to assure that the results of an investigation are objective Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 9 and generalizable. The best empirical studies are experimental in nature: the researcher develops a hypothesis and then tests it through a set of experiments that must meet rigorous crite- ria for design and objectivity. In the chapters that follow, we have distilled the fi ndings of a large body of such studies that have stood up under review by the scientifi c community be- fore being published in professional journals. We are collabo- rators in some of these studies, but not the lion’s share. Where we’re offering theory rather than scientifi cally validated re- sults, we say so. To make our points we use, in addition to tested science, anecdotes from people like Matt Brown whose work requires mastery of complex knowledge and skills, sto- ries that illustrate the underlying principles of how we learn and remember. Discussion of the research studies themselves is kept to a minimum, but you will fi nd many of them cited in the notes at the end of the book if you care to dig further. People Misunderstand Learning It turns out that much of what we’ve been doing as teachers and students isn’t serving us well, but some comparatively simple changes could make a big difference. People commonly believe that if you expose yourself to something enough times— say, a textbook passage or a set of terms from an eighth grade biology class— you can burn it into memory. Not so. Many teachers believe that if they can make learning easier and faster, the learning will be better. Much research turns this belief on its head: when learning is harder, it’s stronger and lasts longer.

It’s widely believed by teachers, trainers, and coaches that the most effective way to master a new skill is to give it dogged, single- minded focus, practicing over and over until you’ve got it down. Our faith in this runs deep, because most of us see fast gains during the learning phase of massed practice. What’s Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 10 apparent from the research is that gains achieved during massed practice are transitory and melt away quickly.The fi nding that rereading textbooks is often labor in vain ought to send a chill up the spines of educators and learners, because it’s the number one study strategy of most people— including more than 80 percent of college students in some surveys—and is central in what we tell ourselves to do during the hours we dedicate to learning. Rereading has three strikes against it. It is time consuming. It doesn’t result in durable memory. And it often involves a kind of unwitting self- deception, as growing familiarity with the text comes to feel like mastery of the content. The hours immersed in rereading can seem like due diligence, but the amount of study time is no mea sure of mastery. 2 You needn’t look far to fi nd training systems that lean heavily on the conviction that mere exposure leads to learn- ing. Consider Matt Brown, the pi lot. When Matt was ready to advance from piston planes, he had a whole new body of knowledge to master in order to get certifi ed for the business jet he was hired to pi lot. We asked him to describe this pro- cess. His employer sent him to eigh teen days of training, ten hours a day, in what Matt called the “fi re hose” method of instruction. The fi rst seven days straight were spent in the classroom being instructed in all the plane’s systems: electri- cal, fuel, pneumatics, and so on, how these systems operated and interacted, and all their fail- safe tolerances like pressures, weights, temperatures, and speeds. Matt is required to have at his immediate command about eighty different “memory ac- tion items”— actions to take without hesitation or thought in order to stabilize the plane the moment any one of a dozen or so unexpected events occur. It might be a sudden decompres- sion, a thrust reverser coming unlocked in fl ight, an engine failure, an electrical fi re.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 11 Matt and his fellow pi lots gazed for hours at mind- numbing PowerPoint illustrations of their airplane’s principal systems. Then something interesting happened. “About the middle of day fi ve,” Matt said, “they fl ash a schematic of the fuel system on the screen, with its pressure sensors, shutoff valves, ejector pumps, bypass lines, and on and on, and you’re struggling to stay focused. Then this one instructor asks us, ‘Has anybody here had the fuel fi lter by- pass light go on in fl ight?’ This pi lot across the room raises his hand. So the instructor says, ‘Tell us what happened,’ and sud- denly you’re thinking, Whoa, what if that was me? “So, this guy was at 33,000 feet or something and he’s about to lose both engines because he got fuel without anti- freeze in it and his fi lters are clogging with ice. You hear that story and, believe me, that schematic comes to life and sticks with you. Jet fuel can commonly have a little water in it, and when it gets cold at high altitude, the water will condense out, and it can freeze and block the line. So whenever you refuel, you make good and sure to look for a sign on the fuel truck saying the fuel has Prist in it, which is an antifreeze. And if you ever see that light go on in fl ight, you’re going to get yourself down to some warmer air in a hurry.” 3 Learning is stronger when it matters, when the abstract is made concrete and personal. Then the nature of Matt’s instruction shifted. The next eleven days were spent in a mix of classroom and fl ight simu- lator training. Here, Matt described the kind of active en- gagement that leads to durable learning, as the pi lots had to grapple with their aircraft to demonstrate mastery of stan- dard operating procedures, respond to unexpected situations, and drill on the rhythm and physical memory of the move- ments that are required in the cockpit for dealing with them.

A fl ight simulator provides retrieval practice, and the practice Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 12 is spaced, interleaved, and varied and involves as far as pos- sible the same mental pro cesses Matt will invoke when he’s at altitude. In a simulator, the abstract is made concrete and personal. A simulator is also a series of tests, in that it helps Matt and his instructors calibrate their judgment of where he needs to focus to bring up his mastery.

In some places, like Matt Brown’s fl ight simulator, teachers and trainers have found their way to highly effective learning techniques, yet in virtually any fi eld, these techniques tend to be the exception, and “fi re hose” lectures (or their equivalent) are too often the norm.In fact, what students are advised to do is often plain wrong.

For instance, study tips published on a website at George Mason University include this advice: “The key to learning something well is repetition; the more times you go over the material the better chance you have of storing it permanently.” 4 Another, from a Dartmouth College website, suggests: “If you intend to remember something, you probably will.” 5 A pub- lic ser vice piece that runs occasionally in the St. Louis Post- Dispatch offering study advice shows a kid with his nose buried in a book. “Concentrate,” the caption reads. “Focus on one thing and one thing only. Repeat, repeat, repeat! Repeat- ing what you have to remember can help burn it into your memory.” 6 Belief in the power of rereading, intentionality, and repetition is pervasive, but the truth is you usually can’t em- bed something in memory simply by repeating it over and over. This tactic might work when looking up a phone num- ber and holding it in your mind while punching it into your phone, but it doesn’t work for durable learning. A simple example, reproduced on the Internet (search “penny memory test”), presents a dozen different images of a Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 13 common penny, only one of which is correct. As many times as you’ve seen a penny, you’re hard pressed to say with confi - dence which one it is. Similarly, a recent study asked faculty and students who worked in the Psychology Building at UCLA to identify the fi re extinguisher closest to their offi ce. Most failed the test. One professor, who had been at UCLA for twenty- fi ve years, left his safety class and decided to look for the fi re extinguisher closest to his offi ce. He discovered that it was actually right next to his offi ce door, just inches from the doorknob he turned every time he went into his offi ce. Thus, in this case, even years of repetitive exposure did not result in his learning where to grab the closest extinguisher if his waste- basket caught fi re. 7 Early Evidence The fallacy in thinking that repetitive exposure builds mem- ory has been well established through a series of investiga- tions going back to the mid- 1960s, when the psychologist Endel Tulving at the University of Toronto began testing people on their ability to remember lists of common En glish nouns. In a fi rst phase of the experiment, the participants simply read a list of paired items six times (for example, a pair on the list might be “chair— 9”); they did not expect a memory test. The fi rst item in each pair was always a noun. After reading the listed pairs six times, participants were then told that they would be getting a list of nouns that they would be asked to remember. For one group of people, the nouns were the same ones they had just read six times in the prior reading phase; for another group, the nouns to be learned were different from those they had previously read. Remarkably, Tulving found that the two groups’ learning of the nouns did not differ— the learning curves were statistically indistinguishable. Intuition Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 14 would suggest otherwise, but prior exposure did not aid later recall. Mere repetition did not enhance learning. Subsequent studies by many researchers have pressed further into ques- tions of whether repeated exposure or longer periods of hold- ing an idea in mind contribute to later recall, and these studies have confi rmed and elaborated on the fi ndings that repetition by itself does not lead to good long- term memory. 8 These results led researchers to investigate the benefi ts of rereading texts. In a 2008 article in Contemporary Educa- tional Psychology, Washington University scientists reported on a series of studies they conducted at their own school and at the University of New Mexico to shed light on rereading as a strategy to improve understanding and memory of prose.

Like most research, these studies stood on the shoulders of earlier work by others; some showed that when the same text is read multiple times the same inferences are made and the same connections between topics are formed, and others sug- gested modest benefi ts from rereading. These benefi ts had been found in two different situations. In the fi rst, some students read and immediately reread study material, whereas other students read the material only once. Both groups took an im- mediate test after reading, and the group who had read twice performed a bit better than the group who had read once.

However, on a delayed test the benefi t of immediate rereading had worn off, and the rereaders performed at the same level as the one- time readers. In the other situation, students read the material the fi rst time and then waited some days before they reread it. This group, having done spaced readings of the text, performed better on the test than the group who did not re- read the material. 9 Subsequent experiments at Washington University, aimed at teasing apart some of the questions the earlier studies had raised, assessed the benefi ts of rereading among students of Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 15 differing abilities, in a learning situation paralleling that faced by students in classes. A total of 148 students read fi ve differ- ent passages taken from textbooks and Scientifi c American.

The students were at two different universities; some were high- ability readers, and others were low- ability; some stu- dents read the material only once, and others read it twice in succession. Then all of them responded to questions to dem- onstrate what they had learned and remembered. In these experiments, multiple readings in close succession did not prove to be a potent study method for either group, at either school, in any of the conditions tested. In fact, the researchers found no rereading benefi t at all under these conditions. What’s the conclusion? It makes sense to reread a text once if there’s been a meaningful lapse of time since the fi rst read- ing, but doing multiple readings in close succession is a time- consuming study strategy that yields negligible benefi ts at the expense of much more effective strategies that take less time.

Yet surveys of college students confi rm what professors have long known: highlighting, underlining, and sustained poring over notes and texts are the most- used study strategies, by far. 10 Illusions of Knowing If rereading is largely in effec tive, why do students favor it?

One reason may be that they’re getting bad study advice. But there’s another, subtler way they’re pushed toward this method of review, the phenomenon mentioned earlier: rising familiar- ity with a text and fl uency in reading it can create an illusion of mastery. As any professor will attest, students work hard to capture the precise wording of phrases they hear in class lec- tures, laboring under the misapprehension that the essence of the subject lies in the syntax in which it’s described. Mastering Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 16 the lecture or the text is not the same as mastering the ideas behind them. However, repeated reading provides the illu- sion of mastery of the underlying ideas. Don’t let yourself be fooled. The fact that you can repeat the phrases in a text or your lecture notes is no indication that you understand the signifi cance of the precepts they describe, their application, or how they relate to what you already know about the subject. Too common is the experience of a college professor an- swering a knock on her offi ce door only to fi nd a fi rst- year student in distress, asking to discuss his low grade on the fi rst test in introductory psychology. How is it possible? He at- tended all the lectures and took diligent notes on them. He read the text and highlighted the critical passages. How did he study for the test? she asks.

Well, he’d gone back and highlighted his notes, and then reviewed the highlighted notes and his highlighted text mate- rial several times until he felt he was thoroughly familiar with all of it. How could it be that he had pulled a D on the exam? Had he used the set of key concepts in the back of each chapter to test himself? Could he look at a concept like “con- ditioned stimulus,” defi ne it, and use it in a paragraph? While he was reading, had he thought of converting the main points of the text into a series of questions and then later tried to answer them while he was studying? Had he at least re- phrased the main ideas in his own words as he read? Had he tried to relate them to what he already knew? Had he looked for examples outside the text? The answer was no in every case. He sees himself as the model student, diligent to a fault, but the truth is he doesn’t know how to study effectively. The illusion of mastery is an example of poor metacogni- tion: what we know about what we know. Being accurate in your judgment of what you know and don’t know is critical Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 17 for decision making. The problem was famously (and pro- phetically) summed up by Secretary of State Donald Rums- feld in a 2002 press briefi ng about US intelligence on Iraq’s possible possession of weapons of mass destruction: “There are known knowns; there are things we know that we know.

There are known unknowns; that is to say, there are things that we now know we don’t know. But there are also un- known unknowns—there are things we do not know we don’t know.”The emphasis here is ours. We make it to drive home the point that students who don’t quiz themselves (and most do not) tend to overestimate how well they have mastered class material. Why? When they hear a lecture or read a text that is a paragon of clarity, the ease with which they follow the argu- ment gives them the feeling that they already know it and don’t need to study it. In other words, they tend not to know what they don’t know; when put to the test, they fi nd they cannot recall the critical ideas or apply them in a new context.

Likewise, when they’ve reread their lecture notes and texts to the point of fl uency, their fl uency gives them the false sense that they’re in possession of the underlying content, princi- ples, and implications that constitute real learning, confi dent that they can recall them at a moment’s notice. The upshot is that even the most diligent students are often hobbled by two liabilities: a failure to know the areas where their learning is weak— that is, where they need to do more work to bring up their knowledge— and a preference for study methods that create a false sense of mastery. 11 Knowledge: Not Sufﬁ cient, but Necessary Albert Einstein declared “creativity is more important than knowledge,” and the sentiment appears to be widely shared by Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 18 college students, if their choice in t-shirt proclamations is any indication. And why wouldn’t they seize on the sentiment? It embodies an obvious and profound truth, for without cre- ativity where would our scientifi c, social, or economic break- throughs come from? Besides which, accumulating knowledge can feel like a grind, while creativity sounds like a lot more fun.

But of course the dichotomy is false. You wouldn’t want to see that t-shirt on your neurosurgeon or on the captain who’s fl y- ing your plane across the Pacifi c. But the sentiment has gained some currency as a reaction to standardized testing, fearing that this kind of testing leads to an emphasis on memorization at the expense of high- level skills. Notwithstanding the pitfalls of standardized testing, what we really ought to ask is how to do better at building knowledge and creativity, for without knowledge you don’t have the foundation for the higher- level skills of analysis, synthesis, and creative problem solving. As the psychologist Robert Sternberg and two colleagues put it, “one cannot apply what one knows in a practical manner if one does not know anything to apply.” 12 Mastery in any fi eld, from cooking to chess to brain sur- gery, is a gradual accretion of knowledge, conceptual under- standing, judgment, and skill. These are the fruits of variety in the practice of new skills, and of striving, refl ection, and men- tal rehearsal. Memorizing facts is like stocking a construction site with the supplies to put up a house. Building the house requires not only knowledge of countless different fi ttings and materials but conceptual understanding, too, of aspects like the load- bearing properties of a header or roof truss system, or the principles of energy transfer and conservation that will keep the house warm but the roof deck cold so the own er doesn’t call six months later with ice dam problems. Mastery requires both the possession of ready knowledge and the con- ceptual understanding of how to use it.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 19 When Matt Brown had to decide whether or not to kill his right engine he was problem solving, and he needed to know from memory the procedures for fl ying with a dead engine and the tolerances of his plane in order to predict whether he would fall out of the air or be unable to straighten up for landing. The would- be neurosurgeon in her fi rst year of med school has to memorize the whole ner vous system, the whole skeletal system, the whole muscular system, the humeral sys- tem. If she can’t, she’s not going to be a neurosurgeon. Her success will depend on diligence, of course, but also on fi nding study strategies that will enable her to learn the sheer volume of material required in the limited hours available. Testing: Dipstick versus Learning Tool There are few surer ways to raise the hackles of many stu- dents and educators than talking about testing. The growing focus over recent years on standardized assessment, in par- tic u lar, has turned testing into a lightning rod for frustration over how to achieve the country’s education goals. Online forums and news articles are besieged by readers who charge that emphasis on testing favors memorization at the expense of a larger grasp of context or creative ability; that testing cre- ates extra stress for students and gives a false mea sure of abil- ity; and so on. But if we stop thinking of testing as a dipstick to mea sure learning— if we think of it as practicing retrieval of learning from memory rather than “testing,” we open our- selves to another possibility: the use of testing as a tool for learning. One of the most striking research fi ndings is the power of active retrieval— testing—to strengthen memory, and that the more effortful the retrieval, the stronger the benefi t. Think fl ight simulator versus PowerPoint lecture. Think quiz versus Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 20 rereading. The act of retrieving learning from memory has two profound benefi ts. One, it tells you what you know and don’t know, and therefore where to focus further study to improve the areas where you’re weak. Two, recalling what you have learned causes your brain to reconsolidate the mem- ory, which strengthens its connections to what you already know and makes it easier for you to recall in the future. In effect, retrieval— testing—interrupts forgetting. Consider an eighth grade science class. For the class in question, at a mid- dle school in Columbia, Illinois, researchers arranged for part of the material covered during the course to be the subject of low- stakes quizzing (with feedback) at three points in the se- mester. Another part of the material was never quizzed but was studied three times in review. In a test a month later, which material was better recalled? The students averaged A- on the material that was quizzed and C+ on the material that was not quizzed but reviewed. 13 In Matt Brown’s case, even after ten years pi loting the same business jet, his employer reinforces his mastery every six months in a battery of tests and fl ight simulations that re- quire him to retrieve the information and maneuvers that are essential to stay in control of his plane. As Matt points out, you hardly ever have an emergency, so if you don’t practice what to do, there’s no way to keep it fresh. Both of these cases— the research in the classroom and the experience of Matt Brown in updating his knowledge— point to the critical role of retrieval practice in keeping our knowl- edge accessible to us when we need it. The power of active retrieval is the topic of Chapter 2. 14 Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Learning Is Misunderstood ê 21 The Takeaway For the most part, we are going about learning in the wrong ways, and we are giving poor advice to those who are coming up behind us. A great deal of what we think we know about how to learn is taken on faith and based on intuition but does not hold up under empirical research. Per sis tent illusions of knowing lead us to labor at unproductive strategies; as recounted in Chapter 3, this is true even of people who have participated in empirical studies and seen the evidence for themselves, fi rsthand. Illusions are potent persuaders. One of the best habits a learner can instill in herself is regular self- quizzing to recalibrate her understanding of what she does and does not know. Second Lieutenant Kiley Hunkler, a 2013 graduate of West Point and winner of a Rhodes Scholarship, whom we write about in Chapter 8, uses the phrase “shooting an azimuth” to describe how she takes practice tests to help refocus her studying. In overland navigation, shooting an azi- muth means climbing to a height, sighting an object on the horizon in the direction you’re traveling, and adjusting your compass heading to make sure you’re still gaining on your objective as you beat through the forest below. The good news is that we now know of simple and practical strategies that anybody can use, at any point in life, to learn better and remember longer: various forms of retrieval prac- tice, such as low- stakes quizzing and self- testing, spacing out practice, interleaving the practice of different but related top- ics or skills, trying to solve a problem before being taught the solution, distilling the underlying principles or rules that dif- ferentiate types of problems, and so on. In the chapters that follow we describe these in depth. And because learning is an iterative pro cess that requires that you revisit what you have Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 22 learned earlier and continually update it and connect it with new knowledge, we circle through these topics several times along the way. At the end, in Chapter 8, we pull it all to- gether with specifi c tips and examples for putting these tools to work.

Created from liberty on 2021-11-21 07:19:56.

23 Mike Ebersol\f go\b calle\f into a hospi- tal emergency room one afternoon late in 2011 to examine a Wisconsin deer hunter who’d been found lying unconscious in a cornfi eld. The man had blood at the back of his head, and the men who’d found and brought him in supposed he’d maybe stumbled and cracked his skull on something. Ebersold is a neurosurgeon. The injury had brain protrud- ing, and he recognized it as a gunshot wound. The hunter re- gained consciousness in the ER, but when asked how he’d hurt himself, he had no idea. Recounting the incident later, Ebersold said, “Somebody from some distance away must have fi red what appeared to be a 12- gauge shotgun, which arced over God only knows what distance, hit this guy in the back of his head, fractured his skull, and lodged into the brain about an inch. It must have been pretty much spent, or it would have gone deeper.” 1 2 To Learn, Retrieve Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 24 Ebersold is tall, slender, and counts among his forebears the Dakota chiefs named Wapasha and the French fur traders named Rocque who populated this part of the Mississippi River Valley where the Mayo brothers would later found their famous clinic. Ebersold’s formal training included four years of college, four years of medical school, and seven years of neurosurgery training— building a foundation of knowl- edge and skills that has been broadened and deepened through continuing medical education classes, consultations with his colleagues, and his practice at the Mayo Clinic and elsewhere.

He carries himself with a midwestern modesty that belies a career that counts a long list of high- profi le patients who have sought out his ser vices. When President Ronald Reagan needed treatment for injuries after a fall from his horse, Ebersold par- ticipated in the surgery and postsurgical care. When Sheikh Zayed bin Sultan Al Nahyan, president of the United Arab Emirates, needed delicate spinal repair, he and what seemed like half the nation’s ministry and security forces settled in Rochester while Mike Ebersold made the repair and oversaw Zayed’s recovery. Following a long career at Mayo, Mike had returned to help out at the clinic in Wisconsin, feeling indebted to it for his early medical training. The hunter whose bad luck put him in the way of an errant 12- gauge slug was luckier than he likely knows that Mike was on the job that day. The bullet had entered an area of the skull beneath which there is a large venous sinus, a soft- tissue channel that drains the brain cavity. As he examined the hunter, Ebersold knew from experience that when he opened up the wound, there was a high probability he would fi nd this vein was torn. As he described it, You say to yourself, “This patient is going to need surgery.

There’s brain coming out of the wound. We have to clean this Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 25 up and repair this as best we can, but in so doing we may get into this big vein and that could be very, very serious.” So you go through the checklist. You say, “I might need a blood trans- fusion for this patient,” so you set up some blood. You review the steps, A, B, C, and D. You set up the operating room, tell- ing them ahead of time what you might be encountering. All of this is sort of protocol, pretty much like a cop getting ready to pull over a car, you know what the book says, you’ve gone through all these steps.Then you get to the operating room, and now you’re still in this mode where you have time to think through it. You say, “Gee, I don’t want to just go and pull that bullet out if there might be major bleeding. What I’ll try to do is I’ll work around the edges and get things freed up so I’m ready for what could go wrong, and then I’ll pull it out.” It turned out that the bullet and bone were lodged in the vein, serving as plugs, another lucky turn for the hunter. If the wound hadn’t corked itself in the fi eld, he would not have lived for more than two or three minutes. When Ebersold re- moved the bullet, the fractured bone chips fell away, and the vein let loose in a torrent. “Within fi ve minutes, you’ve lost two or so units of blood and now you sort of transfer out of the mode where you’re thinking through this, going through the options. Now it becomes refl ex, mechanical. You know it’s going to bleed very, very much, so you have a very short time.

You’re just thinking, ‘I have to get a suture around this struc- ture, and I know from previous experience I have to do it in this par tic u lar way.’ ” The vein in question, which is about the size of an adult’s small fi nger, was torn in several places over a distance of about an inch and a half. It needed to be tied off above and below the rupture, but it’s a fl at structure that he knows well: you Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 26 can’t just put a stitch around it, because when you tighten it, the tissue tears, and the ligature leaks. Working urgently and mechanically, he fell back on a technique he’d developed out of necessity in past surgeries involving this vein. He cut two little pieces of muscle, from where the patient’s skin had been opened up in surgery, and imported them to the site and stitched the ends of the torn vein to them. These plugs of muscle served to close the vein without defl ecting its natural shape or tearing its tissue. It’s a solution Mike has taught himself— one he says you won’t fi nd written anywhere, but handy in the moment, to say the least. In the sixty or so sec- onds it took to do, the patient lost another two hundred cubic centimeters of blood, but once the plugs were in place, the bleeding stopped. “Some people can’t tolerate this sinus vein being closed off. They get increased brain pressure because the blood doesn’t drain properly. But this patient was one of the fortunate who can.” The hunter left the hospital a week later. He was minus some peripheral vision but otherwise re- markably unscathed from a very close brush with mortality. Reﬂ ection Is a Form of Practice What inferences can we draw from this story about how we learn and remember? In neurosurgery (and, arguably, in all aspects of life from the moment you leave the womb), there’s an essential kind of learning that comes from refl ection on personal experience. Ebersold described it this way: A lot of times something would come up in surgery that I had diffi culty with, and then I’d go home that night thinking about what happened and what could I do, for example, to improve the way a suturing went. How can I take a bigger bite with my needle, or a smaller bite, or should the stitches be closer to- gether? What if I modifi ed it this way or that way? Then the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 27 next day back, I’d try that and see if it worked better. Or even if it wasn’t the next day, at least I’ve thought through this, and in so doing I’ve not only revisited things that I learned from lectures or from watching others performing surgery but also I’ve complemented that by adding something of my own to it that I missed during the teaching pro cess.

Refl ection can involve several cognitive activities that lead to stronger learning: retrieving knowledge and earlier training from memory, connecting these to new experiences, and visu- alizing and mentally rehearsing what you might do differently next time. It was this kind of refl ection that originally had led Eber- sold to try a new technique for repairing the sinus vein at the back of the head, a technique he practiced in his mind and in the operating room until it became the kind of refl exive maneu- ver you can depend on when your patient is spouting blood at two hundred cubic centimeters a minute. To make sure the new learning is available when it’ s needed, Ebersold points out, “you memorize the list of things that you need to worry about in a given situation: steps A, B, C, and D,” and you drill on them. Then there comes a time when you get into a tight situation and it’s no longer a matter of thinking through the steps, it’s a matter of refl exively taking the cor- rect action. “Unless you keep recalling this maneuver, it will not become a refl ex. Like a race car driver in a tight situation or a quarterback dodging a tackle, you’ve got to act out of re- fl ex before you’ve even had time to think. Recalling it over and over, practicing it over and over. That’s just so important.” Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 28 The Testing Effect A child stringing cranberries on a thread goes to hang them on the tree, only to fi nd they’ve slipped off the other end. With- out the knot, there’s no making a string. Without the knot there’s no necklace, there’s no beaded purse, no magnifi cent tapestry.

Retrieval ties the knot for memory. Repeated retrieval snugs it up and adds a loop to make it fast. Since as far back as 1885, psychologists have been plotting “forgetting curves” that illustrate just how fast our cranberries slip off the string. In very short order we lose something like 70 percent of what we’ve just heard or read. After that, forget- ting begins to slow, and the last 30 percent or so falls away more slowly, but the lesson is clear: a central challenge to im- proving the way we learn is fi nding a way to interrupt the pro- cess of forgetting. 2 The power of retrieval as a learning tool is known among psychologists as the testing effect. In its most common form, testing is used to mea sure learning and assign grades in school, but we’ve long known that the act of retrieving knowledge from memory has the effect of making that knowledge easier to call up again in the future. In his essay on memory, Aristotle wrote: “exercise in repeatedly recalling a thing strengthens the memory.” Francis Bacon wrote about this phenomenon, as did the psychologist William James. Today, we know from empiri- cal research that practicing retrieval makes learning stick far better than reexposure to the original material does. This is the testing effect, also known as the retrieval- practice effect. 3 To be most effective, retrieval must be repeated again and again, in spaced out sessions so that the recall, rather than becoming a mindless recitation, requires some cognitive ef- fort. Repeated recall appears to help memory consolidate into a cohesive repre sen ta tion in the brain and to strengthen and Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 29 multiply the neural routes by which the knowledge can later be retrieved. In recent de cades, studies have confi rmed what Mike Ebersold and every seasoned quarterback, jet pi lot, and teenaged texter knows from experience— that repeated re- trieval can so embed knowledge and skills that they become refl exive: the brain acts before the mind has time to think. Yet despite what research and personal experience tell us about the power of testing as a learning tool, teachers and stu- dents in traditional educational settings rarely use it as such, and the technique remains little understood or utilized by teach- ers or students as a learning tool in traditional educational settings. Far from it. In 2010 the New York Times reported on a scientifi c study that showed that students who read a passage of text and then took a test asking them to recall what they had read retained an astonishing 50 percent more of the information a week later than students who had not been tested. This would seem like good news, but here’s how it was greeted in many online comments: “Once again, another author confuses learning with recalling information.” “I personally would like to avoid as many tests as possible, especially with my grade on the line. Trying to learn in a stress- ful environment is no way to help retain information.” “Nobody should care whether memorization is enhanced by practice testing or not. Our children cannot do much of any- thing anymore.” 4 Forget memorization, many commenters argued; education should be about high- order skills. Hmmm. If memorization is irrelevant to complex problem solving, don’t tell your Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 30 neurosurgeon. The frustration many people feel toward stan- dardized, “dipstick” tests given for the sole purpose of mea- sur ing learning is understandable, but it steers us away from appreciating one of the most potent learning tools available to us. Pitting the learning of basic knowledge against the de- velopment of creative thinking is a false choice. Both need to be cultivated. The stronger one’s knowledge about the subject at hand, the more nuanced one’s creativity can be in address- ing a new problem. Just as knowledge amounts to little with- out the exercise of ingenuity and imagination, creativity ab- sent a sturdy foundation of knowledge builds a shaky house. Studying the Testing Effect in the Lab The testing effect has a solid pedigree in empirical research.

The fi rst large- scale investigation was published in 1917.

Children in grades 3, 5, 6, and 8 studied brief biographies from Who’s Who in America. Some of them were directed to spend varying lengths of the study time looking up from the material and silently reciting to themselves what it contained.

Those who did not do so simply continued to reread the ma- terial. At the end of the period, all the children were asked to write down what they could remember. The recall test was repeated three to four hours later. All the groups who had engaged in the recitation showed better retention than those who had not done so but had merely continued to review the material. The best results were from those spending about 60 percent of the study time in recitation. A second landmark study, published in 1939, tested over three thousand sixth graders across Iowa. The kids studied six- hundred- word articles and then took tests at various times before a fi nal test two months later. The experiment showed a couple of interesting results: the longer the fi rst test was de- layed, the greater the forgetting, and second, once a student Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 31 had taken a test, the forgetting nearly stopped, and the stu- dent’s score on subsequent tests dropped very little. 5 Around 1940, interest turned to the study of forgetting, and investigating the potential of testing as a form of retrieval practice and as a learning tool fell out of favor. So did the use of testing as a research tool: since testing interrupts forgetting, you can’t use it to mea sure forgetting because that “contami- nates” the subject. Interest in the testing effect resurfaced in 1967 with the publication of a study showing that research subjects who were presented with lists of thirty- six words learned as much from repeated testing after initial exposure to the words as they did from repeated studying. These results— that testing led to as much learning as studying did— challenged the re- ceived wisdom, turned researchers’ attention back to the po- tential of testing as a learning tool, and stimulated a boomlet in testing research. In 1978, researchers found that massed studying (cram- ming) leads to higher scores on an immediate test but results in faster forgetting compared to practicing retrieval. In a sec- ond test two days after an initial test, the crammers had for- gotten 50 percent of what they had been able to recall on the initial test, while those who had spent the same period prac- ticing retrieval instead of studying had forgotten only 13 per- cent of the information recalled initially. A subsequent study was aimed at understanding what ef- fect taking multiple tests would have on subjects’ long- term retention. Students heard a story that named sixty concrete objects. Those students who were tested immediately after exposure recalled 53 percent of the objects on this initial test but only 39 percent a week later. On the other hand, a group of students who learned the same material but were not tested at all until a week later recalled 28 percent. Thus, taking a single test boosted per for mance by 11 percent after a week.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 32 But what effect would three immediate tests have relative to one? Another group of students were tested three times after initial exposure and a week later they were able to recall 53 percent of the objects— the same as on the initial test for the group receiving one test. In effect, the group that received three tests had been “immunized” against forgetting, com- pared to the one- test group, and the one- test group remem- bered more than those who had received no test immediately following exposure. Thus, and in agreement with later research, multiple sessions of retrieval practice are generally better than one, especially if the test sessions are spaced out. 6 In another study, researchers showed that simply asking a subject to fi ll in a word’s missing letters resulted in better memory of the word. Consider a list of word pairs. For a pair like foot- shoe, those who studied the pair intact had lower sub- sequent recall than those who studied the pair from a clue as obvious as foot- s_ _e. This experiment was a demonstration of what researchers call the “generation effect.” The modest effort required to generate the cued answer while studying the pairs strengthened memory of the target word tested later (shoe).

Interestingly, this study found that the ability to recall the word pair on later tests was greater if the practice retrieval was de- layed by twenty intervening word pairs than when it came im- mediately after fi rst studying the pair. 7 Why would that be?

One argument suggested that the greater effort required by the delayed recall solidifi ed the memory better. Researchers began to ask whether the schedule of testing mattered. The answer is yes. When retrieval practice is spaced, allow- ing some forgetting to occur between tests, it leads to stronger long- term retention than when it is massed. Researchers began looking for opportunities to take their inquiries out of the lab and into the classroom, using the kinds of materials students are required to learn in school.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 33 Studying the Testing Effect “In the Wild” In 2005, we and our colleagues approached Roger Cham- berlain, the principal of a middle school in nearby Columbia, Illinois, with a proposition. The positive effects of retrieval practice had been demonstrated many times in controlled lab- oratory settings but rarely in a regular classroom setting.

Would the principal, teachers, kids, and parents of Colum- bia Middle School be willing subjects in a study to see how the testing effect would work “in the wild”?Chamberlain had concerns. If this was just about memori- zation, he wasn’t especially interested. His aim is to raise the school’s students to higher forms of learning— analysis, synthe- sis, and application, as he put it. And he was concerned about his teachers, an energetic faculty with curricula and varied instructional methods he was loath to disrupt. On the other hand, the study’s results could be instructive, and participa- tion would bring enticements in the form of smart boards and “clickers”— automated response systems—for the classrooms of participating teachers. Money for new technology is fa- mously tight. A sixth grade social studies teacher, Patrice Bain, was eager to give it a try. For the researchers, a chance to work in the classroom was compelling, and the school’s terms were ac- cepted: the study would be minimally intrusive by fi tting within existing curricula, lesson plans, test formats, and teaching methods. The same textbooks would be used. The only differ- ence in the class would be the introduction of occasional short quizzes. The study would run for three semesters (a year and a half), through several chapters of the social studies textbook, covering topics such as ancient Egypt, Mesopotamia, India, and China. The project was launched in 2006. It would prove to be a good decision.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 34 For the six social studies classes a research assistant, Pooja Agarwal, designed a series of quizzes that would test students on roughly one- third of the material covered by the teacher.

These quizzes were for “no stakes,” meaning that scores were not counted toward a grade. The teacher excused herself from the classroom for each quiz so as to remain unaware of which material was being tested. One quiz was given at the start of class, on material from assigned reading that hadn’t yet been discussed. A second was given at the end of class after the teacher had covered the material for the day’s lesson. And a review quiz was given twenty- four hours before each unit exam. There was concern that if students tested better in the fi nal exam on material that had been quizzed than on material not quizzed, it could be argued that the simple act of reexposing them to the material in the quizzes was responsible for the superior learning, not the retrieval practice. To counter this possibility, some of the nonquizzed material was interspersed with the quiz material, provided as simple review statements, like “The Nile River has two major tributaries: the White Nile and the Blue Nile,” with no retrieval required. The facts were quizzed for some classes but just restudied for others. The quizzes took only a few minutes of classroom time.

After the teacher stepped out of the room, Agarwal projected a series of slides onto the board at the front of the room and read them to the students. Each slide presented either a mul- tiple choice question or a statement of fact. When the slide contained a question, students used clickers (handheld, cell- phone- like remotes) to indicate their answer choice: A, B, C, or D. When all had responded, the correct answer was revealed, so as to provide feedback and correct errors. (Although teachers were not present for these quizzes, under normal circumstances, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 35 with teachers administering quizzes, they would see immedi- ately how well students are tracking the study material and use the results to guide further discussion or study.)Unit exams were the normal pencil- and- paper tests given by the teacher. Exams were also given at the end of the se- mester and at the end of the year. Students had been exposed to all of the material tested in these exams through the teacher’s normal classroom lessons, homework, worksheets, and so on, but they had also been quizzed three times on one- third of the material, and they had seen another third presented for additional study three times. The balance of the material was neither quizzed nor additionally reviewed in class beyond the initial lesson and what ever reading a student may have done. The results were compelling: The kids scored a full grade level higher on the material that had been quizzed than on the material that had not been quizzed. Moreover, test results for the material that had been reviewed as statements of fact but not quizzed were no better than those for the nonreviewed material. Again, mere rereading does not much help. In 2007, the research was extended to eighth grade science classes, covering ge ne tics, evolution, and anatomy. The regi- men was the same, and the results equally impressive. At the end of three semesters, the eighth graders averaged 79 percent (C+ ) on the science material that had not been quizzed, com- pared to 92 percent (A− ) on the material that had been quizzed.

The testing effect persisted eight months later at the end- of- year exams, confi rming what many laboratory studies have shown about the long- term benefi ts of retrieval practice. The effect doubtless would have been greater if the retrieval prac- tice had continued and occurred once a month, say, in the in- tervening months. 8 Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 36 The lesson from these studies has been taken to heart by many of the teachers at Columbia Middle School. Long after concluding their participation in the research studies, Patrice Bain’s sixth grade social studies classes continue today to fol- low a schedule of quizzes before lessons, quizzes after lessons, and then a review quiz prior to the chapter test. Jon Wehren- berg, an eighth grade history teacher who was not part of the research, has knitted retrieval practice into his classroom in many different forms, including quizzing, and he provides ad- ditional online tools at his website, like fl ashcards and games.

After reading passages on the history of slavery, for example, his students are asked to write down ten facts about slavery they hadn’t known before reading the passages. You don’t need electronic gadgetry to practice retrieval. Seven sixth and seventh graders needing to improve their reading and comprehension skills sat in Michelle Spivey’s En- glish classroom one period recently with their reading books open to an amusing story. Each student was invited to read a paragraph aloud. Where a student stumbled, Miss Spivey had him try again. When he’d gotten it right, she probed the class to explain the meaning of the passage and what might have been going on in the characters’ minds. Retrieval and elabora- tion; again, no technology required. Quizzes at Columbia Middle School are not onerous events.

Following completion of the research studies, students’ views were surveyed on this question. Sixty- four percent said the quizzing reduced their anxiety over unit exams, and 89 percent felt it increased learning. The kids expressed disappointment on days when clickers were not used, because the activity broke up the teacher’s lecture and proved enjoyable. Principal Chamberlain, when asked what he thought the study results indicated, replied simply: “Retrieval practice has a signifi cant impact on kids’ learning. This is telling us that Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 37 it’s valuable, and that teachers are well advised to incorporate it into their instructional technique.” 9 Are similar effects found at a later age?Andrew Sobel teaches a class in international po liti cal eco- nomics at Washington University in St. Louis, a lecture course populated by 160– 170 students, mostly freshmen and sopho- mores. Over a period of several years he noticed a growing problem with attendance. On any given day by midsemester, 25– 35 percent of the class would be absent, compared to ear- lier in the semester when maybe 10 percent would be absent.

The problem wasn’t unique to his class, he says. A lot of pro- fessors give students their PowerPoint slides, so the students just stop coming to class. Sobel fought back by withholding his slides, but by the end of the semester, many students stopped showing up anyway. The class syllabus included two big tests, a midterm and a fi nal. Looking for some way to leverage attendance, Sobel replaced the big tests with nine pop quizzes. Because the quizzes would determine the course grade and would be unannounced, students would be well advised to show up for class. The results were distressing. Over the semester, a third or more of the students bailed out. “I really got hammered in the teaching reviews,” Sobel told us. “The kids hated it. If they didn’t do well on a quiz they dropped the course rather than get a bad grade in it. Of those who stayed, I got this bifurcation between those who actually showed up and did the work, and those who didn’t. I found myself handing out A-plusses, which I’d never given before, and more Cs than I’d ever given.” 10 With so much pushback, he had little choice but to drop the experiment and reinstate the old format, lectures with a mid- term and fi nal. A couple of years later, however, after hearing a Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 38 pre sen ta tion about the learning benefi ts of testing, he added a third major test during the semester to see what effect it might have on his students’ learning. They did better, but not by as much as he’d hoped, and the attendance problems persisted. He scratched his head and changed the syllabus once again.

This time he announced that there would be nine quizzes dur- ing the semester, and he was explicit about when they would be. No surprises, and no midterm or fi nal exams, because he didn’t want to give up that much of his lecture time. Despite fears that enrollments would plummet again, they actually increased by a handful. “Unlike the pop quizzes, which kids hate, these were all on the syllabus. If they missed one it was their own fault. It wasn’t because I surprised them or was being pernicious. They were comfortable with that.” Sobel took satisfaction in seeing attendance improve as well. “They would skip some classes on the days they didn’t have a quiz, particu- larly the spring semester, but they showed up for the quizzes.” Like the course, the quizzes were cumulative, and the ques- tions were similar to those on the exams he used to give, but the quality of the answers he was getting by midsemester was much better than he was accustomed to seeing on the mid- terms. Five years into this new format, he’s sold on it. “The quality of discussions in class has gone way up. I see that big a difference in their written work, just by going from three exams to nine quizzes.” By the end of the semester he has them writing paragraphs on the concepts covered in class, some- times a full- page essay, and the quality is comparable to what he’s seeing in his upper division classes. “Anybody can design this structure. But I also realize that, Oh, god, if I’d done this years ago I would have taught them that much more stuff. The interesting thing about adopting this strategy is I now recognize that as good a teacher as I Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 39 might think I am, my teaching is only a component of their learning, and how I structure it has a lot to do with it, maybe even more.” Meanwhile, the course enrollment has grown to 185 and counting. Exploring Nuances Andy Sobel’s example is anecdotal and likely refl ects a variety of benefi cial infl uences, not least being the cumulative learn- ing effects that accrue like compounded interest when course material is carried forward in a regime of quizzes across an entire semester. Nonetheless, his experience squares with em- pirical research designed to tease apart the effects and nu- ances of testing. For example, in one experiment college students studied prose passages on various scientifi c topics like those taught in college and then either took an immediate recall test after the initial exposure or restudied the material. After a delay of two days, the students who took the initial test recalled more of the material than those who simply restudied it (68 v. 54 per- cent), and this advantage was sustained a week later (56 v. 42 percent). Another experiment found that after one week a study- only group showed the most forgetting of what they ini- tially had been able to recall, forgetting 52 percent, compared to a repeated- testing group, who forgot only 10 percent. 11 How does giving feedback on wrong answers to test questions affect learning? Studies show that giving feedback strengthens retention more than testing alone does, and, interestingly, some evidence shows that delaying the feedback briefl y pro- duces better long- term learning than immediate feedback. This fi nding is counterintuitive but is consistent with researchers’ Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 40 discoveries about how we learn motor tasks, like making lay- ups or driving a golf ball toward a distant green. In motor learning, trial and error with delayed feedback is a more awk- ward but effective way of acquiring a skill than trial and cor- rection through immediate feedback; immediate feedback is like the training wheels on a bicycle: the learner quickly comes to depend on the continued presence of the correction.In the case of learning motor skills, one theory holds that when there’s immediate feedback it comes to be part of the task, so that later, in a real- world setting, its absence becomes a gap in the established pattern that disrupts per for mance.

Another idea holds that frequent interruptions for feedback make the learning sessions too variable, preventing establish- ment of a stabilized pattern of per for mance. 12 In the classroom, delayed feedback also yields better long- term learning than immediate feedback does. In the case of the students studying prose passages on science topics, some were shown the passage again even while they were asked to answer questions about it, in effect providing them with con- tinuous feedback during the test, analogous to an open- book exam. The other group took the test without the study mate- rial at hand and only afterward were given the passage and instructed to look over their responses. Of course, the open- book group performed best on the immediate test, but those who got corrective feedback after completing the test retained the learning better on a later test. Delayed feedback on writ- ten tests may help because it gives the student practice that’s spaced out in time; as discussed in the next chapter, spacing practice improves retention. 13 Are some kinds of retrieval practice more effective for long- term learning than others? Tests that require the learner to Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 41 supply the answer, like an essay or short- answer test, or sim- ply practice with fl ashcards, appear to be more effective than simple recognition tests like multiple choice or true/false tests.

However, even multiple choice tests like those used at Colum- bia Middle School can yield strong benefi ts. While any kind of retrieval practice generally benefi ts learning, the implication seems to be that where more cognitive effort is required for retrieval, greater retention results. Retrieval practice has been studied extensively in recent years, and an analysis of these studies shows that even a single test in a class can produce a large improvement in fi nal exam scores, and gains in learning continue to increase as the number of tests increases. 14 Whichever theories science eventually tells us are correct about how repeated retrieval strengthens memory, empirical research shows us that the testing effect is real— that the act of retrieving a memory changes the memory, making it easier to retrieve again later.

How widely is retrieval practice used as a study technique? In one survey, college students were largely unaware of its effec- tiveness. In another survey, only 11 percent of college students said they use this study strategy. Even when they did report testing themselves, they mostly said they did it to discover what they didn’t know, so they could study that material more.

That’s a perfectly valid use of testing, but few students realize that retrieval itself creates greater retention. 15 Is repeated testing simply a way to expedite rote learning? In fact, research indicates that testing, compared to rereading, can facilitate better transfer of knowledge to new contexts and problems, and that it improves one’s ability to retain and Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 42 retrieve material that is related but not tested. Further re- search is needed on this point, but it seems that retrieval prac- tice can make information more accessible when it is needed in various contexts.

Do students resist testing as a tool for learning? Students do generally dislike the idea of tests, and it’s not hard to see why, in par tic u lar in the case of high- stakes tests like midterms and fi nals, where the score comes with signifi cant consequences.

Yet in all studies of testing that reported students’ attitudes, the students who were tested frequently rated their classes more favorably at the end of the semester than those tested less frequently. Those who were frequently tested reached the end of the semester on top of the material and did not need to cram for exams.

How does taking a test affect subsequent studying? After a test, students spend more time restudying the material they missed, and they learn more from it than do their peers who restudy the material without having been tested. Students whose study strategies emphasize rereading but not self- testing show over- confi dence in their mastery. Students who have been quizzed have a double advantage over those who have not: a more accurate sense of what they know and don’t know, and the strengthening of learning that accrues from retrieval practice. 16 Are there any further, indirect benefi ts of regular, low- stakes classroom testing? Besides strengthening learning and reten- tion, a regime of this kind of testing improves student atten- dance. It increases studying before class (because students Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 43 know they’ll be quizzed), increases attentiveness during class if students are tested at the end of class, and enables students to better calibrate what they know and where they need to bone up. It’s an antidote to mistaking fl uency with the text, resulting from repeated readings, for mastery of the subject.

Frequent low- stakes testing helps dial down test anxiety among students by diversifying the consequences over a much larger sample: no single test is a make- or- break event. And this kind of testing enables instructors to identify gaps in stu- dents’ understanding and adapt their instruction to fi ll them.

These benefi ts of low- stakes testing accrue whether instruc- tion is delivered online or in the classroom. 17 The Takeaway Practice at retrieving new knowledge or skill from memory is a potent tool for learning and durable retention. This is true for anything the brain is asked to remember and call up again in the future— facts, complex concepts, problem- solving tech- niques, motor skills. Effortful retrieval makes for stronger learning and reten- tion. We’re easily seduced into believing that learning is better when it’s easier, but the research shows the opposite: when the mind has to work, learning sticks better. The greater the effort to retrieve learning, provided that you succeed, the more that learning is strengthened by retrieval. After an initial test, delaying subsequent retrieval practice is more potent for rein- forcing retention than immediate practice, because delayed retrieval requires more effort. Repeated retrieval not only makes memories more durable but produces knowledge that can be retrieved more readily, in more varied settings, and applied to a wider variety of problems.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 44 While cramming can produce better scores on an immedi- ate exam, the advantage quickly fades because there is much greater forgetting after rereading than after retrieval practice.

The benefi ts of retrieval practice are long- term. Simply including one test (retrieval practice) in a class yields a large improvement in fi nal exam scores, and gains continue to increase as the frequency of classroom testing increases. Testing doesn’t need to be initiated by the instructor. Stu- dents can practice retrieval anywhere; no quizzes in the class- room are necessary. Think fl ashcards— the way second grad- ers learn the multiplication tables can work just as well for learners at any age to quiz themselves on anatomy, mathemat- ics, or law. Self- testing may be unappealing because it takes more effort than rereading, but as noted already, the greater the effort at retrieval, the more will be retained. Students who take practice tests have a better gr asp of their progress than those who simply reread the material. Similarly, such testing enables an instructor to spot gaps and miscon- ceptions and adapt instruction to correct them. Giving students corrective feedback after tests keeps them from incorrectly retaining material they have misunderstood and produces better learning of the correct answers. Students in classes that incorporate low- stakes quizzing come to embrace the practice. Students who are tested fre- quently rate their classes more favorably.

What about Principal Roger Chamberlain’s initial concerns about practice quizzing at Columbia Middle School— that it might be nothing more than a glorifi ed path to rote learning? When we asked him this question after the study was com- pleted, he paused for a moment to gather his thoughts. “What Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

To Learn, Retrieve ê 45 I’ve really gained a comfort level with is this: for kids to be able to evaluate, synthesize, and apply a concept in different settings, they’re going to be much more effi cient at getting there when they have the base of knowledge and the reten- tion, so they’re not wasting time trying to go back and fi gure out what that word might mean or what that concept was about. It allows them to go to a higher level.” Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

46 I\b may no\b be in\bui\bive that retrieval practice is a more powerful learning strategy than repeated review and rereading, yet most of us take for granted the importance of testing in sports. It’s what we call “practice- practice- practice.” Well, here’s a study that may surprise you.

A group of eight- year- olds practiced tossing beanbags into buckets in gym class. Half of the kids tossed into a bucket three feet away. The other half mixed it up by tossing into buckets two feet and four feet away. After twelve weeks of this they were all tested on tossing into a three- foot bucket. The kids who did the best by far were those who’d practiced on two- and four- foot buckets but never on three- foot buckets. 1 Why is this? We will come back to the beanbags, but fi rst a little insight into a widely held myth about how we learn. 3 Mix Up Your Practice Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 47 The Myth of Massed Practice Most of us believe that learning is better when you go at something with single- minded purpose: the practice- practice- practice that’s supposed to burn a skill into memory. Faith in focused, repetitive practice of one thing at a time until we’ve got it nailed is pervasive among classroom teachers, athletes, corporate trainers, and students. Researchers call this kind of practice “massed,” and our faith rests in large part on the simple fact that when we do it, we can see it making a differ- ence. Nevertheless, despite what our eyes tell us, this faith is misplaced. If learning can be defi ned as picking up new knowledge or skills and being able to apply them later, then how quickly you pick something up is only part of the story. Is it still there when you need to use it out in the everyday world? While practicing is vital to learning and memory, studies have shown that practice is far more effective when it’s broken into sepa- rate periods of training that are spaced out. The rapid gains produced by massed practice are often evident, but the rapid forgetting that follows is not. Practice that’s spaced out, inter- leaved with other learning, and varied produces better mas- tery, longer retention, and more versatility. But these benefi ts come at a price: when practice is spaced, interleaved, and varied, it requires more effort. You feel the increased effort, but not the benefi ts the effort produces. Learning feels slower from this kind of practice, and you don’t get the rapid im- provements and affi rmations you’re accustomed to seeing from massed practice. Even in studies where the participants have shown superior results from spaced learning, they don’t perceive the improvement; they believe they learned better on the material where practice was massed.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 48 Almost everywhere you look, you fi nd examples of massed practice: summer language boot camps, colleges that offer con- centration in a single subject with the promise of fast learning, continuing education seminars for professionals where train- ing is condensed into a single weekend. Cramming for exams is a form of massed practice. It feels like a productive strategy, and it may get you through the next day’s midterm, but most of the material will be long forgotten by the time you sit down for the fi nal. Spacing out your practice feels less productive for the very reason that some forgetting has set in and you’ve got to work harder to recall the concepts. It doesn’t feel like you’re on top of it. What you don’t sense in the moment is that this added effort is making the learning stronger. 2 Spaced Practice The benefi ts of spacing out practice sessions are long estab- lished, but for a vivid example consider this study of thirty- eight surgical residents. They took a series of four short lessons in microsurgery: how to reattach tiny vessels. Each lesson included some instruction followed by some prac- tice. Half the docs completed all four lessons in a single day, which is the normal in- service schedule. The others completed the same four lessons but with a week’s interval between them. 3 In a test given a month after their last session, those whose lessons had been spaced a week apart outperformed their col- leagues in all areas— elapsed time to complete a surgery, num- ber of hand movements, and success at reattaching the sev- ered, pulsating aortas of live rats. The difference in per for mance between the two groups was impressive. The residents who had taken all four sessions in a single day not only scored lower on all mea sures, but 16 percent of them damaged the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 49 rats’ vessels beyond repair and were unable to complete their surgeries.Why is spaced practice more effective than massed practice?

It appears that embedding new learning in long- term memory requires a pro cess of consolidation, in which memory traces (the brain’s repre sen ta tions of the new learning) are strength- ened, given meaning, and connected to prior knowledge— a pro cess that unfolds over hours and may take several days.

Rapid- fi re practice leans on short- term memory. Durable learning, however, requires time for mental rehearsal and the other pro cesses of consolidation. Hence, spaced practice works better. The increased effort required to retrieve the learning after a little forgetting has the effect of retriggering consoli- dation, further strengthening memory. We explore some of the theories about this pro cess in the next chapter. Interleaved Practice Interleaving the practice of two or more subjects or skills is also a more potent alternative to massed practice, and here’s a quick example of that. Two groups of college students were taught how to fi nd the volumes of four obscure geometric solids (wedge, spheroid, spherical cone, and half cone). One group then worked a set of practice problems that were clus- tered by problem type (practice four problems for computing the volume of a wedge, then four problems for a spheroid, etc.). The other group worked the same practice problems, but the sequence was mixed (interleaved) rather than clustered by type of problem. Given what we’ve already presented, the results may not surprise you. During practice, the students who worked the problems in clusters (that is, massed) averaged 89 percent correct, compared to only 60 percent for those who worked the problems in a mixed sequence. But in the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 50 fi nal test a week later, the students who had practiced solving problems clustered by type averaged only 20 percent correct, while the students whose practice was interleaved averaged 63 percent. The mixing of problem types, which boosted fi nal test per for mance by a remarkable 215 percent, actually im- peded per for mance during initial learning. 4 Now, suppose you’re a trainer in a company trying to teach employees a complicated new pro cess that involves ten proce- dures. The typical way of doing this is to train up in proce- dure 1, repeating it many times until the trainees really seem to have it down cold. Then you go to procedure 2, you do many repetitions of 2, you get that down, and so on. That appears to produce fast learning. What would interleaved practice look like? You practice procedure 1 just a few times, then switch to procedure 4, then switch to 3, then to 7, and so on. (Chapter 8 tells how Farmers Insurance trains new agents in a spiraling series of exercises that cycle back to key skillsets in a seem- ingly random sequence that adds layers of context and mean- ing at each turn.) The learning from interleaved practice feels slower than learning from massed practice. Teachers and students sense the difference. They can see that their grasp of each element is coming more slowly, and the compensating long- term advan- tage is not apparent to them. As a result, interleaving is unpop- u lar and seldom used. Teachers dislike it because it feels slug- gish. Students fi nd it confusing: they’re just starting to get a handle on new material and don’t feel on top of it yet when they are forced to switch. But the research shows unequivo- cally that mastery and long- term retention are much better if you interleave practice than if you mass it.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 51 Varied Practice Okay, what about the beanbag study where the kids who did best had never practiced the three- foot toss that the other kids had only practiced?The beanbag study focused on mastery of motor skills, but much evidence has shown that the underlying principle ap- plies to cognitive learning as well. The basic idea is that varied practice— like tossing your beanbags into baskets at mixed distances— improves your ability to transfer learning from one situation and apply it successfully to another. You develop a broader understanding of the relationships between different conditions and the movements required to succeed in them; you discern context better and develop a more fl exible “move- ment vocabulary”— different movements for different situa- tions. Whether the scope of variable training (e.g., the two- and four- foot tosses) must encompass the par tic u lar task (the three- foot toss) is subject for further study. The evidence favoring variable training has been supported by recent neuroimaging studies that suggest that different kinds of practice engage different parts of the brain. The learning of motor skills from varied practice, which is more cognitively challenging than massed practice, appears to be consolidated in an area of the brain associated with the more diffi cult pro- cess of learning higher- order motor skills. The learning of mo- tor skills from massed practice, on the other hand, appears to be consolidated in a different area of the brain that is used for learning more cognitively simple and less challenging motor skills. The inference is that learning gained through the less challenging, massed form of practice is encoded in a simpler or comparatively impoverished repre sen ta tion than the learn- ing gained from the varied and more challenging practice Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 52 which demands more brain power and encodes the learning in a more fl exible repre sen ta tion that can be applied more broadly. 5 Among athletes, massed practice has long been the rule:

take your hook shot, knock the twenty- foot putt, work your backhand return, throw the pass while rolling out: again and again and again— to get it right and train your “muscle mem- ory.” Or so the notion holds. The benefi ts of variable training for motor learning have been gaining broader ac cep tance, albeit slowly. Consider the one- touch pass in hockey. That’s where you receive the puck and immediately pass it to a team- mate who’s moving down the ice, keeping the opposition off balance and unable to put pressure on the puck carrier. Jamie Kompon, when he was assistant coach of the Los Angeles Kings, was in the habit of running team practice on one- touch passes from the same position on the rink. Even if this move is interleaved with a sequence of other moves in practice, if you only do it at the same place on the rink or in the same sequence of moves, you are only, as it were, throwing your beanbags into the three- foot bucket. Kompon is onto the difference now and has changed up his drills. Since we talked, he’s gone over to the Chicago Blackhawks. We would have said “Keep an eye on those Blackhawks” here, but as we revise to go into produc- tion, Kompon and team have already won the Stanley Cup.

Perhaps no coincidence? The benefi ts of variable practice for cognitive as opposed to motor skills learning were shown in a recent experiment that adapted the beanbag test to verbal learning: in this case, the students solved anagrams– that is, they rearranged letters to form words (tmoce becomes comet). Some subjects practiced the same anagram over and over, whereas others practiced mul- tiple anagrams for the word. When they were all tested on the same anagram that the former group had practiced on, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 53 the latter group performed better on it! The same benefi ts will apply whether you are practicing to identify tree species, differentiate the principles of case law , or master a new com- puter program. 6 Developing Discrimination Skills Compared to massed practice, a signifi cant advantage of in- terleaving and variation is that they help us learn better how to assess context and discriminate between problems, selecting and applying the correct solution from a range of possibilities.

In math education, massing is embedded in the textbook: each chapter is dedicated to a par tic u lar kind of problem, which you study in class and then practice by working, say, twenty ex- amples for homework before you move on. The next chapter has a different type of problem, and you dive into the same kind of concentrated learning and practice of that solution.

On you march, chapter by chapter, through the semester. But then, on the fi nal exam, lo and behold, the problems are all mixed up: you’re staring at each one in turn, asking yourself Which algorithm do I use? Was it in chapter 5, 6, or 7? When you have learned under conditions of massed or blocked repe- tition, you have had no practice on that critical sorting pro cess.

But this is the way life usually unfolds: problems and oppor- tunities come at us unpredictably, out of sequence. For our learning to have practical value, we must be adept at discerning “What kind of problem is this?” so we can select and apply an appropriate solution. Several studies have demonstrated the improved powers of discrimination to be gained through interleaved and varied practice. One study involved learning to attribute paintings to the artists who created them, and another focused on learning to identify and classify birds.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 54 Researchers initially predicted that massed practice in identifying paint ers’ works (that is, studying many examples of one paint er’s works before moving on to study many ex- amples of another’s works) would best help students learn the defi ning characteristics of each artist’s style. Massed practice of each artist’s works, one artist at a time, would better enable students to match artworks to artists later, compared to inter- leaved exposure to the works of different artists. The idea was that interleaving would be too hard and confusing; students would never be able to sort out the relevant dimensions. The researchers were wrong. The commonalities among one paint- er’s works that the students learned through massed practice proved less useful than the differences between the works of multiple paint ers that the students learned through interleav- ing. Interleaving enabled better discrimination and produced better scores on a later test that required matching the works with their paint ers. The interleaving group was also better able to match paint ers’ names correctly to new examples of their work that the group had never viewed during the learn- ing phase. Despite these results, the students who participated in these experiments persisted in preferring massed practice, convinced that it served them better. Even after they took the test and could have realized from their own per for mance that interleaving was the better strategy for learning, they clung to their belief that the concentrated viewing of paintings by one artist was better. The myths of massed practice are hard to ex- orcise, even when you’re experiencing the evidence yourself. 7 The power of interleaving practice to improve discrim- inability has been reaffi rmed in studies of people learning bird classifi cation. The challenge here is more complex than it might seem. One study addressed twenty different bird fami- lies (thrashers, swallows, wrens, fi nches, and so on). Within each family, students were presented with a dozen species Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 55 (brown thrasher, curve- billed thrasher, Bendire’s thrasher, etc.). To identify a bird’s family, you consider a wide range of traits like size, plumage, behavior, location, beak shape, iris color, and so on. A problem in bird identifi cation is that mem- bers of a family share many traits in common but not all. For instance, many but not all thrashers have a long, slightly hooked beak. There are traits that are typical of a family but none that occur in all members of that family and can serve as unique identifi ers. Because rules for classifi cation can only rely on these characteristic traits rather than on defi ning traits (ones that hold for every member), bird classifi cation is a mat- ter of learning concepts and making judgments, not simply memorizing features. Interleaved and variable practice proved more helpful than massed practice for learning the underlying concepts that unite and differentiate the species and families. To paraphrase a conclusion from one of these studies, re- call and recognition require “factual knowledge,” considered to be a lower level of learning than “conceptual knowledge.” Conceptual knowledge requires an understanding of the in- terrelationships of the basic elements within a larger structure that enable them to function together. Conceptual knowledge is required for classifi cation. Following this logic, some people argue that practicing retrieval of facts and exemplars would fall short as a strategy for comprehending general characteris- tics that are required for higher levels of intellectual behavior.

The bird classifi cation studies suggest the opposite: strategies of learning that help students identify and discern complex prototypes (family resemblances) can help them grasp the kinds of contextual and functional differences that go beyond the acquisition of simple forms of knowledge and reach into the higher sphere of comprehension. 8 Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 56 Improving Complex Mastery for Medical Students The distinction between straightforward knowledge of facts and deeper learning that permits fl exible use of the knowledge may be a little fuzzy, but it resonates with Douglas Larsen at Washington University Medical School in St. Louis, who says that the skills required for bird classifi cation are similar to those required of a doctor diagnosing what’s wrong with a patient. “The reason variety is important is it helps us see more nuances in the things that we can compare against,” he says. “That comes up a lot in medicine, in the sense that every patient visit is a test. There are many layers of explicit and implicit memory involved in the ability to discriminate be- tween symptoms and their interrelationships.” Implicit mem- ory is your automatic retrieval of past experience in interpret- ing a new one. For example, the patient comes in and gives you a story. As you listen, you’re consciously thinking through your mental library to see what fi ts, while also unconsciously polling your past experiences to help interpret what the pa- tient is telling you. “Then you’re left with making a judgment call,” Larsen says. 9 Larsen is a pediatric neurologist seeing patients in the uni- versity clinic and hospital. He’s a busy guy: in addition to practicing medicine, he supervises the work of physicians in training, he teaches, and as time permits, he conducts research into medical education, working in collaboration with cogni- tive psychologists. He’s drawing on all of these roles to redesign and strengthen the school’s training curriculum in pediatric neurology. As you’d expect, the medical school employs a wide spec- trum of instructional techniques. Besides classroom lectures Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 57 and labs, students practice resuscitations and other procedures on high- tech mannequins in three simulation centers the school maintains. Each “patient” is hooked up to monitors, has a heartbeat, blood pressure, pupils that dilate and constrict, and the ability to listen and speak, thanks to a controller who observes and operates the mannequin from a back room. The school also makes use of “standardized patients,” actors who follow scripts and exhibit symptoms the students are required to diagnose. The center is set up like a regular medical clinic, and students must show profi ciency in all aspects of a patient encounter, from bedside manner, physical exam skills, and re- membering to ask the full spectrum of pertinent questions to arriving at a diagnosis and treatment plan. From studies of these teaching methods, Larsen has drawn some interesting conclusions. First— and this may seem self- evident: you do better on a test to demonstrate your compe- tency at seeing patients in a clinic if your learning experience has involved seeing patients in a clinic. Simply reading about patients is not enough. However, on written fi nal exams, medi- cal students who have examined patients and those who have learned via written tests do equally well. The reason is that in a written test the student is being given considerable structure and being asked for specifi c information. When examining the patient, you have to come up on your own with the right mental model and the steps to follow. Having practiced these steps on patients or simulated patients improves per for mance relative to just reading about how to do it. In other words, the kind of retrieval practice that proves most effective is one that refl ects what you’ll be doing with the knowledge later. It’s not just what you know, but how you practice what you know that determines how well the learning serves you later. As the sports adage goes, “practice like you play and you will play Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 58 like you practice.” This conclusion lines up with other research into learning, and with some of the more sophisticated training practices in science and industry, including the increasingly broad use of simulators— not just for jet pi lots and medical students but for cops, towboat pi lots, and people in almost any fi eld you can name that requires mastery of complex knowl- edge and skills and where the stakes for getting it right are high. Book learning is not enough in these cases; actual hands- on practice is needed. Second, while it is important for a medical student to build breadth by seeing a wide variety of patients manifesting dif- ferent diseases, placing too much emphasis on variety runs the risk of underemphasizing repeated retrieval practice on the basics— on the typical way the disease presents itself in most patients. “There’s a certain set of diseases that we want you to know very well,” Larsen says. “So we’re going to have you see these standardized patients again and again, and assess your per- for mance until you really have that down and can show us, ‘I really do that well.’ It’s not either/or, variety versus repetition.

We need to make sure we’re appropriately balanced, and also recognize that we sometimes fall into the trap of familiarity.

‘I’ve already seen a bunch of patients with this problem, I don’t need to keep seeing them.’ But really, repeated retrieval practice is crucial to long- term retention, and it’s a critical as- pect of training.” A third critical aspect is practical experience. For a doctor, seeing patients provides a natural cycle of spaced retrieval practice, interleaving, and variety. “So much of medicine is based on learning by experience, which is why, after the fi rst two years, we take students out of the classroom and start putting them into clinical settings. A huge question is, what is Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 59 it about learning and experience that come together? We have lots of experiences we don’t learn from. What differentiates those that teach us something?”One form of practice that helps us learn from experience, as the neurosurgeon Mike Ebersold recounted in Chapter 2, is refl ection. Some people are more given to the act of refl ection than others, so Doug Larsen has broadened his research to study how you might structure refl ection as an integral part of the training, helping students cultivate it as a habit. He is experimenting with requiring students to write daily or weekly summaries of what they did, how it worked, and what they might do differently next time to get better results. He specu- lates that daily refl ection, as a form of spaced retrieval prac- tice, is probably just as critical in the real- world application of medicine as quizzing and testing are in building competencies in medical school. What about the classroom lecture, or the typical in- service training conference that’s compressed over a couple of days?

Larsen fi gures his school’s interns spend 10 percent of their time sitting in conferences listening to lectures. It may be a talk on metabolic diseases, on different infectious diseases, or on different drugs. The speaker puts the PowerPoint slideshow up and starts going through it. Usually there’s lunch, and the docs eat, listen, and leave. “In my mind, considering how much forgetting occurs, it’s very discouraging that we’re putting so many resources into an activity that, the way it is currently done, learning research tells us is so in effec tive. Medical students and residents go to these conferences and they have no repeated exposure what- soever to it. It’s just a matter of happenstance whether they end up fi nally seeing a patient in the future whose problem relates back to the conference topic. Otherwise, they don’t study the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 60 material, they are certainly not tested on the material, they just listen then they walk out.”At a minimum, Larsen would like to see something done to interrupt the forgetting: give a quiz at the end of a conference and follow it with spaced retrieval practice. “Make quizzing a standard part of the culture and the curriculum. You just know every week you’re going to get in your email your ten questions that you need to work through.” He asks, “How are we designing education and training systems that prevent or at least intervene in the amount of forgetting that goes on, and making sure they’re systematic throughout the school in support of what we’re trying to ac- complish? As it stands now, medical resident programs are simply dictating: you have to have the curriculum, you have to have the conferences, and it ends there. They present these big conferences, they have all the faculty come through and give their talks. And in the end, what we actually accomplish is re- ally kind of minimal.” 10 These Principles Are Broadly Applicable College football might seem an incongruous place to look for a learning model, but a conversation with Coach Vince Dooley about the University of Georgia’s practice regime provides an intriguing case. Dooley is authoritative on the subject. As head coach of Bulldogs football from 1964– 1988, he piled up an astonish- ing 201 wins with only 77 losses and 10 tied games, winning six conference titles and a national championship. He went on to serve as the university’s athletic director, where he built one of the most impressive athletics programs in the country. We asked Coach Dooley how players go about mastering all the complexities of the game. His theories of coaching and Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 61 training revolve around the weekly cycle of one Saturday game to the next. In that short period there’s a lot to learn: studying the opposition’s type of game in the classroom, discussing offensive and defensive strategies for opposing it, taking the discussion onto the playing fi eld, breaking the strategies down to the movements of individual positions and trying them out, knitting the parts into a whole, and then repeating the moves until they run like clockwork. While all this is going on, the players must also keep their fundamental skills in top form: blocking, tackling, catching the ball, bringing the ball in, carry ing the ball. Dooley be- lieves that (1) you have to keep practicing the fundamentals from time to time, forever, so you keep them sharp, other- wise you’re cooked, but (2) you need to change it up in prac- tice because too much repetition is boring. The position coaches work with players individually on specifi c skills and then on how they’re playing their positions during team practice. What else? There’s practicing the kicking game. There’s the matter of each player’s mastery of the playbook. And there are the special plays from the team’s repertoire that often make the difference between winning and losing. In Dooley’s narrative, the special plays stand as exemplars of spaced learn- ing: they’re practiced only on Thursdays, so there’s always a week between sessions, and the plays are run in a varied sequence. With all this to be done, it’s not surprising that a critical aspect of the team’s success is a very specifi c daily and weekly schedule that interleaves the elements of individual and team practice. The start of every day’s practice is strictly focused on the fundamentals of each player’s position. Next, players prac- tice in small groups, working on maneuvers involving several positions. These parts are gradually brought together and run Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 62 as a team. Play is speeded up and slowed down, rehearsed mentally as well as physically. By midweek the team is run- ning the plays in real time, full speed.“You’re coming at it fast, and you’ve got to react fast,” Dooley said. “But as you get closer to game time, you slow it down again. Now it’s a kind of rehearsal without physical contact. The play basically starts out the same each time, but then what the opponent does changes it. So you’ve got to be able to adjust to that. You start into the motion and say, ‘If they react like this, then this is what you would do.’ You prac- tice adjustments. If you do it enough times in different situa- tions, then you’re able to do it pretty well in what ever comes up on the fi eld.” 11 How does a player get on top of his playbook? He takes it home and goes over the plays in his mind. He may walk through them. Everything in practice can’t be physically stren- uous, Dooley said, or you’d wear yourself out, “so if the play calls for you to step this way and then go the other way, you can rehearse that in your mind, maybe just lean your body as if to go that way. And then if something happens where you have to adjust, you can do that mentally. By reading the play- book, rehearsing it in your mind, maybe taking a step or two to walk through it, you simulate something happening. So that kind of rehearsal is added to what you get in the class- room and on the fi eld.” The fi nal quarterback meetings are held on Saturday morn- ing, reviewing the game plan and running through it mentally.

The offensive coaches can make all the plans they want to about the hypothetical game, but once play gets under way, the execution rests in the hands of the quarterback. For Coach Dooley’s team, it’s all there: retrieval, spacing, interleaving, variation, refl ection, and elaboration. The sea- soned quarterback going into Saturday’s game— mentally run- Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 63 ning through the plays, the reactions, the adjustments— is doing the same thing as the seasoned neurosurgeon who’s re- hearsing what’s about to unfold in the operating room. The Takeaway Here’s a quick rundown of what we know today about massed practice and its alternatives. Scientists will continue to deepen our understanding. We harbor deep convictions that we learn better through single- minded focus and dogged repetition, and these beliefs are validated time and again by the visible improvement that comes during “practice- practice- practice.” But scientists call this heightened per for mance during the acquisition phase of a skill “momentary strength” and distinguish it from “underlying habit strength.” The very techniques that build habit strength, like spacing, interleaving, and variation, slow visible acquisi- tion and fail to deliver the improvement during practice that helps to motivate and reinforce our efforts. 12 Cramming, a form of massed practice, has been likened to binge- and- purge eating. A lot goes in, but most of it comes right back out in short order. The simple act of spacing out study and practice in installments and allowing time to elapse between them makes both the learning and the memory stron- ger, in effect building habit strength. How big an interval, you ask? The simple answer: enough so that practice doesn’t become a mindless repetition. At a minimum, enough time so that a little forgetting has set in. A little forgetting between practice sessions can be a good thing, if it leads to more effort in practice, but you do not want so much forgetting that retrieval essentially involves relearning the material. The time periods between sessions of practice let memories consolidate. Sleep seems to play a large role in Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 64 memory consolidation, so practice with at least a day in be- tween sessions is good.Something as simple as a deck of fl ashcards can provide an example of spacing. Between repetitions of any individual card, you work through many others. The German scientist Sebastian Leitner developed his own system for spaced prac- tice of fl ashcards, known as the Leitner box. Think of it as a series of four fi le- card boxes. In the fi rst are the study materi- als (be they musical scores, hockey moves, or Spanish vocabu- lary fl ashcards) that must be practiced frequently because you often make mistakes in them. In the second box are the cards you’re pretty good at, and that box gets practiced less often than the fi rst, perhaps by a half. The cards in the third box are practiced less often than those in the second, and so on. If you miss a question, make mistakes in the music, fl ub the one- touch pass, you move it up a box so you will practice it more often. The underlying idea is simply that the better your mas- tery, the less frequent the practice, but if it’s important to retain, it will never disappear completely from your set of practice boxes. Beware of the familiarity trap: the feeling that you know something and no longer need to practice it. This familiarity can hurt you during self- quizzing if you take shortcuts. Doug Larsen says, “You have to be disciplined to say, ‘All right, I’m going to make myself recall all of this and if I don’t, what did I miss, how did I not know that?’ Whereas if you have an instructor- generated test or quiz, suddenly you have to do it, there’s an expectation, you can’t cheat, you can’t take mental shortcuts around it, you simply have to do that.” The nine quizzes Andy Sobel administers over the twenty- six meetings of his po liti cal economics course are a simple example of spaced retrieval practice, and of interleaving— because he rolls forward into each successive quiz questions pertaining to work from the beginning of the semester.

Created from liberty on 2021-11-21 07:19:56.

Mix Up Your Practice ê 65 Interleaving two or more subjects during practice also pro- vides a form of spacing. Interleaving can also help you develop your ability to discriminate later between different kinds of problems and select the right tool from your growing toolkit of solutions. In interleaving, you don’t move from a complete practice set of one topic to go to another. You switch before each prac- tice is complete. A friend of ours describes his own experience with this: “I go to a hockey class and we’re learning skating skills, puck handling, shooting, and I notice that I get frus- trated because we do a little bit of skating and just when I think I’m getting it, we go to stick handling, and I go home frustrated, saying, ‘Why doesn’t this guy keep letting us do these things until we get it?’ ” This is actually the rare coach who understands that it’s more effective to distribute practice across these different skills than polish each one in turn. The athlete gets frustrated because the learning’s not proceeding quickly, but the next week he will be better at all aspects, the skating, the stick handling, and so on, than if he’d dedicated each session to polishing one skill. Like interleaving, varied practice helps learners build a broad schema, an ability to assess changing conditions and adjust responses to fi t. Arguably, interleaving and variation help learners reach beyond memorization to higher levels of conceptual learning and application, building more rounded, deep, and durable learning, what in motor skills shows up as underlying habit strength. Something the researchers call “blocked practice” is easily mistaken for varied practice. It’s like the old LP rec ords that could only play their songs in the same sequence. In blocked practice, which is commonly (but not only) found in sports, a drill is run over and over. The player moves from one station to the next, performing a different maneuver at each station.

That’s how the LA Kings were practicing their one- touch pass Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 66 before they got religion and started changing it up. It would be like always practicing fl ashcards in the same order. You need to shuffl e your fl ashcards. If you always practice the same skill in the same way, from the same place on the ice or fi eld, in the same set of math problems, or during the same sequence in a fl ight simulator, you’re starving your learning on short rations of variety. Spacing, interleaving, and variability are natural features of how we conduct our lives. Every patient visit or football game is a test and an exercise in retrieval practice. Every routine traf- fi c stop is a test for a cop. And every traffi c stop is different, adding to a cop’s explicit and implicit memory and, if she pays attention, making her more effective in the future. The com- mon term is “learning from experience.” Some people never seem to learn. One difference, perhaps, between those who do and don’t is whether they have cultivated the habit of re- fl ection. Refl ection is a form of retrieval practice (What hap- pened? What did I do? How did it work out?), enhanced with elaboration (What would I do differently next time?). As Doug Larsen reminds us, the connections between the neurons in the brain are very plastic. “Making the brain work is actually what seems to make a difference— bringing in more complex networks, then using those circuits repeatedly, which makes them more robust.” Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

67 When Mia Blun\fe\b\bo, age twenty- three, fi rst lieutenant, U.S. Marine Corps, was billeted to logistics in Okinawa, she had to get her ticket punched at jump school.

Describing that moment two years later, she said, “I hate fall- ing, that feeling in your chest. There’s not a day in my life I wanted to jump out of an airplane. I wouldn’t even go down a water slide until I was in middle school. But I was in charge of a platoon of Marines who rigged parachutes and jumped out of airplanes and dropped cargo. It’s one of the most sought- out billets as a logistics offi cer, very hard to get. My command- ing offi cer said, you know, ‘You will be air delivery platoon commander. If you don’t want to do that, I’ll put you some- where else and we’ll let the next guy have that job.’ There’s no way I could let somebody else have this job that everybody wanted. So I looked him straight in the face and said, ‘Yes, sir, I’ll jump out of planes.’ ” 1 4 Embrace Diﬃ culties Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 68 Mia is fi ve feet seven inches of blonde ambition. Her father, Frank, ex- marine, is in awe. “She’ll do more pull- ups than most of the guys in her class. She has the Mary land state record in the bench press, she was sixth in the NCAA for powerlifting.

Very soft- spoken; you just don’t see it coming.” When we had Mia to ourselves, we asked her if Frank was blowing smoke.

She laughed. “He likes to exaggerate.” But when pressed, she admitted to the facts. Until recently, women in the Marines were required to do fl ex arm hangs instead of pull- ups (where the chin crosses the plane of the pull- up bar), but the newly toughened rules effective in 2014 require a minimum of three pull- ups, the same as the minimum for men. Targets are eight pull- ups for women, twenty for men. Mia does thirteen and is shooting for twenty. As a student at the Naval Academy, she qualifi ed two years in a row for nationals in powerlifting— three sets each of bench press, squats, and dead lifts— setting Mary land state rec ords. So we know she’s tough. An aversion to falling is an in- stinctual refl ex for self- preservation, but her decision to take the assignment was a foregone conclusion, the kind of grit the Marines and the Blundettos are known for. Mia has a sister and two brothers. They’re all active duty Marines. As it turned out, the third time Mia threw herself out the jump door of a C130 troop transport at 1,250 feet, she plum- meted right onto another soldier’s infl ated parachute. But we’re getting ahead of the story. We’re interested in her jump school training because it’s a great example of how some diffi culties that elicit more effort and that slow down learning— spacing, interleaving, mixing up practice, and others— will more than compensate for their incon ve nience by making the learning stronger, more precise, and more enduring. Short- term impediments that make for stronger learning have come to be called desirable diffi culties, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 69 a term coined by the psychologists Elizabeth and Robert Bjork. 2 The army’s jump school at Fort Benning, Georgia, is designed to make sure you get it right and get it done, and it’s a model of learning through desirable diffi culty. You are not allowed to carry a notebook and write notes. You listen, watch, re- hearse, and execute. Jump school is a place where testing is the principal instructional medium, and the test is in the doing.

And, like all things military, jump school adheres to a strict protocol. Get it right or get the boot. The parachute landing fall, or PLF in military parlance, is a technique of hitting the ground and rolling in a way that dis- tributes the impact over the balls of your feet, the side of your calf, the side of your thigh, the side of your hip, and the side of your back. There are six possible directions in which to execute the fall along the length of your body, determined by conditions in the moment such as the direction of your drift, the terrain, wind, and whether you’re oscillating as you ap- proach the ground. In your fi rst exposure to this essential skill of parachuting, you stand in a gravel pit where the PLF is ex- plained and demonstrated. Then you try it: you practice falling along different planes of the body, you get corrective feedback, and you practice it again. Over the ensuing week the diffi culty is notched up. You stand on a platform two feet off the ground. On the command “Ready,” you rock up on the balls of your feet, feet and knees together, arms skyward. On the command “land,” you jump off the wall and execute your PLF. The test becomes more diffi cult. You clip yourself onto a zip line a dozen feet off the ground, grab onto an overhead T-bar, and drift down to a landing site, where, on command, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 70 you release and execute the PLF. You practice falling to the right and left, forward and backward, mixing it up.The diffi culty is increased again. You climb to a platform twelve feet off the ground, where you practice strapping on your harness, checking gear using the buddy system, and jump- ing through a mockup of an airplane jump door. The harness has risers like those from a parachute, hooked to a zip line but allowing for the same long arc of suspension, and when you jump, you have the momentary downward sensation of free fall, followed by the broad oscillations of suspension as you move along the cable, getting familiar with the motions of a real jump. But at the bottom it’s the instructor, not you, who pulls the release and drops you the last two or three feet to earth, so now you’re executing your fall randomly, from all directions, simulating what’s to come. Next, you climb a thirty- four- foot tower to practice all the elements of a jump and the choreography of a mass exit from the aircraft, learning how it feels to fall from a height, how to deal with equipment malfunctions, how to jump with a load of heavy combat equipment. Through demonstration and simulation, in escalating lev- els of diffi culty that must be mastered in order to progress from one to the next, you learn how to board the aircraft as a part of a jump crew and participate in the command sequence of thirty troops positioning for a mass exit over a drop zone.

How to get out the jump door correctly, how to count one- thousand, two- thousand, three- thousand, four- thousand and feel your chute deploy, or if you get to six- thousand, to pull the cord on your reserve chute; how to deal with twisted sus- pension lines, avoid collisions, hold into the wind, sort out a tangled control line; how to avoid stealing air from another jumper; the contingencies for landing in trees, water, or power Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 71 lines; how to jump by day or night, in different wind and weather.The knowledge and skills to be acquired are many, and practice is spaced and interleaved, both by default, as you wait your turn at each of the staging areas, airplane mock- ups, jump platforms, and harness mechanisms, and by necessity, in order to cover all that must be mastered and integrate the disparate components. Finally, if you make it to week 3 with- out washing out, you jump for real, making fi ve exits from a military transport. With successful completion of the training and fi ve successful jumps, you earn your jump wings and Air- borne certifi cate. On Mia’s third jump, she was fi rst in line at the port jump door with fourteen jumpers queued behind her and another fourteen queued behind the guy standing at the opposite door.

“So what the fi rst person does, in this case me, you hand off your static line to the Sergeant Airborne, and there’s a light and it’s red or green, and you get the one- minute warning, then the thirty- second warning. I’m standing at this door for a few minutes and it’s beautiful. It’s probably one of the pretti- est things I’ve ever seen, but I was terrifi ed. There was nothing to get in my way, nothing I had to think about except just wait- ing, waiting for the ‘Go!’ The guy at the other door went, then I jumped, and I’m counting one- thousand, two- thousand—and suddenly, at four thousand, I had a green parachute wrapped all around me! I’m thinking, There’s no way this can be my parachute! I’d felt my chute open, I’d felt that lift. I realized that I was on top of the fi rst jumper, so I just sort of swam out of his parachute and steered away from him.” Jumpers are staggered, but in the four turbulent seconds until your chute opens you have neither awareness nor control over your proximity to other jumpers. The incident, which Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 72 amounted to nothing, thanks to her training, is telling none- theless. Had it frightened her? Not at all, she said. Mia was prepared to handle it, and her confi dence gave her the cool to “just sort of swim out.”It’s one thing to feel confi dent of your knowledge; it’s some- thing else to demonstrate mastery. Testing is not only a power- ful learning strategy, it is a potent reality check on the accuracy of your own judgment of what you know how to do. When confi dence is based on repeated per for mance, demonstrated through testing that simulates real- world conditions, you can lean into it. Facing the jump door may always reawaken feel- ings of terror, but the moment she’s out, Mia says, the fear evaporates. How Learning Occurs To help you understand how diffi culty can be desirable, we’ll briefl y describe here how learning occurs. Encoding Let’s imagine you’re Mia, standing in a gravel pit watching a jump instructor explain and demonstrate the parachute land- ing fall. The brain converts your perceptions into chemical and electrical changes that form a mental repre sen ta tion of the patterns you’ve observed. This pro cess of converting sen- sory perceptions into meaningful repre sen ta tions in the brain is still not perfectly understood. We call the pro cess encoding, and we call the new repre sen ta tions within the brain memory traces. Think of notes jotted or sketched on a scratchpad, our short- term memory. Much of how we run our day- to- day lives is guided by the ephemera that clutter our short- term memory and are, fortu- nately, soon forgotten— how to jigger the broken latch on the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 73 locker you used when you suited up at the gym today; re- membering to stop for an oil change after your workout. But the experiences and learning that we want to salt away for the future must be made stronger and more durable— in Mia’s case, the distinctive moves that will enable her to hit the ground without breaking an ankle, or worse. 3 Consolidation The pro cess of strengthening these mental repre sen ta tions for long- term memory is called consolidation. New learning is labile: its meaning is not fully formed and therefore is easily altered. In consolidation, the brain reorganizes and stabilizes the memory traces. This may occur over several hours or lon- ger and involves deep pro cessing of the new material, during which scientists believe that the brain replays or rehearses the learning, giving it meaning, fi lling in blank spots, and making connections to past experiences and to other knowledge al- ready stored in long- term memory. Prior knowledge is a pre- requisite for making sense of new learning, and forming those connections is an important task of consolidation. Mia’s con- siderable athletic skills, physical self- awareness, and prior ex- perience represent a large body of knowledge to which the elements of a successful PLF would fi nd many connections. As we’ve noted, sleep seems to help memory consolidation, but in any case, consolidation and transition of learning to long- term storage occurs over a period of time. An apt analogy for how the brain consolidates new learn- ing may be the experience of composing an essay. The fi rst draft is rangy, imprecise. You discover what you want to say by trying to write it. After a couple of revisions you have sharp- ened the piece and cut away some of the extraneous points.

You put it aside to let it ferment. When you pick it up again Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 74 a day or two later, what you want to say has become clearer in your mind. Perhaps you now perceive that there are three main points you are making. You connect them to examples and supporting information familiar to your audience. You rearrange and draw together the elements of your argument to make it more effective and elegant.Similarly, the pro cess of learning something often starts out feeling disor ga nized and unwieldy; the most important aspects are not always salient. Consolidation helps or ga nize and solid- ify learning, and, notably, so does retrieval after a lapse of some time, because the act of retrieving a memory from long- term storage can both strengthen the memory traces and at the same time make them modifi able again, enabling them, for example, to connect to more recent learning. This pro cess is called recon- solidation. This is how retrieval practice modifi es and strength- ens learning. Suppose that on day 2 of jump school, you’re put on the spot to execute your parachute landing fall and you struggle to recall the correct posture and compose yourself— feet and knees together, knees slightly bent, eyes on the horizon— but in the refl ex to break your fall you throw your arm out, for- getting to pull your elbows tight to your sides. You could have broken the arm or dislocated your shoulder if this were the real deal. This effort to reconstruct what you learned the day before is ragged, but in making it, critical elements of the ma- neuver come clearer and are reconsolidated for stronger mem- ory. If you’re practicing something over and over in rapid- fi re fashion, whether it’s your parachute landing fall or the conju- gation of foreign verbs, you’re leaning on short- term memory, and very little mental effort is required. You show gratifying improvement rather quickly, but you haven’t done much to strengthen the underlying repre sen ta tion of those skills. Your per for mance in the moment is not an indication of durable Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 75 learning. On the other hand, when you let the memory recede a little, for example by spacing or interleaving the practice, retrieval is harder, your per for mance is less accomplished, and you feel let down, but your learning is deeper and you will retrieve it more easily in the future. 4 Retrieval Learning, remembering, and forgetting work together in in- teresting ways. Durable, robust learning requires that we do two things. First, as we recode and consolidate new material from short- term memory into long- term memory, we must anchor it there securely. Second, we must associate the mate- rial with a diverse set of cues that will make us adept at recall- ing the knowledge later. Having effective retrieval cues is an aspect of learning that often goes overlooked. The task is more than committing knowledge to memory. Being able to retrieve it when we need it is just as important. The reason we don’t remember how to tie knots even af- ter we’ve been taught is because we don’t practice and apply what we’ve learned. Say you’re in the city park one day and come across an Ea gle Scout teaching knots. On a whim you take an hour’s lesson. He demonstrates eight or ten specimens, explains what each is useful for, has you practice tying them, and sends you away with a short length of rope and a cheat sheet. You head home committed to learning these knots, but life is full, and you fail to practice them. They are soon forgot- ten, and this story could end there, with no learning. But then, as it happens, the following spring you buy a small fi shing boat, and you want to attach an anchor on a line. With rope in hand and feeling mildly stumped, you recall from your les- son that there was a knot for putting a loop in the end of a line. You are now practicing retrieval. You fi nd your cheat Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 76 sheet and relearn how to tie a bowline. You put a small loop in the rope and then take the short end and draw it through, silently reciting the little memory device you were given: the rabbit comes up from his hole, goes around the tree, and goes back down. Retrieval again. A little snugging- up, and there you have your knot, a dandy piece of scoutcraft of the kind you’d always fancied knowing. Later, you put a piece of rope beside the chair where you watch TV and practice the bow- line during commercials. You are doing spaced practice. Over the coming weeks you’re surprised at how many little jobs are easier if you have a piece of rope with a loop in the end. More spaced practice. By August you have discovered every possi- ble use and purpose in your life for the bowline knot. Knowledge, skills, and experiences that are vivid and hold signifi cance, and those that are periodically practiced, stay with us. If you know you’re soon to throw yourself out of a troop transport, you listen up good when they’re telling you when and how to pull the rip cord on your reserve chute, or what can go wrong at twelve hundred feet and how to “just sort of swim out of it.” The mental rehearsal you conduct while lying in your bunk too tired to sleep and wishing the next day was already over and well- jumped is a form of spaced practice, and that helps you, too. Extending Learning: Updating Retrieval Cues There’s virtually no limit to how much learning we can re- member as long as we relate it to what we already know. In fact, because new learning depends on prior learning, the more we learn, the more possible connections we create for further learning. Our retrieval capacity, though, is severely limited.

Most of what we’ve learned is not accessible to us at any given moment. This limitation on retrieval is helpful to us: if every Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 77 memory were always readily to hand, you would have a hard time sorting through the sheer volume of material to put your fi nger on the knowledge you need at the moment: where did I put my hat, how do I sync my electronic devices, what goes into a perfect brandy Manhattan? Knowledge is more durable if it’s deeply entrenched, mean- ing that you have fi rmly and thoroughly comprehended a concept, it has practical importance or keen emotional weight in your life, and it is connected with other knowledge that you hold in memory. How readily you can recall knowledge from your internal archives is determined by context, by recent use, and by the number and vividness of cues that you have linked to the knowledge and can call on to help bring it forth. 5 Here’s the tricky part. As you go through life, you often need to forget cues associated with older, competing memo- ries so as to associate them successfully with new ones. To learn Italian in middle age, you may have to forget your high school French, because every time you think “to be” and hope to come up with the Italian essere, up pops etre, despite your most earnest intentions. Traveling in En gland, you have to suppress your cues to drive on the right side of the road so you can establish reliable cues to stay on the left. Knowledge that is well entrenched, like real fl uency in French or years of experience driving on the right side of the road, is easily re- learned later, after a period of disuse or after being interrupted by competition for retrieval cues. It’s not the knowledge itself that has been forgotten, but the cues that enable you to fi nd and retrieve it. The cues for the new learning, driving on the left, displace those for the old, driving on the right (if we are lucky). The paradox is that some forgetting is often essential for new learning. 6 When you change from a PC to a Mac, or from one Windows platform to another, you have to do enormous Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 78 forgetting in order to learn the architecture of the new system and become adept at manipulating it so readily that your at- tention can focus on doing your work and not on working the machine. Jump school training provides another example:

After their military ser vice, many paratroopers take an inter- est in smoke jumping. Smokejumpers use different airplanes, different equipment, and different jump protocols. Having trained at the army’s jump school is cited as a distinct disad- vantage for smoke jumping, because you have to unlearn one set of procedures that you have practiced to the point of re- fl ex and replace them with another. Even in cases where both bodies of learning seem so similar to the uninitiated— jumping out of an airplane with a parachute on your back— you may have to forget the cues to a complex body of learning that you possess if you are to acquire a new one. We know this problem of reassigning cues to memory from our own lives, even on the simplest levels. When our friend Jack fi rst takes up with Joan, we sometimes call the couple “Jack and Jill,” as the cue “Jack and” pulls up the old nursery rhyme that’s so thoroughly embedded in memory. About the time we have “Jack and” reliably cuing “Joan,” alas, Joan throws him over, and he takes up with Jenny. Good grief!

Half of the time that we mean to say Jack and Jenny we catch ourselves saying Jack and Joan. It would have been easier had Jack picked up with Katie, so that the trailing K sound in his name handed us off to the initiating K in hers, but no such luck. Alliteration can be a handy cue, or a subversive one. In all of this turmoil you don’t forget Jill, Joan, or Jenny, but you “repurpose” your cues so that you can keep pace with the changing opera of Jack’s life. 7 It is a critical point that as you learn new things, you don’t lose from long- term memory most of what you have learned well in life; rather, through disuse or the reassignment of cues, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 79 you forget it in the sense that you’re unable to call it up easily.

For example, if you’ve moved several times, you may not be able to recall a previous address from twenty years ago. But if you are given a multiple choice test for the address, you can probably pick it out easily, for it still abides, as it were, in the uncleaned closet of your mind. If you have ever immersed yourself in writing stories of your past, picturing the people and places of earlier days, you may have been surprised by the memories that started fl ooding back, things long forgotten now coming to mind. Context can unleash memories, as when the right key works to open an old lock. In Marcel Proust’s Remembrance of Things Past, the narrator grieves over his inability to recall the days of his adolescence in the French village of his aunt and uncle, until one day the taste of a cake dipped in lime blossom tea brings it all rushing back, all the people and events he thought had long since been lost to time.

Most people have experiences like Proust’s when a sight or sound or smell brings back a memory in full force, even some episode you have not thought about in years. 8 Easier Isn’t Better Psychologists have uncovered a curious inverse relationship between the ease of retrieval practice and the power of that practice to entrench learning: the easier knowledge or a skill is for you to retrieve, the less your retrieval practice will ben- efi t your retention of it. Conversely, the more effort you have to expend to retrieve knowledge or skill, the more the practice of retrieval will entrench it. Not long ago the California Polytechnic State University baseball team, in San Luis Obispo, became involved in an in- teresting experiment in improving their batting skills. They were all highly experienced players, adept at making solid contact Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 80 with the ball, but they agreed to take extra batting practice twice a week, following two different practice regimens, to see which type of practice produced better results.Hitting a baseball is one of the hardest skills in sports. It takes less than half a second for a ball to reach home plate. In this instant, the batter must execute a complex combination of perceptual, cognitive, and motor skills: determining the type of pitch, anticipating how the ball will move, and aiming and timing the swing to arrive at the same place and moment as the ball. This chain of perceptions and responses must be so deeply entrenched as to become automatic, because the ball is in the catcher’s mitt long before you can even begin to think your way through how to connect with it. Part of the Cal Poly team practiced in the standard way.

They practiced hitting forty- fi ve pitches, evenly divided into three sets. Each set consisted of one type of pitch thrown fi fteen times. For example, the fi rst set would be fi fteen fast- balls, the second set fi fteen curveballs, and the third set fi f- teen changeups. This was a form of massed practice. For each set of 15 pitches, as the batter saw more of that type, he got gratifyingly better at anticipating the balls, timing his swings, and connecting. Learning seemed easy. The rest of the team were given a more diffi cult practice regimen: the three types of pitches were randomly interspersed across the block of forty- fi ve throws. For each pitch, the bat- ter had no idea which type to expect. At the end of the forty- fi ve swings, he was still struggling somewhat to connect with the ball. These players didn’t seem to be developing the profi - ciency their teammates were showing. The interleaving and spacing of different pitches made learning more arduous and feel slower. The extra practice sessions continued twice weekly for six weeks. At the end, when the players’ hitting was assessed, the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 81 two groups had clearly benefi ted differently from the extra practice, and not in the way the players expected. Those who had practiced on the randomly interspersed pitches now displayed markedly better hitting relative to those who had practiced on one type of pitch thrown over and over. These results are all the more interesting when you consider that these players were already skilled hitters prior to the extra training. Bringing their per for mance to an even higher level is good evidence of a training regimen’s effectiveness. Here again we see the two familiar lessons. First, that some diffi culties that require more effort and slow down apparent gains— like spacing, interleaving, and mixing up practice— will feel less productive at the time but will more than com- pensate for that by making the learning stronger, precise, and enduring. Second, that our judgments of what learning strate- gies work best for us are often mistaken, colored by illusions of mastery. When the baseball players at Cal Poly practiced curveball after curveball over fi fteen pitches, it became easier for them to remember the perceptions and responses they needed for that type of pitch: the look of the ball’s spin, how the ball changed direction, how fast its direction changed, and how long to wait for it to curve. Per for mance improved, but the growing ease of recalling these perceptions and responses led to little durable learning. It is one skill to hit a curveball when you know a curveball will be thrown; it is a different skill to hit a curveball when you don’t know it’s coming. Baseball players need to build the latter skill, but they often practice the former, which, being a form of massed practice, builds per for mance gains on short- term memory. It was more challenging for the Cal Poly batters to retrieve the necessary skills when practice involved random pitches. Meeting that challenge made the per for mance gains painfully slow but also long lasting.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 82 This paradox is at the heart of the concept of desirable diffi culties in learning: the more effort required to retrieve (or, in effect, relearn) something, the better you learn it. In other words, the more you’ve forgotten about a topic, the more effective relearning will be in shaping your permanent knowledge. 9 How Effort Helps Reconsolidating Memory Effortful recall of learning, as happens in spaced practice, re- quires that you “reload” or reconstruct the components of the skill or material anew from long- term memory rather than mindlessly repeating them from short- term memory. 10 During this focused, effortful recall, the learning is made pliable again:

the most salient aspects of it become clearer, and the conse- quent reconsolidation helps to reinforce meaning, strengthen connections to prior knowledge, bolster the cues and retrieval routes for recalling it later, and weaken competing routes.

Spaced practice, which allows some forgetting to occur be- tween sessions, strengthens both the learning and the cues and routes for fast retrieval when that learning is needed again, as when the pitcher tries to surprise the batter with a curveball after pitching several fastballs. The more effort that is required to recall a memory or to execute a skill, provided that the effort succeeds, the more the act of recalling or exe- cuting benefi ts the learning. 11 Massed practice gives us the warm sensation of mastery because we’re looping information through short- term mem- ory without having to reconstruct the learning from long- term memory. But just as with rereading as a study strategy, the fl uency gained through massed practice is transitory, and our sense of mastery is illusory. It’s the effortful pro cess of Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 83 reconstructing the knowledge that triggers reconsolidation and deeper learning. Creating Mental Models With enough effortful practice, a complex set of interrelated ideas or a sequence of motor skills fuse into a meaningful whole, forming a mental model somewhat akin to a “brain app”. Learning to drive a car involves a host of simultaneous actions that require all of our powers of concentration and dexterity while we are learning them. But over time, these combinations of cognition and motor skills— for example, the perceptions and maneuvers required to parallel park or manipulate a stick shift— become ingrained as sets of mental models associated with driving. Mental models are forms of deeply entrenched and highly effi cient skills (seeing and un- loading on a curveball) or knowledge structures (a memo- rized sequence of chess moves) that, like habits, can be adapted and applied in varied circumstances. Expert per for mance is built through thousands of hours of practice in your area of expertise, in varying conditions, through which you accumu- late a vast library of such mental models that enables you to correctly discern a given situation and instantaneously select and execute the correct response. Broadening Mastery Retrieval practice that you perform at different times and in different contexts and that interleaves different learning ma- terial has the benefi t of linking new associations to the mate- rial. This pro cess builds interconnected networks of knowl- edge that bolster and support mastery of your fi eld. It also multiplies the cues for retrieving the knowledge, increasing the versatility with which you can later apply it.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 84 Think of an experienced chef who has internalized the complex knowledge of how fl avors and textures interact; how ingredients change form under heat; the differing effects to be achieved with a saucepan versus a wok, with copper versus cast iron. Think of the fl y fi sher who can sense the presence of trout and accurately judge the likely species, make the right choice of dry fl y, nymph, or streamer, judge the wind, and know how and where to drop that fl y to make the trout rise.

Think of the kid on the BMX bike who can perform bunny- hops, tail whips, 180s, and wall taps off the features of an unfamiliar streetscape. Interleaving and variation mix up the contexts of practice and the other skills and knowledge with which the new material is associated. This makes our mental models more versatile, enabling us to apply our learning to a broader range of situations.

Fostering Conceptual Learning How do humans learn concepts, for example the difference between dogs and cats? By randomly coming across dissimi- lar examples— Chihuahuas, tabby cats, Great Danes, picture book lions, calico cats, Welsh terriers. Spaced and interleaved exposure characterizes most of humans’ normal experience.

It’s a good way to learn, because this type of exposure strength- ens the skills of discrimination— the pro cess of noticing par- ticulars (a turtle comes up for air but a fi sh doesn’t)— and of induction: surmising the general rule (fi sh can breathe in wa- ter). Recall the interleaved study of birds in one case, and of paintings in another, that helped learners distinguish between bird types or the works of different paint ers while at the same time learning to identify underlying commonalities of the examples within a species or an artist’s body of work. When asked about their preferences and beliefs, the learners thought Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 85 that the experience of studying multiple examples of one spe- cies of bird before studying examples of another species re- sulted in better learning. But the interleaved strategy, which was more diffi cult and felt clunky, produced superior discrimi- nation of differences between types, without hindering the ability to learn commonalities within a type. As was true for the baseball players’ batting practice, interleaving produced diffi culty in retrieving past examples of a par tic u lar species, which further solidifi ed the learning of which birds are repre- sentative of a par tic u lar species. The diffi culty produced by interleaving provides a second type of boost to learning. Interleaved practice of related but dissimilar geometric solids requires that you notice similari- ties and differences in order to select the correct formula for computing the volume. It’s thought that this heightened sensi- tivity to similarities and differences during interleaved prac- tice leads to the encoding of more complex and nuanced repre sen ta tions of the study material— a better understanding of how specimens or types of problems are distinctive and why they call for a different interpretation or solution. Why a northern pike will strike a spoon or a crankbait, say, but a bass will happily powder his nose until you see fi t to throw him a grub or a popper. 12 Improving Versatility The retrieval diffi culties posed by spacing, interleaving, and variation are overcome by invoking the same mental pro- cesses that will be needed later in applying the learning in ev- eryday settings. By mimicking the challenges of practical ex- perience, these learning strategies conform to the admonition to “practice like you play, and you’ll play like you practice,” improving what scientists call transfer of learning, which is Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 86 the ability to apply what you’ve learned in new settings. In the Cal Poly batting practice experiment, the act of overcoming the diffi culties posed by random types of pitches built a broader “vocabulary” of mental pro cesses for discerning the nature of the challenge (e.g., what the pitcher is throwing) and selecting among possible responses than did the narrower mental pro cesses suffi cient for excelling during massed, non- varied experience. Recall the grade school students who proved more adept at tossing beanbags into three- foot baskets after having practiced tossing into two- and four- foot baskets, com- pared to the students who only practiced tossing into three- foot basket. Recall the increasing diffi culty and complexity of the simulation training in jump school, or the cockpit simula- tor of Matt Brown’s business jet. Priming the Mind for Learning When you’re asked to struggle with solving a problem before being shown how to solve it, the subsequent solution is better learned and more durably remembered. When you’ve bought your fi shing boat and are attempting to attach an anchor line, you’re far more likely to learn and remember the bowline knot than when you’re standing in a city park being shown the bow- line by a Boy Scout who thinks you would lead a richer life if you had a handful of knots in your repertoire. Other Learning Strategies That Incorporate Desirable Difﬁ culties We usually think of interference as a detriment to learning, but certain kinds of interference can produce learning bene- fi ts, and the positive effects are sometimes surprising. Would you rather read an article that has normal type or type that’s somewhat out of focus? Almost surely you would opt for the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 87 former. Yet when text on a page is slightly out of focus or presented in a font that is a little diffi cult to decipher, people recall the content better. Should the outline of a lecture follow the precise fl ow of a chapter in a textbook, or is it better if the lecture mismatches the text in some ways? It turns out that when the outline of a lecture proceeds in a different order from the textbook passage, the effort to discern the main ideas and reconcile the discrepancy produces better recall of the con- tent. In another surprise, when letters are omitted from words in a text, requiring the reader to supply them, reading is slowed, and retention improves. In all of these examples, the change from normal pre sen ta tion introduces a diffi culty— disruption of fl uency— that makes the learner work harder to construct an interpretation that makes sense. The added effort increases comprehension and learning. (Of course, learning will not improve if the diffi culty completely obscures the meaning or cannot be overcome.) 13 The act of trying to answer a question or attempting to solve a problem rather than being presented with the information or the solution is known as generation. Even if you’re being quizzed on material you’re familiar with, the simple act of fi ll- ing in a blank has the effect of strengthening your memory of the material and your ability to recall it later. In testing, being required to supply an answer rather than select from multiple choice options often provides stronger learning benefi ts. Hav- ing to write a short essay makes them stronger still. Overcom- ing these mild diffi culties is a form of active learning, where students engage in higher- order thinking tasks rather than passively receiving knowledge conferred by others. When you’re asked to supply an answer or a solution to something that’s new to you, the power of generation to aid Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 88 learning is even more evident. One explanation for this effect is the idea that as you cast about for a solution, retrieving re- lated knowledge from memory, you strengthen the route to a gap in your learning even before the answer is provided to fi ll it and, when you do fi ll it, connections are made to the related material that is fresh in your mind from the effort. For ex- ample, if you’re from Vermont and are asked to name the capital of Texas you might start ruminating on possibilities:

Dallas? San Antonio? El Paso? Houston? Even if you’re un- sure, thinking about alternatives before you hit on (or are given) the correct answer will help you. (Austin, of course.) Wrestling with the question, you rack your brain for some- thing that might give you an idea. You may get curious, even stumped or frustrated and acutely aware of the hole in your knowledge that needs fi lling. When you’re then shown the so- lution, a light goes on. Unsuccessful attempts to solve a prob- lem encourage deep pro cessing of the answer when it is later supplied, creating fertile ground for its encoding, in a way that simply reading the answer cannot. It’s better to solve a problem than to memorize a solution. It’s better to attempt a solution and supply the incorrect answer than not to make the attempt. 14 The act of taking a few minutes to review what has been learned from an experience (or in a recent class) and asking yourself questions is known as refl ection. After a lecture or reading assignment, for example, you might ask yourself:

What are the key ideas? What are some examples? How do these relate to what I already know? Following an experience where you are practicing new knowledge or skills, you might ask: What went well? What could have gone better? What Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 89 might I need to learn for better mastery, or what strategies might I use the next time to get better results?Refl ection can involve several cognitive activities we have discussed that lead to stronger learning. These include re- trieval (recalling recently learned knowledge to mind), elabo- ration (for example, connecting new knowledge to what you already know), and generation (for example, rephrasing key ideas in your own words or visualizing and mentally rehears- ing what you might do differently next time). One form of refl ection that is gaining currency in class- room settings is called “write to learn.” In essence, students refl ect on a recent class topic in a brief writing assignment, where they may express the main ideas in their own words and relate them to other concepts covered in class, or perhaps outside class. (For an example, read in Chapter 8 about the “learning paragraphs” Mary Pat Wenderoth assigns her stu- dents in her human physiology course.) The learning benefi ts from the various cognitive activities that are engaged during refl ection (retrieval, elaboration, generation) have been well established through empirical studies. An interesting recent study specifi cally examined “write to learn” as a learning tool. Over eight hundred college students in several introductory psychology classes listened to lectures throughout the semester. Following the pre sen ta tion of a key concept within a given lecture, the instructor asked students to write to learn. Students generated their own written sum- maries of the key ideas, for example restating concepts in their own words and elaborating on the concepts by generating examples of them. For other key concepts presented during the lecture, students were shown a set of slides summarizing the concepts and spent a few minutes copying down key ideas and examples verbatim from the slide.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 90 What was the result? On exams administered during the semester, the students were asked questions that assessed their understanding of the key concepts that they had worked on learning. They scored signifi cantly (approximately half a let- ter grade) better on the ones they had written about in their own words than on those they had copied, showing that it was not simply exposure to the concepts that produced the learning benefi t. In follow- up tests approximately two months later to mea sure retention, the benefi ts of writing to learn as a form of refl ection had dropped but remained robust. 15 Failure and the Myth of Errorless Learning In the 1950s and 1960s, the psychologist B. F. Skinner advo- cated the adoption of “errorless learning” methods in educa- tion in the belief that errors by learners are counterproductive and result from faulty instruction. The theory of errorless learning gave rise to instructional techniques in which learn- ers were spoonfed new material in small bites and immedi- ately quizzed on them while they still remained on the tongue, so to speak, fresh in short- term memory and easily spit out onto the test form. There was virtually no chance of making an error. Since those days we’ve come to understand that re- trieval from short- term memory is an in effec tive learning strategy and that errors are an integral part of striving to in- crease one’s mastery over new material. Yet in our Western culture, where achievement is seen as an indicator of ability, many learners view errors as failure and do what they can to avoid committing them. The aversion to failure may be rein- forced by instructors who labor under the belief that when learners are allowed to make errors it’s the errors that they will learn. 16 This is a misguided impulse. When learners commit errors and are given corrective feedback, the errors are not learned.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 91 Even strategies that are highly likely to result in errors, like asking someone to try to solve a problem before being shown how to do it, produce stronger learning and retention of the correct information than more passive learning strategies, pro- vided there is corrective feedback. Moreover, people who are taught that learning is a struggle that often involves making errors will go on to exhibit a greater propensity to tackle tough challenges and will tend to see mistakes not as failures but as lessons and turning points along the path to mastery.

To see the truth of this, look no further than the kid down the hall who is deeply absorbed in working his avatar up through the levels of an action game on his Xbox video console.A fear of failure can poison learning by creating aversions to the kinds of experimentation and risk taking that charac- terize striving, or by diminishing per for mance under pressure, as in a test setting. In the latter instance, students who have a high fear of making errors when taking tests may actually do worse on the test because of their anxiety. Why? It seems that a signifi cant portion of their working memory capacity is ex- pended to monitor their per for mance (How am I doing? Am I making mistakes?), leaving less working memory capacity available to solve the problems posed by the test. “Working memory” refers to the amount of information you can hold in mind while working through a problem, especially in the face of distraction. Everyone’s working memory is severely limited, some more than others, and larger working memory capaci- ties correlate with higher IQs. To explore this theory about how fear of failure reduces test per for mance, sixth graders in France were given very dif- fi cult anagram problems that none of them could solve. After struggling unsuccessfully with the problems, half of the kids received a ten- minute lesson in which they were taught that diffi culty is a crucial part of learning, errors are natural and to be expected, and practice helps, just as in learning to ride a Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 92 bicycle. The other kids were simply asked how they had gone about trying to solve the anagrams. Then both groups were given a diffi cult test whose results provided a mea sure of working memory. The kids who had been taught that errors are a natural part of learning showed signifi cantly better use of working memory than did the others. These children did not expend their working memory capacity in agonizing over the diffi culty of the task. The theory was further tested in varia- tions of the original study. The results support the fi nding that diffi culty can create feelings of incompetence that engender anxiety, which in turn disrupts learning, and that “students do better when given room to struggle with diffi culty.” 17 These studies point out that not all diffi culties in learning are desirable ones. Anxiety while taking a test seems to repre- sent an undesirable diffi culty. These studies also underscore the importance of learners understanding that diffi culty in learning new things is not only to be expected but can be ben- efi cial. To this point, the French study stands on the shoulders of many others, among the foremost being the works of Carol Dweck and of Anders Ericsson, both of whom we discuss in Chapter 7 in relation to the topic of increasing intellectual abilities. Dweck’s work shows that people who believe that their intellectual ability is fi xed from birth, wired in their genes, tend to avoid challenges at which they may not succeed, because failure would appear to be an indication of lesser na- tive ability. By contrast, people who are helped to understand that effort and learning change the brain, and that their intel- lectual abilities lie to a large degree within their own control, are more likely to tackle diffi cult challenges and persist at them. They view failure as a sign of effort and as a turn in the road rather than as a mea sure of inability and the end of the road. Anders Ericsson’s work investigating the nature of expert per for mance shows that to achieve expertise requires thou- Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 93 sands of hours of dedicated practice in which one strives to surpass one’s current level of ability, a pro cess in which failure becomes an essential experience on the path to mastery.The study of the French sixth graders received wide public- ity and inspired the staging of a “Festival of Errors” by an elite graduate school in Paris, aimed at teaching French schoolchil- dren that making mistakes is a constructive part of learning:

not a sign of failure but of effort. Festival organizers argued that modern society’s focus on showing results has led to a culture of intellectual timorousness, starving the kind of intel- lectual ferment and risk-taking that produced the great dis- coveries that mark French history. It doesn’t require a great conceptual leap to get from Par- is’s “Festival of Errors” to San Francisco’s “FailCon,” where technology entrepreneurs and venture capitalists meet once a year to study failures that gave them critical insights they needed in order to pivot in their business strategies so as to succeed. Thomas Edison called failure the source of inspira- tion, and is said to have remarked, “I’ve not failed. I’ve just found 10,000 ways that don’t work.” He argued that perse- verance in the face of failure is the key to success. Failure underlies the scientifi c method, which has advanced our understanding of the world we inhabit. The qualities of per sis tence and resiliency, where failure is seen as useful infor- mation, underlie successful innovation in every sphere and lie at the core of nearly all successful learning. Failure points to the need for redoubled effort, or liberates us to try different approaches. Steve Jobs, in his remarks to the Stanford Univer- sity graduating class of 2005, spoke of being fi red at age thirty in 1985 from Apple Computer, which he had cofounded. “I didn’t see it then, but it turned out that getting fi red from Ap- ple was the best thing that could have ever happened to me.

The heaviness of being successful was replaced by the lightness Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 94 of being a beginner again, less sure about everything. It freed me to enter one of the most creative periods of my life.”It’s not the failure that’s desirable, it’s the dauntless effort despite the risks, the discovery of what works and what doesn’t that sometimes only failure can reveal. It’s trusting that trying to solve a puzzle serves us better than being spoon- fed the so- lution, even if we fall short in our fi rst attempts at an answer. An Example of Generative Learning As we said earlier, the pro cess of trying to solve a problem without the benefi t of having been taught how is called gen- erative learning, meaning that the learner is generating the answer rather than recalling it. Generation is another name for old- fashioned trial and error. We’re all familiar with the stories of skinny kids in Silicon Valley garages messing around with computers and coming out billionaires. We would like to serve up a different kind of example here: Minnesota’s Bonnie Blodgett.

Bonnie is a writer and a self- taught ornamental gardener in a constant argument with a voice in her head that keeps nat- tering about all the ways her latest whim is sure to go haywire and embarrass her. While she is a woman of strong aesthetic sensibilities, she is also one of epic doubts. Her “learning style” might be called leap- before- you- look- because- if- you- look- fi rst- you- probably- won’t-like- what- you- see. Her garden writing appears under the name “The Blundering Gardener.” This moniker is a way of telling her voices of doubt to take a hike, because what ever the consequences of the next whim, she’s already rolling up her sleeves. “Blundering means that you get going on your project before you have fi gured out how to do it in the proper way, before you know what you’re getting into. For me, the risk of knowing what you’re getting Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 95 into is that it becomes an overwhelming obstacle to getting started.” 18 Bonnie’s success shows how struggling with a problem makes for strong learning, and how a sustained commitment to advancing in a par tic u lar fi eld of endeavor through trial- and- error effort leads to complex mastery and greater knowl- edge of the interrelationships of things. When we spoke, she had just traveled to southern Minnesota to meet with a group of farmers who wanted her gardening insights on a gamut of issues ranging from layout and design to pest control and ir- rigation. In the years since she fi rst sank her spade, Bonnie’s garden writing has won national recognition and found a de- voted following far and wide through many outlets, and her garden has become a destination for other gardeners. She came to ornamental gardening about the time she found herself eyeballing middle age. She had no training, just a burning desire to get her hands dirty making beautiful spaces on the corner lot of the home she shares with her husband in a historic neighborhood of St. Paul. “The experience of creating beauty calms me down,” she says, but it’s strictly a discovery pro cess. She has always been a writer, and some years after having launched herself into the garden, she began publishing the Garden Letter, a quarterly for northern gardeners in which she chronicles her exploits, mishaps, lessons, and successes. She writes the same way that she gardens, with boldness and self- effacing humor, passing along the entertaining snafus and unexpected insights that are the fruits of experience. In calling herself the Blundering Gar- dener, she is giving herself and us, her readers, permission to make mistakes and get on with it. Note that in writing about her experiences, Bonnie is en- gaging two potent learning pro cesses beyond the act of gar- dening itself. She is retrieving the details and the story of what Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 96 she has discovered— say, about an experiment in grafting two species of fruit trees— and then she is elaborating by explain- ing the experience to her readers, connecting the outcome to what she already knows about the subject or has learned as a result.Her leap- taking impulses have taken her through vast swaths of the plant kingdom, of course, and deeply into the Latin nomenclature and the classic horticultural literature.

These impulses have also drawn her into the aesthetics of space and structure and the mechanics thereof: building stone walls; digging and wiring water features; putting a cupola on the garage; building paths, stairs, and gates; ripping out a Gothic picket fence and reusing the wood to create something more open and with stronger horizontal lines to pull down the soaring verticality of her three- story Victorian house and connect it with the gardens that surround it; making the out- door spaces airier and more easily seen from the street, while still circumscribed, so as to impart that essential sense of pri- vacy that makes a garden a room of its own. Her spaces are idiosyncratic and asymmetrical, giving the illusion of having evolved naturally, yet they cohere, through the repetition of textures, lines, and geometry. A simple example of how she has backed into more and more complex mastery is the manner in which she came to embrace plant classifi cation and the Latin terminology . “When I started, the world of plants was a completely foreign lan- guage to me. I would read gardening books and be completely lost. I didn’t know what plant names were, common or Latin.

I wasn’t thinking about learning this stuff, ever. I’m like, Why would you want to do that? Why wouldn’t you just get out- side and dig a hole and put something in it?” What she rel- ished were pictures that gave her ideas and passages of text where the designers used phrases like “my pro cess” in describing Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 97 how they had achieved the desired effect. It was the posses- sive pronoun, my pro cess, that affi rmed Bonnie in her head- long rush to learn by doing. The notion is that every gardener’s pro cess is uniquely his or her own. Bonnie’s pro cess did not involve taking direction from experts, much less mastering the Linnaean taxonomy or the Latin names of what she stuck in holes and dragged her water hose to. But as she thrashed around, working to achieve in dirt the magical spaces that danced in her mind, she came to Latin and Linnaeus despite herself. “You begin to discover that the Latin names are helpful.

They can give you a shortcut to understanding the nature of the plants, and they can help you remember. Tardiva , which is a species name, comes after hydrangea, which is a genus.” Bonnie had taken Latin in high school, along with French, and of course En glish, and the cues to those memories began to reawaken. “I can easily see that tardiva means late, like tardy. The same word comes after many plant varieties, so you see the genus and then the species is tardiva, and now you know that par tic u lar plant is a late bloomer. So you be- gin to realize that the Latin names are a way of helping you remember, and you fi nd yourself using them more and more.

Also you remember plants better, because it’s second nature to you that procumbus means prostrate, crawling on the ground. It makes sense. So now it’s not so hard to remember that par tic u lar species name when it’s attached to a genus.

It’s also important to know the Latin names because then you can be absolutely specifi c about a plant. Plants have common names, and common names are regional. Actaea racemosa has a common name of black cohosh, but it’s also known as snakeroot, and those names are often given to other plants. There’s only one Actaea racemosa.” Gradually, and despite her inclination to resist, she came to grasp the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 98 classical taxonomy of ornamental plants and to appreciate how Linnaeus’s schema frames family connections and com- municates attributes.Bonnie said that the farmers she had recently met were particularly interested in what she has learned about the ad- vantages of composting and earthworms over chemical fertil- izers for building nutrients and soil aeration, and how to get strong root growth on low rations of water through a home- made system of drip irrigation. She paused in recounting her meeting with them, refl ecting on how all of this knowledge has sneaked up on her. It was never something she set out to conquer. “Look, blundering’s really not a bad thing. It’s a good thing in that you get stuff done. A lot of people, when they contemplate the enormity of the task and they see all that’s entailed, they’re stopped in their tracks.” Of course, in some settings— like learning to jump out of airplanes and walk away with your life— blundering is not the optimal learning strategy. Undesirable Difﬁ culties Elizabeth and Robert Bjork, who coined the phrase “desirable diffi culties,” write that diffi culties are desirable because “they trigger encoding and retrieval pro cesses that support learning, comprehension, and remembering. If, however, the learner does not have the background knowledge or skills to respond to them successfully, they become undesirable diffi culties.” 19 Cognitive scientists know from empirical studies that testing, spacing, interleaving, variation, generation, and certain kinds of contextual interference lead to stronger learning and reten- tion. Beyond that, we have an intuitive sense of what kinds of diffi culties are undesirable but, for lack of the needed research, we cannot yet be defi nitive.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 99 Clearly, impediments that you cannot overcome are not desirable. Outlining a lesson in a sequence different from the one in the textbook is not a desirable diffi culty for learners who lack the reading skills or language fl uency required to hold a train of thought long enough to reconcile the discrep- ancy. If your textbook is written in Lithuanian and you don’t know the language, this hardly represents a desirable diffi - culty. To be desirable, a diffi culty must be something learners can overcome through increased effort. Intuitively it makes sense that diffi culties that don’t strengthen the skills you will need, or the kinds of challenges you are likely to encounter in the real- world application of your learning, are not desirable. Having somebody whisper in your ear while you read the news may be essential training for a TV anchor. Being heckled by role- playing protestors while honing your campaign speech may help train up a poli- tician. But neither of these diffi culties is likely to be helpful for Rotary Club presidents or aspiring YouTube bloggers who want to improve their stage presence. A cub towboat pi lot on the Mississippi might be required in training to push a string of high- riding empty barges into a lock against a strong side wind. A baseball player might practice hitting with a weight on his bat to strengthen his swing. You might teach a football player some of the principles of ballet for learning balance and movement, but you probably would not teach him the tech- niques for an effective golf drive or backhand tennis serve. Is there an overarching rule that determines the kinds of impediments that make learning stronger? Time and further research may yield an answer. But the kinds of diffi culties we’ve just described, whose desirability is well documented, offer a large and diverse toolkit already at hand.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 100 The Takeaway Learning is at least a three- step pro cess: initial encoding of information is held in short- term working memory before be- ing consolidated into a cohesive repre sen ta tion of knowledge in long- term memory. Consolidation reorganizes and stabi- lizes memory traces, gives them meaning, and makes con- nections to past experiences and to other knowledge already stored in long- term memory. Retrieval updates learning and enables you to apply it when you need it. Learning always builds on a store of prior knowledge. We interpret and remember events by building connections to what we already know. Long- term memory capacity is virtually limitless: the more you know, the more possible connections you have for adding new knowledge. Because of the vast capacity of long- term memory, having the ability to locate and recall what you know when you need it is key; your facility for calling up what you know depends on the repeated use of the information (to keep retrieval routes strong) and on your establishing powerful retrieval cues that can reactivate the memories. Periodic retrieval of learning helps strengthen connections to the memory and the cues for recalling it, while also weak- ening routes to competing memories. Retrieval practice that’s easy does little to strengthen learning; the more diffi cult the practice, the greater the benefi t. When you recall learning from short- term memory, as in rapid- fi re practice, little mental effort is required, and little long- term benefi t accrues. But when you recall it after some time has elapsed and your grasp of it has become a little rusty, you have to make an effort to reconstruct it. This effortful retrieval both strengthens the memory but also makes the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Embrace Difficulties ê 101 learning pliable again, leading to its reconsolidation. Recon- solidation helps update your memories with new information and connect them to more recent learning. Repeated effortful recall or practice helps integrate learn- ing into mental models, in which a set of interrelated ideas or a sequence of motor skills are fused into a meaningful whole that can be adapted and applied in later settings. Examples are the perceptions and manipulations involved in driving a car or in knocking a curveball out of the ballpark. When practice conditions are varied or retrieval is inter- leaved with the practice of other material, we increase our abilities of discrimination and induction and the versatility with which we can apply the learning in new settings at a later date. Interleaving and variation build new connections, expanding and more fi rmly entrenching knowledge in mem- ory and increasing the number of cues for retrieval. Trying to come up with an answer rather than having it presented to you, or trying to solve a problem before being shown the solution, leads to better learning and longer reten- tion of the correct answer or solution, even when your at- tempted response is wrong, so long as corrective feedback is provided.

Created from liberty on 2021-11-21 07:19:56.

102 A\b \bhe roo\b of our effectiveness is our ability to grasp the world around us and to take the mea sure of our own per for mance. We’re constantly making judgments about what we know and don’t know and whether we’re ca- pable of handling a task or solving a problem. As we work at something, we keep an eye on ourselves, adjusting our think- ing or actions as we progress. Monitoring your own thinking is what psychologists call metacognition (meta is Greek for “about”). Learning to be accurate self- observers helps us to stay out of blind alleys, make good decisions, and refl ect on how we might do better next time. An important part of this skill is being sensitive to the ways we can delude ourselves. One problem with poor judgment is that we usually don’t know when we’ve got it.

Another problem is the sheer scope of the ways our judgment can be led astray.

1 5 Avoid Illusions of Knowing Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 103 In this chapter we discuss perceptual illusions, cognitive bi- ases, and distortions of memory that commonly mislead peo- ple. Then we suggest techniques for keeping your judgment squared with reality. The consequences of poor judgment fi ll the daily papers.

During the summer of 2008, three stickup artists in Minne- apolis had a system going of phoning in large fast- food orders and then relieving the delivery man of all the goods and cash he carried. As a livelihood it was a model of simplicity. They kept at it, failing to consider the wisdom of always placing their orders from the same two cell phones and taking deliv- ery at the same two addresses. David Garman, a Minneapolis cop, was working under- cover that summer. “It was getting more aggressive. At the beginning, it was ‘maybe they had a gun,’ then all of a sudden there were a couple of guns, and then they were hurting the people when they were robbing them.” It was a night in August when Garman got a call about a large order phoned in to a Chinese restaurant. He or ga nized a small team on short notice and prepared to pose as the deliv- ery guy. He pulled on a bulletproof vest, covered it with a ca- sual shirt, and shoved his .45 automatic into his pants. While his colleagues staked out positions near the delivery address, Garman picked up the food, drove there, and parked with his brights shining on the front door. He’d cut a slit in the bottom of the food bag and tucked a .38 inside to rest in his hand as he carried the package. “The .38 has a covered hammer on it, so I can shoot it in a bag. If I were to put the automatic in there, it’d jam and I’d be screwed.” So I walk up with the package and I say, “Hey, sir, did you order some food?” He says, “Yup,” and I’m thinking this guy’s really just going to pay me and I’m going to be out of here, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 104 and this is going to be the dumbest thing we’ve ever done. I’m thinking if he hands me $40, I don’t even know how much this food is. But he turns his head to look halfway back and two other guys start to come up, and as they’re walking to- wards me they fl ip hoods over their heads. That’s when I know it’s game time. The fi rst guy whips a gun out of his pocket and racks it and puts it to my head all in one motion, saying, “Give me everything you’ve got motherfucker or I’ll kill you.” I ended up shooting him through the bag. It was four rounds. 2 Not such a great livelihood after all. The guy was hit low and survived, although he is a lesser man as a result. Garman would have aimed higher if the food package hadn’t been so heavy, and he took a lesson from the experience: he’s better prepared for the next time, though he’d rather we didn’t de- scribe just how. We like to think we’re smarter than the average doodle, and even if we’re not, we feel affi rmed in this delusion each year when the newest crop of Darwin Awards circulates by email, that short list of self- infl icted fatalities caused by spec- tacularly poor judgment, as in the case of the attorney in To- ronto who was demonstrating the strength of the windows in his twenty- two- story offi ce tower by throwing his shoulder against the glass when he broke it and fell through. The truth is that we’re all hardwired to make errors in judgment. Good judgment is a skill one must acquire, becoming an astute observer of one’s own thinking and per for mance. We start at a disadvantage for several reasons. One is that when we’re incompetent, we tend to overestimate our competence and see little reason to change. Another is that, as humans, we are read- ily misled by illusions, cognitive biases, and the stories we con- struct to explain the world around us and our place within Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 105 it. To become more competent, or even expert, we must learn to recognize competence when we see it in others, become more accurate judges of what we ourselves know and don’t know, adopt learning strategies that get results, and fi nd ob- jective ways to track our progress. Two Systems of Knowing In his book Thinking, Fast and Slow, Daniel Kahneman de- scribes our two analytic systems. What he calls System 1 (or the automatic system) is unconscious, intuitive, and immedi- ate. It draws on our senses and memories to size up a situation in the blink of an eye. It’ s the running back dodging tackles in his dash for the end zone. It’s the Minneapolis cop, walking up to a driver he’s pulled over on a chilly day, taking evasive action even before he’s fully aware that his eye has seen a bead of sweat run down the driver’s temple. System 2 (the controlled system) is our slower pro cess of conscious analysis and reasoning. It’s the part of thinking that considers choices, makes decisions, and exerts self- control.

We also use it to train System 1 to recognize and respond to par tic u lar situations that demand refl exive action. The run- ning back is using System 2 when he walks through the moves in his playbook. The cop is using it when he practices taking a gun from a shooter. The neurosurgeon is using it when he re- hearses his repair of the torn sinus. System 1 is automatic and deeply infl uential, but it is sus- ceptible to illusion, and you depend on System 2 to help you manage yourself: by checking your impulses, planning ahead, identifying choices, thinking through their implications, and staying in charge of your actions. When a guy in a restaurant walks past a mother with an infant and the infant cries out “Dada!” that’s System 1. When the blushing mother says, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 106 “No, dear, that’s not Dada, that’s a man,” she is acting as a surrogate System 2, helping the infant refi ne her System 1.System 1 is powerful because it draws on our accumulated years of experience and our deep emotions. System 1 gives us the survival refl ex in moments of danger, and the astonishing deftness earned through thousands of hours of deliberate practice in a chosen fi eld of expertise. In the interplay be- tween Systems 1 and 2— the topic of Malcolm Gladwell’s book Blink—your instantaneous ability to size up a situation plays against your capacity for skepticism and thoughtful analysis. Of course, when System 1’s conclusions arise out of misperception or illusion, they can steer you into trouble.

Learning when to trust your intuition and when to question it is a big part of how you improve your competence in the world at large and in any fi eld where you want to be expert.

It’s not just the dullards who fall victim. We all do, to varying degrees. Pi lots, for example, are susceptible to a host of per- ceptual illusions. They are trained to beware of them and to use their instruments to know that they’re getting things right.

A frightening example with a happy ending is China Airlines Flight 006 on a winter day in 1985. The Boeing 747 was 41,000 feet above the Pacifi c, almost ten hours into its eleven- hour fl ight from Taipei to LA, when engine number 4 lost power. The plane began to lose airspeed. Rather than taking manual control and descending below 30,000 feet to restart the engine, as prescribed in the fl ight book, the crew held at 41,000 with the autopi lot engaged and attempted a restart.

Meanwhile, loss of the outboard engine gave the plane asym- metrical thrust. The autopi lot tried to correct for this and keep the plane level, but as the plane continued to slow it also began to roll to the right. The captain was aware of the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 107 deceleration, but not the extent to which the plane had en- tered a right bank; his System 1 clue would have been his vestibular refl ex— how the inner ear senses balance and spatial orientation—but because of the plane’s trajectory, he had the sensation of fl ying level. His System 2 clues would have been a glimpse at the horizon and his instruments. Cor- rect procedure called for applying left rudder to help raise the right wing, but his System 2 focus was on the airspeed indica- tor and on the efforts of the fi rst offi cer and engineer to re- start the engine. As its bank increased, the plane descended through 37,000 feet into high clouds, which obscured the horizon. The cap- tain switched off the autopi lot and pushed the nose down to get more speed, but the plane had already rolled beyond 45 degrees and now turned upside down and fell into an uncon- trolled descent. The crew were confused by the situation. They understood the plane was behaving erratically but were un- aware they had overturned and were in a dive. They could no longer discern thrust from engines 1– 3 and concluded those engines had quit as well. The plane’s dive was evident from their fl ight gauges, but the angle was so unlikely the crew de- cided the gauges had failed. At 11,000 feet they broke through the clouds, astonished to see that they were roaring toward earth. The captain and fi rst offi cer both pulled back hard on the stick, exerting enormous forces on the plane but manag- ing to level off. Landing gear hung from the plane’s belly, and they’d lost one of their hydraulic systems, but all four engines came to life, and the captain was able to fl y on, diverting suc- cessfully to San Francisco. An inspection revealed just how severe their maneuver had been. Strains fi ve times the force of gravity had bent the plane’s wings permanently upward, bro- ken two landing gear struts, and torn away two landing gear doors and large parts of the rear horizontal stabilizers.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 108 “Spatial disorientation” is the aeronautical term for a deadly combination of two elements: losing sight of the hori- zon and relying on human sensory perception that doesn’t jibe with reality but is so convincing that pi lots conclude their cockpit instruments have failed. As Kahneman says, System 1, the instinctual, refl exive system that detects danger and keeps us safe, can be very hard to overrule. Flight 006’s initial inci- dent, the loss of an engine cruising at altitude, is not consid- ered an emergency, but it quickly became one as a result of the captain’s actions. Rather than following prescribed proce- dure, and rather than fully engaging his System 2 analytic re- sources by monitoring all his instruments, he let himself be- come preoccupied with the engine restart and with a single fl ight indicator, airspeed. Then, when things spiraled out of control, he trusted his senses over his gauges, in effect trying to construct his own narrative of what was happening to the plane. There’s a long list of illusions to which pi lots can fall prey (some with mordant names like “the leans,” “graveyard spin,” and “the black hole approach”) and sites on the Internet where you can listen to the chilling last words of pi lots struggling and failing to understand and correct what’s gone wrong in the sky. Spatial disorientation was deemed the probable cause of the crash that killed Mel Carnahan, the governor of Mis- souri, while being fl own through a thunderstorm one night in October 2000, and the probable cause of the crash that killed John F. Kennedy Jr. and his wife and her sister off the shore of Martha’s Vineyard on a hazy night in July 1999. Fortunately, the China Airlines incident came to a good end, but the Na- tional Transportation Safety Board report of that incident re- veals just how quickly training and professionalism can be hijacked by System 1 illusion, and therefore why we need to Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 109 cultivate a disciplined System 2, conscious analysis and rea- soning, that always keeps one eye on the fl ight instruments. 3 Illusions and Memory Distortions The fi lmmaker Errol Morris, in a series of articles on illusion in the New York Times, quotes the social psychologist David Dunning on humans’ penchant for “motivated reasoning,” or, as Dunning put it, the “sheer genius people have at convinc- ing themselves of congenial conclusions while denying the truth of incon ve nient ones.” 4 (The British prime minister Ben- jamin Disraeli once said of a po liti cal opponent that his con- science was not his guide but his accomplice.) There are many ways that our System 1 and System 2 judgments can be led astray: perceptual illusions like those experienced by pi lots, faulty narrative, distortions of memory, failure to recognize when a new kind of problem requires a new kind of solution, and a variety of cognitive biases to which we’re prone. We describe a number of these hazards here, and then we offer mea sures you can take, akin to scanning the cockpit instru- ments, to help keep your thinking aligned with reality.

Our understanding of the world is shaped by a hunger for narrative that rises out of our discomfort with ambiguity and arbitrary events. When surprising things happen, we search for an explanation. The urge to resolve ambiguity can be sur- prisingly potent, even when the subject is inconsequential. In a study where participants thought they were being mea sured for reading comprehension and their ability to solve anagrams, they were exposed to the distraction of a background phone conversation. Some heard only one side of a conversation, Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 110 and others heard both sides. The participants, not knowing that the distraction itself was the subject of the study, tried to ignore what they were hearing so as to stay focused on the reading and anagram solutions. The results showed that over- hearing one side of a conversation proved more distracting than overhearing both sides, and the content of those partial conversations was better recalled later by the unintentional eavesdroppers. Why was this? Presumably, those overhearing half a conversation were strongly compelled to try to infer the missing half in a way that made for a complete narrative. As the authors point out, the study may help explain why we fi nd one- sided cell phone conversations in public spaces so intru- sive, but it also reveals the ineluctable way we are drawn to imbue the events around us with rational explanations. The discomfort with ambiguity and arbitrariness is equally powerful, or more so, in our need for a rational understand- ing of our own lives. We strive to fi t the events of our lives into a cohesive story that accounts for our circumstances, the things that befall us, and the choices we make. Each of us has a different narrative that has many threads woven into it from our shared culture and experience of being human, as well as many distinct threads that explain the singular events of one’s personal past. All these experiences infl uence what comes to mind in a current situation and the narrative through which you make sense of it: Why nobody in my family at- tended college until me. Why my father never made a fortune in business. Why I’d never want to work in a corporation, or, maybe, Why I would never want to work for myself. We gravitate to the narratives that best explain our emotions. In this way, narrative and memory become one. The memories we or ga nize meaningfully become those that are better re- membered. Narrative provides not only meaning but also a Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 111 mental framework for imbuing future experiences and infor- mation with meaning, in effect shaping new memories to fi t our established constructs of the world and ourselves. No reader, when asked to account for the choices made under pressure by a novel’s protagonist, can keep her own life experience from shading her explanation of what must have been going on in the character’s interior world. The success of a magician or politician, like that of a novelist, relies on the seductive powers of narrative and on the audience’s willing suspension of disbe- lief. Nowhere is this more evident than in the national po liti cal debate, where like- minded people gather online, at community meetings, and in the media to fi nd common purpose and ex- pand the story they feel best explains their sense of how the world works and how humans and politicians should behave. You can see how quickly personal narrative is invoked to explain emotions when you read an article online whose au- thor has argued a position on almost any subject— for exam- ple, an op- ed piece supporting the use of testing as a powerful tool for learning. Scan the comments posted by readers: some sing hallelujah while others can scarcely contain their um- brage, each invoking a personal story that supports or refutes the column’s main argument. The psychologists Larry Jacoby, Bob Bjork, and Colleen Kelley, summing up studies on illu- sions of comprehension, competence, and remembering, write that it is nearly impossible to avoid basing one’s judgments on subjective experience. Humans do not give greater credence to an objective record of a past event than to their subjective remembering of it, and we are surprisingly insensitive to the ways our par tic u lar construals of a situation are unique to ourselves. Thus the narrative of memory becomes central to our intuitions regarding the judgments we make and the ac- tions we take. 5 Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 112 It is a confounding paradox, then, that the changeable na- ture of our memory not only can skew our perceptions but also is essential to our ability to learn. As will be familiar to you by now, every time we call up a memory, we make the mind’s routes to that memory stronger, and this capacity to strengthen, expand, and modify memory is central to how we deepen our learning and broaden the connections to what we know and what we can do. Memory has some similarities to a Google search algorithm, in the sense that the more you connect what you learn to what you already know, and the more associations you make to a memory (for example, link- ing it with a visual image, a place, or a larger story), then the more mental cues you have through which to fi nd and retrieve the memory again later. This capacity expands our agency:

our ability to take action and be effective in the world. At the same time, because memory is a shape- shifter, reconciling the competing demands of emotion, suggestions, and narrative, it serves you well to stay open to the fallibility of your certain- ties: even your most cherished memories may not represent events in the exact way they occurred. Memory can be distorted in many ways. People interpret a story in light of their world knowledge, imposing order where none had been present so as to make a more logical story.

Memory is a reconstruction. We cannot remember every as- pect of an event, so we remember those elements that have greatest emotional signifi cance for us, and we fi ll in the gaps with details of our own that are consistent with our narrative but may be wrong. People remember things that were implied but not specifi - cally stated. The literature is full of examples. In one, many people who read a paragraph about a troubled girl named Helen Keller later mistakenly recalled the phrase “deaf, dumb, and blind” as being in the text. This mistake was rarely made Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 113 by another group who read the same paragraph about a girl named Carol Harris. 6 Imagination infl ation refers to the tendency of people who, when asked to imagine an event vividly, will sometimes begin to believe, when asked about it later, that the event actually occurred. Adults who were asked “Did you ever break a win- dow with your hand?” were more likely on a later life inven- tory to report that they believed this event occurred during their lifetimes. It seems that asking the question led them to imagine the event, and the act of having imagined it had the effect, later, of making them more likely to think it had oc- curred (relative to another group who answered the question without having previously imagined it occurring). Hypothetical events that are imagined vividly can seat them- selves in the mind as fi rmly as memories of actual events. For instance, when it is suspected that a child is being sexually abused and he is interviewed and questioned about it, he may imagine experiences that the interviewer describes and then later come to “remember” them as having occurred. 7 (Sadly, of course, many memories of childhood sexual abuse are ab- solutely true, usually ones reported soon after the occurrence.) Another type of memory illusion is one caused by suggestion, which may arise simply in the way a question is asked. In one example, people watched a video of a car running a stop sign at an intersection and colliding with another car passing through. Those who were later asked to judge the speed of the vehicles when they “contacted” each other gave an average estimate of thirty- two miles per hour. Those who were asked to judge the speed when the two vehicles “smashed” into each Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 114 other estimated on average forty- one miles per hour. If the speed limit was thirty miles per hour, asking the question the second way rather than the fi rst could lead to the driver’s be- ing charged with speeding. Of course, the legal system knows the danger of witnesses being asked “leading questions” (ones that encourage a par tic u lar answer), but such questions are diffi cult to avoid completely, because suggestibility can be very subtle. After all, in the case just discussed, the two cars did “smash together.” 8 Some witnesses to crimes who are struggling to recall them are instructed to let their minds roam freely, to generate what- ever comes to mind, even if it is a guess. However, the act of guessing about possible events causes people to provide their own misinformation, which, if left uncorrected, they may later come to retrieve as memories. That is one reason why people who have been interviewed after being hypnotized are barred from testifying in court in almost all states and Cana- dian provinces. The hypnotic interview typically encourages people to let their thoughts roam freely and produce every- thing that comes to mind, in hopes that they will retrieve in- formation that would not otherwise be produced. However, this pro cess causes them to produce much erroneous informa- tion, and studies have shown that when they are tested later, under instructions only to tell exactly what they remember of the actual events, their guesses made while under hypnosis cloud their memories about what truly happened. In par tic u- lar, they remember events they produced under hypnosis as actual experiences, even under conditions (in the laboratory) when it is known that the events in question did not occur. 9 Interference from other events can distort memory. Suppose the police interview a witness shortly after a crime, showing Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 115 pictures of possible suspects. Time passes, but eventually the police nab a suspect, one whose picture had been viewed by the witness. If the witness is now asked to view a lineup, he may mistakenly remember one of the suspects whose photo he saw as having been present at the crime. A particularly vivid example of a related pro cess happened to the Australian psychologist Donald M. Thomson. A woman in Sydney was watching tele vi sion in midday when she heard a knock at the door. When she answered it, she was attacked, raped, and left unconscious. When she awoke and dialed the police, they came to her aid, got a description of her assailant, and launched a search. They spotted Donald Thomson walking down a Sydney street, and he matched the description. They arrested him on the spot. It turns out that Thomson had an airtight alibi— at the exact time of the rape, he was being interviewed on a live tele vi sion show. The police did not believe him and sneered when he was being interrogated. However, the story was true. The woman had been watching the show when she heard the knock on the door. The description she gave the police was apparently of the man she saw on tele vi sion, Don- ald Thomson, rather than the rapist. Her System 1 reaction— quick but sometimes mistaken— provided the wrong descrip- tion, probably due to her extreme emotional state. 10 What psychologists call the curse of knowledge is our ten- dency to underestimate how long it will take another person to learn something new or perform a task that we have al- ready mastered. Teachers often suffer this illusion— the calcu- lus instructor who fi nds calculus so easy that she can no lon- ger place herself in the shoes of the student who is just starting out and struggling with the subject. The curse-of-knowledge effect is close kin to hindsight bias, or what is often called the Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 116 knew- it- all- along effect, in which we view events after the fact as having been more predictable than they were before they occurred. Stock market pundits will confi dently announce on the eve ning news why the stock market behaved as it did that day, even though they could not have predicted the move- ments that morning. 11 Accounts that sound familiar can create the feeling of know- ing and be mistaken for true. This is one reason that po liti cal or advertising claims that are not factual but are repeated can gain traction with the public, particularly if they have emo- tional resonance. Something you once heard that you hear again later carries a warmth of familiarity that can be mis- taken for memory, a shred of something you once knew and cannot quite place but are inclined to believe. In the world of propaganda, this is called “the big lie” technique— even a big lie told repeatedly can come to be accepted as truth.

Fluency illusions result from our tendency to mistake fl uency with a text for mastery of its content. For example, if you read a particularly lucid pre sen ta tion of a diffi cult concept, you can get the idea that it is actually pretty simple and perhaps even that you knew it all along. As discussed earlier, students who study by rereading their texts can mistake their fl uency with a text, gained from rereading, for possession of accessible knowl- edge of the subject and consequently overestimate how well they will do on a test.

Our memories are also subject to social infl uence and tend to align with the memories of the people around us. If you are in Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 117 a group reminiscing about past experiences and someone adds a wrong detail about the story, you will tend to incorpo- rate this detail into your own memory and later remember the experience with the erroneous detail. This pro cess is called “memory conformity” or the “social contagion of memory”:

one person’s error can “infect” another person’s memory. Of course, social infl uences are not always bad. If someone recalls details of joint memory on which you are somewhat hazy, your subsequent memory will be updated and will hold a more accurate record of the past event. 12 In the obverse of the social infl uence effect, humans are pre- disposed to assume that others share their beliefs, a pro cess called the false consensus effect. We generally fail to recognize the idiosyncratic nature of our personal understanding of the world and interpretation of events and that ours differ from others’. Recall how surprised you were recently, on commiser- ating with a friend about the general state of affairs, to discover that she sees in an entirely different light matters on which you thought the correct view was fundamental and obvious: cli- mate change, gun control, fracking of gas wells— or perhaps something very local, such as whether to pass a bond issue for a school building or to oppose construction of a big box store in the neighborhood. 13 Confi dence in a memory is not a reliable indication of its ac- curacy. We can have utmost faith in a vivid, nearly literal memory of an event and yet fi nd that we actually have it all wrong. National tragedies, like the assassination of President John Kennedy or the events surrounding 9/11, create what psychologists call “fl ashbulb” memories, named for the vivid Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 118 images that we retain: where we were when we got the news, how we learned it, how we felt, what we did. These memories are thought to be indelible, burned into our minds, and it is true that the broad outlines of such catastrophes, thoroughly reported in the media, are well remembered, but your mem- ory of your personal circumstances surrounding the events may not necessarily be accurate. There have been numerous studies of this phenomenon, including surveys of fi fteen hun- dred Americans’ memories of the September 11 attacks. In this study, the respondents’ memories were surveyed a week after the attacks, again a year later, and then again three years and ten years later. Respondents’ most emotional memories of their personal details at the time they learned of the attacks are also those of which they are most confi dent and, paradoxi- cally, the ones that have most changed over the years relative to other memories about 9/11. 14 Mental Models As we develop mastery in the various areas of our lives, we tend to bundle together the incremental steps that are required to solve different kinds of problems. To use an analogy from a previous chapter, you could think of them as something like smart- phone apps in the brain. We call them mental mod- els. Two examples in police work are the choreography of the routine traffi c stop and the moves to take a weapon from an assailant at close quarters. Each of these maneuvers involves a set of perceptions and actions that cops can adapt with little conscious thought in response to context and situation. For a barista, a mental model would be the steps and ingredients to produce a perfect sixteen- ounce decaf frappuccino. For the receptionist at urgent care, it’s triage and registration.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 119 The better you know something, the more diffi cult it be- comes to teach it. So says physicist and educator Eric Mazur of Harvard. Why? As you get more expert in complex areas, your models in those areas grow more complex, and the com- ponent steps that compose them fade into the background of memory (the curse of knowledge). A physicist, for example, will create a mental library of the principles of physics she can use to solve the various kinds of problems she encounters in her work: Newton’s laws of motion, for example, or the laws of conservation of momentum. She will tend to sort problems based on their underlying principles, whereas a novice will group them by similarity of surface features, like the appara- tus being manipulated in the problem (pulley, inclined plane, etc.). One day, when she goes to teach an intro physics class, she explains how a par tic u lar problem calls for something from Newtonian mechanics, forgetting that her students have yet to master the underlying steps she has long ago bundled into one unifi ed mental model. This presumption by the pro- fessor that her students will readily follow something com- plex that appears fundamental in her own mind is a metacog- nitive error, a misjudgment of the matchup between what she knows and what her students know. Mazur says that the person who knows best what a student is struggling with in assimilating new concepts is not the professor, it’s another student. 15 This problem is illustrated through a very simple experiment in which one person plays a common tune inside her head and taps the rhythm with her knuckles and another person hearing the rhythmic taps must guess the tune. Each tune comes from a fi xed set of twenty- fi ve, so the statistical chance of guessing it is 4 percent. Tellingly, the participants who have the tune in mind estimate that the other person will guess correctly 50 percent of the time, but in fact the listeners Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 120 guess correctly only 2.5 percent of the time, no better than chance. 16 Like Coach Dooley’s football players memorizing their play- books, we all build mental libraries of myriad useful solutions that we can call on at will to help us work our way from one Saturday game to the next. But we can be tripped by these models, too, when we fail to recognize a new problem that appears to be a familiar one is actually something quite differ- ent and we pull out a solution to address it that doesn’t work or makes things worse. The failure to recognize when your solution doesn’t fi t the problem is another form of faulty self- observation that can lead you into trouble.Mike Ebersold, the neurosurgeon, was called into the op- erating room one day to help a surgical resident who, in the midst of removing a brain tumor, was losing the patient. The usual model for cutting out a tumor calls for taking your time, working carefully around the growth, getting a clean margin, saving the surrounding nerves. But when the growth is in the brain, and if you get bleeding behind it, pressure on the brain can turn fatal. Instead of slow- and- careful, you need just the opposite, cutting the growth out very quickly so the blood can drain, and then working to repair the bleeding. “Initially you might be a little timid to take the big step,” Mike says. “It’s not pretty, but the patient’s survival depends on your knowing to switch gears and do it fast.” Mike assisted, and the surgery was successful. Like the infant who calls the stranger Dada, we must culti- vate the ability to discern when our mental models aren’t working: when a situation that seems familiar is actually dif- ferent and requires that we reach for a different solution and do something new.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 121 Unskilled and Unaware of It Incompetent people lack the skills to improve because they are unable to distinguish between incompetence and competence.

This phenomenon, of par tic u lar interest for metacognition, has been named the Dunning- Kruger effect after the psycholo- gists David Dunning and Justin Kruger. Their research showed that incompetent people overestimate their own competence and, failing to sense a mismatch between their per for mance and what is desirable, see no need to try to improve. (The title of their initial paper on the topic was “Unskilled and Unaware of It.”) Dunning and Kruger have also shown that incompe- tent people can be taught to raise their competence by learning the skills to judge their own per for mance more accurately, in short, to make their metacognition more accurate. In one se- ries of studies that demonstrate this fi nding, they gave students a test of logic and asked them to rate their own per for mance. In the fi rst experiment the results confi rmed expectations that the least competent students were the most out of touch with their per for mance: students who scored at the twelfth percentile on average believed that their general logical reasoning ability fell at the sixty- eighth percentile. In a second experiment, after taking an initial test and rat- ing their own per for mance, the students were shown the other students’ answers and then their own answers and asked to reestimate the number of test questions they had answered correctly. The students whose per for mance was in the bottom quartile failed to judge their own per for mance more accu- rately after seeing the more competent choices of their peers and in fact tended to raise their already infl ated estimates of their own ability. A third experiment explored whether poor performers could learn to improve their judgment. The students were given ten Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 122 problems in logical reasoning and after the test were asked to rate their logical reasoning skills and test per for mance. Once again, the students in the bottom quartile grossly overesti- mated their per for mance. Next, half the students received ten minutes of training in logic (how to test the accuracy of a syllogism); the other half of the students were given an unre- lated task. All the students were then asked to estimate again how well they had performed on the test. Now the students in the bottom quartile who had received the training were much more accurate estimators of the number of questions they got right and of how they performed compared to the other stu- dents. Those in the bottom quartile who didn’t receive the training held to their mistaken conviction that they had per- formed well. How is it that incompetent people fail to learn through experience that they are unskilled? Dunning and Kruger offer several theories. One is that people seldom receive negative feedback about their skills and abilities from others in every- day life, because people don’t like to deliver the bad news.

Even if people get negative feedback, they must come to an accurate understanding of why the failure occurred. For suc- cess everything must go right, but by contrast, failure can be attributed to any number of external causes: it’s easy to blame the tool for what the hand cannot do. Finally, Dunning and Kruger suggest that some people are just not astute at reading how other people are performing and are therefore less able to spot competence when they see it, making them less able to make comparative judgments of their own per for mance. These effects are more likely to occur in some contexts and with some skills than with others. In some domains, the reve- lation of one’s incompetence can be brutally frank. The au- thors can all remember from their childhoods when a teacher would appoint two boys to pick other kids for softball teams.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 123 The good players are picked fi rst, the worst last. You learn your peers’ judgments of your softball abilities in a very pub- lic manner, so it would be hard for the last- picked player to think “I must be really good at softball.” However, most realms of life do not render such stark judgments of ability. 17 To sum up, the means by which we navigate the world— Daniel Kahneman’s Systems 1 and 2— rely on our perceptual systems, intuition, memory, and cognition, with all their tics, warts, biases, and fl aws. Each of us is an astounding bundle of perceptual and cognitive abilities, coexisting with the seeds of our own undoing. When it comes to learning, what we choose to do is guided by our judgments of what works and what doesn’t, and we are easily misled.Our susceptibility to illusion and misjudgment should give us all pause, and especially so to the advocates of “student- directed learning,” a theory now current among some parents and educators. This theory holds that students know best what they need to study to master a subject, and what pace and methods work best for them. For example, at Manhattan Free School in East Harlem, opened in 2008, students “do not re- ceive grades, take tests or have to do anything they do not feel like doing.” The Brooklyn Free School, which opened in 2004, along with a new crop of homeschooling families who call themselves “unschoolers,” follows the precept that what ever intrigues the learner is what will result in the best learning. 18 The intent is laudatory. We know that students need to take more control of their own learning by employing strategies like those we have discussed. For example, they need to test themselves, both to attain the direct benefi ts of increased re- tention and to determine what they know and don’t know to more accurately judge their progress and focus on material Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 124 that needs more work. But few students practice these strate- gies, and those who do will need more than encouragement if they are to practice them effectively: It turns out that even when students understand that retrieval practice is a superior strategy, they often fail to persist long enough to get the last- ing benefi t. For example, when students are presented with a body of material to master, say a stack of foreign vocabulary fl ashcards, and are free to decide when to drop a card out of the deck because they’ve learned it, most students drop the card when they’ve gotten it right once or twice, far sooner than they should. The paradox is that those students who employ the least effective study strategies overestimate their learning the most and, as a consequence of their misplaced confi dence, they are not inclined to change their habits. The football player preparing for next Saturday’s game doesn’t leave his per for mance to intuition, he runs through his plays and mixes it up to discover the rough edges and work them out on the fi eld well before suiting up for the big game.

If this kind of behavior were anywhere close to the norm for students in their academics today, then self- directed learning would be highly effective. But of course the football player is not self- directed, his practice is guided by a coach. Likewise, most students will learn academics better under an instructor who knows where improvement is needed and structures the practice required to achieve it. 19 The answer to illusion and misjudgment is to replace sub- jective experience as the basis for decisions with a set of ob- jective gauges outside ourselves, so that our judgment squares with the real world around us. When we have reliable refer- ence points, like cockpit instruments, and make a habit of checking them, we can make good decisions about where to focus our efforts, recognize when we’ve lost our bearings, and fi nd our way back again. Here are some examples.

Created from liberty on 2021-11-21 07:19:56.

Avoid Illusions of Knowing ê 125 Tools and Habits for Calibrating Your Judgment Most important is to make frequent use of testing and re- trieval practice to verify what you really do know versus what you think you know. Frequent low- stakes quizzes in class help the instructor verify that students are in fact learning as well as they appear to be and reveal the areas where extra atten- tion is needed. Doing cumulative quizzing, as Andy Sobel does in his po liti cal economics course, is especially powerful for consolidating learning and knitting the concepts from one stage of a course into new material encountered later. As a learner, you can use any number of practice techniques to self- test your mastery, from answering fl ashcards to explaining key concepts in your own words, and to peer instruction (see below). Don’t make the mistake of dropping material from your testing regime once you’ve gotten it correct a couple of times.

If it’s important, it needs to be practiced, and practiced again.

And don’t put stock in momentary gains that result from massed practice. Space your testing, vary your practice, keep the long view.

Peer instruction, a learning model developed by Eric Mazur, incorporates many of the foregoing principles. The material to be covered in class is assigned for reading beforehand. In class, the lecture is interspersed with quick tests that present students with a conceptual question and give them a minute or two to grapple with it; they then try, in small groups, to reach a consensus on the correct answer. In Mazur’s experi- ence, this pro cess engages the students in the underlying con- cepts of the lecture material; reveals students’ problems in Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard University Press, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/liberty/detail.action?docID=3301452.

Created from liberty on 2021-11-21 07:19:56.

Make It Stick ê 126 reaching understanding; and provides opportunities for them to explain their understanding, receive feedback, and assess their learning compared to other students. Likewise, the pro- cess serves as a gauge for the instructor of how well the stu- dents are assimilating the material and in what areas more or less work is needed. Mazur tries to pair students who initially had different answers to a question so that they can see an- other point of view and try to convince one another of who is right.For two more examples of this technique, see the pro- fi les of the professors Mary Pat Wenderoth and Michael D. Matthews in Chapter 8. 20 Pay attention to the cues you’re using to judge what you have learned. Whether something feels familiar or fl uent is not al- ways a reliable indicator of learning. Neither is your level of ease in retrieving a fact or a phrase on a quiz shortly after encountering it in a lecture or text. (Ease of retrieval after a delay, however, is a good indicator of learning.) Far better is to create a mental model of the material that integrates the various ideas across a text, connects them to what you al- ready know, and enables you to draw inferences. How ably you can explain a text is an excellent cue for judging compre- hension, because you must recall the salient points from memory, put them into your own words, and explain why they are signifi cant— how they relate to the larger subject.

Instructors should give corrective feedback, and learners should seek it. In his interview with Errol Morris, the psy- chologist David Dunning argues that the path to self- insight leads through other people. “So it really depends on what sort Brown, Peter C., et al. Make It Stick : The Science of Successful Learning, Harvard Uni