Research Report Review

Objective: The objective was to examine the rela- tionship between cockpit automation use and task- related and task-unrelated thought among airline pilots. Background: Studies find that cockpit automation can sometimes relieve pilots of tedious control tasks and afford them more time to think ahead. Paradoxi- cally, automation has also been shown to lead to lesser awareness. These results prompt the question of what pilots think about while using automation. Method: A total of 18 airline pilots flew a Boeing 747- 400 simulator while we recorded which of two levels of automation they used. As they worked, pilots were ver - bally probed about what they were thinking. Pilots were asked to categorize their thoughts as pertaining to (a) a specific task at hand, (b) higher-level flight-related thoughts (e.g., planning ahead), or (c) thoughts unrelated to the flight.

Pilots’ performance was also measured. Results: Pilots reported a smaller percentage of task-at-hand thoughts (27% vs. 50%) and a greater per - centage of higher-level flight-related thoughts (56% vs.

29%) when using the higher level of automation. How- ever, when all was going according to plan, using either level of automation, pilots also reported a higher per - centage of task-unrelated thoughts (21%) than they did when in the midst of an unsuccessful performance (7%).

Task-unrelated thoughts peaked at 25% when pilots were not interacting with the automation. Conclusion: Although cockpit automation may pro- vide pilots with more time to think, it may encourage pilots to reinvest only some of this mental free time in thinking flight-related thoughts. Application: This research informs the design of human–automation systems that more meaningfully engage the human operator.

Keywords: cockpit automation, awareness, mind wandering, attention IntroductIon An often-touted benefit of the introduction of automation to the airline cockpit is that it frees pilots’ attention from tedious control tasks and affords them more time to look up, think ahead, and focus on “the big picture” of the flight.

Time once spent staring at a few instruments can now be devoted to planning around potential weather hazards, monitoring the health of the airplane’s many systems, fielding requests from air traffic control, or contemplating alternatives should anything go amiss (Norman & Orlady, 1988; Wiener, 1988). Numerous studies have confirmed that, at least during some phases of flight, automation can indeed help lower pilot workload and free up time (Casner, 2009; Roscoe, 1992; Wiener, 1989). This leaves us with the question of how pilots make use of this free time. It is every flight instructor’s dream that pilots would use this time in the ways just described: to think ahead or mentally prepare themselves for any contingencies that might arise. But contrary to these hopes, studies of pilots flying while using high levels of automa- tion cast some doubt on the idea that pilots are using their free time in this way. In more than one experiment, when awareness was tested, pilots failed to answer basic questions about their situation (Endsley & Kiris, 1995) or even know where they were (Casner, 2005). Two explanations have been proposed for the loss of awareness associated with the use of automation. A first explanation is that, during periods in which the automation is used and things are going well, pilots might engage in what psychologists have referred to as “task- unrelated thought” or “mind wandering” (Schooler et al., 2011; Smallwood & Schooler, 2006) and cease to meaningfully follow the events that are transpiring in front of them. A second explanation is that, during periods in which the automation is used and difficulties are 501550 HFS XX X 10.1177/0018720813501550Human FactorsAutomation and Task-Unrelated Thought 2013 Address correspondence to Stephen M. Casner, NASA Ames Research Center, Mail Stop 262-4, Moffett Field, CA 94035, USA; [email protected].

Author(s) Note: The author(s) of this article are U.S.

government employees and created the article within the scope of their employment. As a work of the U.S. federal government, the content of the article is in the public domain.

Thoughts in Flight: Automation Use and Pilots’ Task-Related and Task-Unrelated Thought Stephen M. Casner, National Aeronautics and Space Administration, Moffett Field, California, USA, and Jonathan W. Schooler, University of California, Santa Barbara, USA HUMAN FACTORS Vol. 56, No. 3, May 2014, pp. 433 –442 DOI: 10.1177/0018720813501550 434 May 2014 - Human Factors encountered, pilots’ thoughts can sometimes become absorbed by the automation itself.

Numerous studies and accident reports describe situations in which pilots become concerned about the behavior of the automation, which then becomes the focus of pilots’ attention (Sar- ter, Woods, & Billings, 1997). Sorting out these potential effects of automa- tion on what pilots are thinking is difficult since previous studies have focused on measuring per - formance outcomes while pilots used different levels of automation. That is, no one has observed pilots using different levels of automa- tion while asking them what they were thinking about. This was the purpose of our study. MeasurIng What PIlots thInk WhIle FlyIng WIth autoMatIon We placed 18 Boeing 747-400 pilots in a simulator and asked them to fly an arrival into a busy New York airport. As we will soon discuss, the 747-400 practically offers the pilot two levels of automation when flying an arrival: one in which the problem of navigating the airplane along the assigned flight route is largely under the control of the automation, and one in which the pilots assume more responsi- bility for performing this task. Rather than cre- ate an artificial task in which pilots were asked to use a higher level of automation half of the time and a lower level during other times, we allowed pilots to fly as they normally would, deciding to use the automation as they saw fit.

As pilots made their way along the route, we periodically asked them to tell us whether their thoughts were directed at the “task at hand,” if they were thinking higher-order thoughts about the flight, such as planning ahead, or were thinking about something entirely unre- lated to the flight. Along the way, the levels of automation they used and various aspects of their performance were recorded. Our aim was to examine the relationship among the level of automation pilots were using, how well they were doing, and the focus of their thinking. the arrival Procedure We asked pilots to fly the published arrival procedure into New York’s John F. Kennedy International Airport (JFK) shown in Figure 1. This procedure requires pilots to cross a series of six named geographical locations called waypoints. Of particular interest are the two waypoints in the middle of the procedure, HARTY and LENDY, commonly referred to as crossing restrictions. When issued a crossing restriction, in addition to having to reach the waypoint, pilots are required to cross the way- point at an assigned altitude and sometimes also at an assigned airspeed. In our procedure, pilots were instructed to cross HARTY at an assigned altitude of 16,000 ft. and LENDY at an altitude Figure 1. Arrival procedure into JFK airport. AutomAtion And tAsk -u nrelAted thought 435 of 6,000 ft. and an airspeed of 250 knots. An acceptable performance is to pass within 1 nm of each waypoint, within 300 ft. of each assigned altitude, and within 10 knots of the assigned air- speed (Federal Aviation Administration, 2008).

t wo levels of automation To fly an arrival procedure such as the one shown in Figure 1, pilots must first program the procedure into the flight management computer that is standard equipment aboard every modern airliner. This step provides the flight manage- ment computer with the sequence of waypoints, altitudes, and speeds that make up the arrival procedure. Once the route is programmed, pilots must use a lateral navigation function called LNAV that automatically steers the airplane lat- erally between the six waypoints in the arrival.

To achieve the two prescribed altitudes associ- ated with the HARTY and LENDY waypoints and the assigned speed at the LENDY waypoint, the flight crew has a choice between two levels of automation. The difference between these two levels of automation lies in how much of the work of following the route the flight crew wishes to hand over to the flight management computer and how much they would like to perform themselves. To use the higher level of automation, pilots can engage the vertical navigation (VNAV) function. The VNAV function largely automates the process of meeting the altitude and speed restrictions at the two waypoints. The VNAV function performs a fairly complex task. The VNAV function must decide when to commence a descent to obey the altitudes associated with the two waypoints. In this case, if the airplane is cruising at 39,000 ft. and needs to cross HARTY at 16,000 ft., then the airplane must descend at total of 23,000 ft. If the computer chooses a descent rate of 2,000 ft./min, then the descent will take 11.5 min to complete. If the airplane is traveling at a speed of 300 knots, then it tra- verses 5.5 nm per minute. During the 11.5 min needed to complete the descent, the airplane will traverse 63.25 nm. The computer concludes that the descent must be commenced 63.25 nm prior to HARTY. The flight management computer will also need to figure out a way to reduce the speed of the airplane from its current speed of 300 knots to the required speed of 250 knots at LENDY. Since the VNAV function attempts to automatically manage the vertical progress of the airplane, it is commonly referred to as a managed function.

The LNAV and VNAV functions are engaged by pressing two buttons on the mode control panel shown in Figure 2. While the LNAV and VNAV functions do their work, it is important for the flight crew to monitor the progress of the airplane and to inter - vene in any situation in which the flight crew feels that the crossing restrictions may not be met. To use the lower level of automation, pilots will engage the LNAV function to automate the navigation between the waypoints, but take a more active part in guiding the airplane down to the altitudes and speeds prescribed for the way- points. Using the flight level change (FLCH), vertical speed (V/S), and altitude (ALT) func- tions illustrated in Figure 2, pilots can assume responsibility for performing the descent plan- ning calculations described earlier and carry out the descent manually. For example, the pilot might think through the problem of arriving at HARTY at 16,000 ft.

and estimate that a good place to start the descent is roughly 75 nm prior to HARTY. After moni- toring the progress of the airplane and noting that the 75 nm point has been reached, the pilot can dial an altitude of 16,000 ft. into the altitude (ALT) window shown in Figure 2, dial an air - speed of 250 knots into the speed (IAS/MACH) window, and engage the flight level change function by pressing the FL CH button. The pilot must then monitor the progress of the airplane as the descent continues since the decision about when to start the descent was the result of an informal estimation. Upon reaching the HARTY intersection at 16,000 ft., the pilot must then repeat all of the steps and initiate a new descent that will guide the airplane down to the next waypoint, LENDY, at the next assigned altitude and speed. Since functions like flight level change require the pilot to select each altitude and speed that must be achieved, they are com- monly referred to as selected functions. Pilots’ choices about which level of automa- tion to use are often more nuanced than it might 436 May 2014 - Human Factors seem. To use the VNAV function, pilots must have the time available to make (correct) alti- tude and speed entries into the flight manage- ment computer. If pilots suspect that air traffic control will change their minds about the alti- tudes and speeds and issue different ones later, pilots might simply use the selected vertical guidance functions (Casner, 1994). Similarly, if pilots feel that they might encounter unpredict- able changes in wind during the descent, they may not feel that the descent calculated by the flight management computer will ultimately work. Both of these situations could trigger sig- nificant extra work in reprogramming a flight management computer that is (ironically) designed to relieve pilots of work. Some pilots understand or trust highly automated functions such as VNAV more than others.

the thought Probes We adapted a thought sampling technique used to measure the frequency of pilots’ task-related and task-unrelated thoughts while they flew. This technique was first used by Singer (1966) and has since enjoyed popular use owing to its avoidance of disruption to participants engaged in complex or fast-paced tasks (Ericsson & Simon, 1980).

The reliability of the technique when compared to other measures of mind wandering behavior has been demonstrated in a number of studies (for a recent review, see Schooler et al., 2011).

Using this technique, pilots were asked to respond to verbal prompts from the experimenter in which they were asked to reveal the subject of their current thoughts using one of three thought categories. A response of 1 would indicate that pilots were thinking about a “task at hand.” Concen- trating on following a localizer and glide slope needles during an instrument approach or typing entries in a flight management computer were offered to pilots as examples of thoughts that should be categorized as a 1. A response of 2 would indicate that pilots were thinking about something related to the flight, but not something that was happening in front of them at that moment. Thinking about which instrument approach they might later receive or what might need to be entered into the flight management computer were offered to pilots as examples of thoughts that should be categorized as a 2. A response of 3 would indicate that pilots were thinking about something not related to the flight. Thinking about an upcoming vacation or what one might have for dinner that evening Figure 2. Mode control panel used to engage automation functions. AutomAtion And tAsk -u nrelAted thought 437 were offered to pilots as examples of thoughts that should be categorized as a 3.

hypotheses If automation fulfills its promise of relieving pilots from focusing on low-level control tasks and providing them with more time to focus on the “big picture” of the flight, we might expect pilots to report fewer 1s and more 2s when the higher level of automation is used (Hypoth- esis 1). If, on the other hand, higher levels of automation prompt pilots to drift “out of the loop,” we might expect pilots to report more 3s (Hypothesis 2). In addition, if pilots modulate their task-unrelated thoughts (3s) in response to the task demands, when difficulties in meeting the crossing restrictions are encountered, we might expect pilots to report more 1s and 2s and fewer 3s (Hypothesis 3). Method Participants A total of 18 active Boeing 747-400 pilots, 9 captains and 9 first officers, participated in the study on a voluntary basis. Pilots had an average of 11,056 hr of total flight time (SD = 3,670) and an average of 356 hr during the past 12 months ( SD = 178). Pilots reported having spent an average of 69% of their total flight time oper - ating advanced cockpit aircraft (SD = 23). In exchange for their participation, pilots received a standard hourly participant pay rate and reim- bursement for any travel expenses incurred. apparatus The Boeing 747-400 (Level D) flight simula- tor located at the NASA Ames Research Center was used for the experiment. The checklists and quick reference handbooks used by each employer of each pilot were provided for the simulator session. A handheld timer was used to time the delivery of the thought probes. A notebook and pen were used to record pilots’ responses to the probes. The accuracy at which pilots crossed the waypoints in the arrival pro- cedure was captured by the simulator’s flight data recorder functions. Video cameras captured activity inside the cockpit area from four differ - ent angles. Procedure The 18 pilots were asked to fly the JFK arrival procedure. All participant pilots sat in their respective seat: Captains occupied the left seat, first officers occupied the right seat.

Occupying the other seat was a confederate pilot who is also rated in the 747-400 aircraft and is presently employed by an air carrier company.

Sitting behind the two pilots was the experi- menter, whose function was to issue the air traf- fic control clearance for the arrival procedure to the pilots and to verbally prompt the experiment pilot for thought probes. After the descent clearance was issued, pilots were verbally prompted to categorize their thoughts every 2 min until each crossing fix had been reached. We elected to use a fixed time interval to maintain consistency between subjects, to avoid aligning our prompts with events that might seem favorable to the experi- menters, and knowing that subjects’ perceptions of time intervals vary wildly with changes in busyness, task engagement, enjoyment (Fried- man, 1990), caffeine consumption (Stine, O’Connor, Yatko, Grunberg, & Klein, 2002), and amount of sleep (Fichten, Creti, Amsel, Bailes, & Libman, 2005). The intervals between prompts were measured using a digital timer. At the time each prompt was delivered, the experimenter noted the automation functions that were in use.

When pilots were using a selected function (e.g., flight level change, vertical speed, altitude hold), we recorded that a selected function was in use.

When pilots were using a managed function (VNAV), we recorded that a managed function was in use. During the arrival procedure, each participant pilot was asked to assume control of the airplane and serve as the “pilot flying.” Since we attempted to place pilots in the most realistic flight setting possible, all pilots were instructed to fly the airplane as they would during any rou- tine flight operation. Hence, we refrained from asking pilots to use particular automation func- tions at particular times. As in any normal flying situation, pilots could manipulate the mode con- trol panel and flight management computer themselves, or instruct the other (confederate) pilot to make these inputs under their direction and supervision. We also noted whether or not 438 May 2014 - Human Factors either pilot was interacting with the mode con- trol panel or flight management computer at the time that each thought probe was delivered.The simulator’s flight data recorder noted the actual altitude and speed of the airplane at each of the two crossing restriction waypoints. results and dIscussIon Since we probed our 18 pilots about what they were thinking on a total of eight occasions each, our outcome variable, which we will refer to as thought category, amounted to a total of 144 responses. Since these data are categorical (nominal) and contain repeated measures obtained from each pilot, we used multinomial logistic regression to examine the association between the three independent variables we recorded and our outcome variable, thought category. The binary automation level variable reflects whether pilots used a selected or managed auto- mation function at the time they were probed.

The binary success variable reflects whether pilots were in the midst of making or missing a crossing restriction at the time that each probe was given. A tolerance of ±300 ft. and ±10 knots was used when deciding whether or not pilots made or missed the assigned altitude and speed associated with the crossing restrictions. The binary interacting variable reflects whether or not pilots were configuring or manipulating (i.e., “hands on”) the mode control panel or flight management computer at the time that the thought prompt was made by the experimenter.

Values for this variable were verified by review- ing the video and audio footage from each simu- lator session. Since we found the success and interacting variables to be significantly correlated (ø = .26, p < .01), we set aside the interacting variable and built a model using the automation level and success variables, using a thought code of 1 as the base outcome. A test of this model compared to a constant-only model was statistically sig- nificant, χ 2(4) = 12.52, p < .05, and produced one significant coefficient for each of the two predictor variables. More automation: From 1s to 2s To examine the relationship between the level of automation that pilots used and what they were thinking, Figure 3 aggregates the data with respect to these two variables. Differences between the two rows in Figure 3 depict a significant shift from task-at-hand thoughts (1s) to higher-level thoughts about the flight (2s) when the higher level of automation was used: β = 1.16, p < .05. This finding sup- ports Hypothesis 1 considered in our introduc- tion and what is perhaps the most closely held belief about automation: that the use of a higher- level of automation is associated with pilots thinking fewer task-at-hand thoughts (1s) and more higher-level thoughts about the flight (2s). success: From 1s to 3s To examine the relationship between the level of success pilots experienced and what they were thinking, Figure 4 aggregates the data with respect to these two variables. Differences between the two rows in Figure 4 depict a significant shift from task-at-hand thoughts (1s) to task-unrelated thoughts (3s) THOUGHT CATEGORY 1 Task-At-Hand Thoughts 2 Higher-Level (Task-Related) Thoughts 3 Task-Unrelated Thoughts Selected (Less Automated) 50% (12) 29% (7) 21% (5) Managed (More Automated) 27% (32) 56% (67) 18% (21) Figure 3. Data aggregated to examine the relationship between the automation leve\ l and thought category variables. AutomAtion And tAsk -u nrelAted thought 439 when pilots were probed in the midst of making a crossing restriction: β = 1.68, p < .05. The dif- ference in the percentage of higher-level task- related thoughts (2s) was not significant. This finding addresses Hypothesis 2 considered in our introduction and adds an interesting twist.

Pilots’ thoughts did not drift onto other topics when a higher level of automation was used, but rather when either level of automation was used successfully. This finding adds a similar twist to Hypothesis 3. Difficulties in meeting the cross- ing restrictions were associated with pilots reporting more task-at-hand thoughts (1s), sug- gesting that automation struggles diverted pilots’ attention away from higher-level thoughts about the flight (2s). However, when pilots enjoyed more success and reported fewer task-at-hand thoughts (1s), their thoughts seemed to move on to task-unrelated topics (3s).

automation Interactions: From 1s to 3s Hypothesis 3 alludes to a situation in which pilots become engrossed in interactions with the automation as their awareness of other aspects of the flight decreases. Our data allow us to directly examine the relationship between auto- mation interactions and pilots’ thought catego- ries. Figure 5 aggregates the data with respect to these two variables.

Using the interacting variable in place of the success variable, we again obtain a significant regression model, χ 2(4) = 23.34, p < .01. As was the case with the success variable, interacting with the automation was not associated with fewer higher-level thoughts about the flight (2s):

It was associated with fewer task-unrelated thoughts (3s): β = −3.09, p < .01. suMMary and conclusIon Our results help to reconcile some of the most basic and seemingly contradictory claims about the effect of automation on pilots’ thinking. Our data support the most closely held belief about automation: that the use of more automation allows pilots to engage in fewer task-at-hand thoughts and more higher-level thoughts about THOUGHT CATEGORY 1 Task-At-Hand Thoughts 2 Higher-Level (Task-Related) Thoughts 3 Task-Unrelated Thoughts Missed 50% (14) 43% (12) 7% (2) Made 26% (30) 53% (62) 21% (24) Figure 4. Data aggregated to examine the relationship between the success and tho\ ught category variables. THOUGHT CATEGORY 1 Task-At-Hand Thoughts 2 Higher-Level (Task-Related) Thoughts 3 Task-Unrelated Thoughts Hands On 47% (20) 51% (22) 2% (1) Hands Off 24% (24) 52% (52) 25% (25) Figure 5. Data aggregated to examine the relationship between the interacting and\ thought category variables. 440 May 2014 - Human Factors the flight. At the same time, our findings are also consistent with studies that demonstrate that, when more automation is used, mea- sures of pilot awareness show that less, not more, higher-level flight-related thinking has taken place. We found that when difficulties were encountered, the use of a higher level of automation may have substituted one sort of attention-demanding work for another. When all was going to plan, and the task of managing the airplane was seemingly under control, pilots often opted to think about something else.

the Pros and cons of task-unrelated thought In fairness, we must consider the idea that the activity that we refer to as task-unrelated thought is nuanced with advantages and dis- advantages. A striking result found throughout the literature concerns the ubiquity of task- unrelated thought, or mind wandering. Studies demonstrate that people appear to spend about 30% of their entire waking life engaged in task- unrelated thought (Kane et al., 2007), roughly the proportion of time that we observed among our pilots. This invites a question: If mind wan- dering was a wasteful use of a precious cogni- tive resource in a world filled with persistent challenges, fleeting opportunities, and pop-up hazards, how could we have even survived?

A number of researchers have made a case for mind wandering as being a crucial part of human cognition that serves a number of impor - tant functions. Ariga and Lleras (2011) demon- strate a link between brief mental breaks and improvements in vigilance performance. Baird et al. (2012) report evidence indicating that mind wandering can facilitate creative problem solving or even discovery. Baird, Smallwood, and Schooler (2011) offer evidence that task- unrelated thought enables “autobiographical planning.” When engaged in autobiographical planning, pilots may be thinking ahead not only to future portions of a flight but rather to future portions of their life. Although task-unrelated thought may offer benefits, other studies have demonstrated detri- mental effects. Task-unrelated thought has been demonstrated to lead to a greater propensity for error (Smallwood, Fishman, & Schooler, 2007), predictable slumps in reading comprehension (Smallwood, McSpadden, Luus, & Schooler, 2008), and more careless response in a go/no-go decision task (Smallwood et al., 2008).

The apparent advantages and disadvantages of task-unrelated thought invite the question of whether there is such a thing as “good” mind wandering and “bad” mind wandering. Schooler and colleagues (2011) have demonstrated a piv- otal characteristic of episodes in which the mind wanders: that people are sometimes aware of it and sometimes not. Being aware of one’s own mind wandering appears to be a critical fac- tor in determining its impact on performance.

When mind wandering evades one’s own aware- ness, it is more likely to happen at inopportune times and result in deleterious effects. Schooler, Reichle, and Halpern (2005) found that readers were often unaware that they were mind wan- dering while reading passages of War and Peace, an exercise that was followed by a comprehen- sion test. Schooler and colleagues refer to this type of mind wandering, which seems to offer little benefit, as “zoning out.” That the mind wandering that we observed was largely associ- ated with better performance might suggest that pilots knew when they were mind wandering, but without direct evidence of pilots’ own aware- ness of mind wandering, we cannot be sure. A future study might make use of various tech- niques for assessing the relative proportion of aware versus unaware mind wandering episodes (Schooler et al., 2011). Another pivotal charac- teristic of task-unrelated thought is whether the episodes happen during opportune or inoppor - tune moments. Our results suggest that pilots are disciplined in distinguishing good times and bad times for task-unrelated thoughts. the design of cockpit automation (and Procedures for using It) We must consider the possibility that the thought patterns we observe among pilots are the rational outcome of the way we have designed cockpit automation systems. By introducing systems that automate much of what pilots do, and that do it so reliably (when properly configured), we may have left pilots with little incentive to think beyond the steps needed to configure the automation and the aircraft behav- iors that these steps produce. And since pilots receive little procedural guidance about how to AutomAtion And tAsk -u nrelAted thought 441 actively monitor automated systems, we may have effectively left them with the question:

“What else is there to think about?” If this is the case, we might wonder if we could encour - age a different use of pilots’ mental free time.

One idea, suggested by Sumwalt (2003), is to design specific procedures for actively monitor - ing in the automated cockpit. Another idea might be to design automated systems that “check in” by challenging pilots with tasks that bring them back into the loop when things are under control and going well. A follow-up study by Casner (2006) found that even perfunctory conversa- tion among pilots about where they were and where they were going was enough to reverse the “out-of-the-loop” effects seemingly caused by using advanced navigational automation.

Another idea might be to wholly redesign the way that humans and automated systems share the job of operating a complex system such as an aircraft. We might start from the beginning, consider the strengths and limitations of both humans and computers, and combine them in ways that exploit the best features of both. In no uncertain terms, our results suggest that efforts aimed at “working the bugs out” of current automation systems might simply lead to more task-unrelated thought.

limitations of our study We must acknowledge a number of limita- tions of our study. A first limitation is that we did not directly measure pilots’ awareness out of concern for alerting pilots to the purpose of our study and influencing responses to our thought probes. However, the fact that when pilots per - formed well they registered fewer task-at-hand thoughts and more task-unrelated thoughts, with only a slight increase in the number of higher- level flight-related thoughts does not inspire much confidence in the idea that pilots devoted much time to attaining greater awareness. A second limitation is that pilots may have been reluctant to fully admit the degree to which their minds were drifting. Although it is impos- sible to fully assess the role that such concerns may have played in this experiment, a number of findings speak to this issue. First, it is notable that virtually all of the participants acknowl- edged a substantial proportion of task-unrelated thoughts. Thus, it seems participants were not making any overall effort to avoiding reporting task-unrelated thoughts to protect their reputa- tions. Second, the occurrence of self-reported mind wandering varied in systematic ways that were consistent with existing theories and empirical findings. Future research might fur - ther explore this issue by combining self-report measures with indirect physiological indices of mind wandering such as patterns of pupil dila- tion (Smallwood et al., 2011). A third issue deserving of further investiga- tion is the role that task-related and -unrelated thoughts play in actual flying situations.

Although the present study placed actual pilots in a realistic simulator, it nevertheless repre- sented a simulated experience. In this regard it is notable that the present findings almost certainly underestimated the frequency with which task- unrelated thought is likely to take place in actual flying situations. The present study presented a relatively short and unusually demanding situa- tion in which the pilots knew they were being evaluated. The fact that even under these situa- tions we observed a significant frequency of task-unrelated thoughts suggests that under the less demanding and extended durations of long- range flight, pilots may experience a substan- tially larger proportion of task-unrelated thoughts. Future research might profitably explore this issue by employing the present par - adigm in the context of actual flight situations. A fourth limitation of our study is that we have studied only a single task that was per - formed at two fairly high levels of automation. A future study might look at a greater variety of tasks for which automation is used to free up or burden the operator in a greater variety of ways.

In light of our finding that it was success (and not necessarily automation) that led to more task-unrelated thought, we might find that task- unrelated thought is an issue for humans per - forming in any sort of supervisory role and may have been a human factor of interest long before the introduction of automation. acknoWledgMents This work was supported by the Aviation Safety Program at the National Aeronautics and Space Administration. 442 May 2014 - Human Factors key PoInts • When higher levels of automation are used, pilots’ thoughts move away from specific tasks-at-hand and onto higher-level matters related to the flight (e.g., planning ahead).

• When pilots are in the midst of a successful per - formance, regardless of level of automation being used, pilots’ thoughts often drift to matters unre- lated to the flight.

• Although automation is associated with more higher-order thoughts about the flight, these thoughts may often center on struggles with get- ting the automation to work as desired. reFerences Ariga, A., & Lleras, A. (2011). Brief and rare mental “breaks” keep you focused: Deactivation and reactivation of task goals pre- empt vigilance decrements. Cognition, 11 8 , 439–443.

Baird, B., Smallwood, J., Mrazek, M. D., Kam, J., Franklin, M. S., & Schooler, J. W. (2012). Inspired by distraction: Mind-wan- dering facilitates creative incubation. Psychological Science, 23(10), 1117–1122.

Baird, B., Smallwood, J., & Schooler, J. W. (2011). Back to the future: Autobiographical planning and the functionality of mind-wandering. Consciousness and Cognition, 20, 1604–1611.

Casner, S. M. (1994). Understanding the determinants of problem- solving behavior in a complex environment. Human Factors, 36, 580–596.

Casner, S. M. (2005). The effect of GPS and moving map displays on navigational awareness while flying under VFR. Interna- tional Journal of Applied Aviation Studies, 5, 153–165.

Casner, S. M. (2006). Mitigating the loss of navigational aware- ness while flying under VFR. International Journal of Applied Aviation Studies, 6, 121–129.

Casner, S. M. (2009). Perceived vs. measured effects of advanced cockpit systems on pilot workload and error: Are pilots’ beliefs misaligned with reality? Applied Ergonomics, 40, 448–456.

Endsley, M. R., & Kiris, E. O. (1995). The out-of-the-loop per - formance problem and level of control in automation. Human Factors, 37, 381–394.

Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.

Federal Aviation Administration. (2008). Airline transport pilot and aircraft type rating practical test standards for airplane (FAA-S-8081-5F). Washington, DC: Federal Aviation Admin- istration, Flight Standards Service.

Fichten, C. S., Creti, L., Amsel, R., Bailes, S., & Libman, E. (2005). Time estimation in good and poor sleepers. Journal of Behavioral Medicine, 28, 537–553.

Friedman, W. (1990). About time: Inventing the fourth dimension.

Cambridge, MA: MIT Press.

Kane, M. J., Brown, L. H., McVay, J. C., Silvia, P. J., Myin-Ger - meys, I., & Kwapil, T. R. (2007). For whom the mind wanders, and when. Psychological Science, 18, 614–621.

Norman, S. D., & Orlady, H. W. (1988). Flight deck: Promises and realities (NASA Technical Memorandum 10036). Moffett Field, CA: NASA Ames Research Center. Roscoe, A. H. (1992, April).

Workload in the glass cockpit. Flight safety digest. Alexandria, VA: Flight Safety Foundation.

Sarter, N. B., Woods, D. D., & Billings, C. (1997). Automation sur - prises. In G. Salvendy (Ed.), Handbook of human factors/ergo- nomics (2nd ed., pp. 1926-1943). New York, NY: John Wiley.

Schooler, J. W., Reichle, E. D., & Halpern, D. V. (2005). Zoning- out during reading: Evidence for dissociations between experi- ence and meta-consciousness. In D. T. Levin (Ed.), Thinking and seeing: Visual metacognition in adults and children (pp. 204–226). Cambridge, MA: MIT Press.

Schooler, J. W., Smallwood, J., Christoff, K., Handy, T. C., Reichle, E. D., & Sayette, M. A. (2011). Meta-awareness, perceptual decoupling and the wandering mind. Trends in Cognitive Science, 15, 319–326.

Singer, J. L. (1966). Daydreaming: An introduction to the experimen- tal study of inner experience. New York, NY: Random House.

Smallwood, J., Brown, K. S., Tipper, C., Giesbrecht, B., Franklin, M. S., Mrazek, M. D., Carlson, J. M., & Schooler, J. W. (2011).

Pupillometric evidence for the decoupling of attention from perceptual input during offline thought. PLoS ONE, 6, e18298.

Smallwood, J., Fishman, D. F., & Schooler, J. W. (2007). Counting the cost of the absent mind. Psychonomic Bulletin and Review, 14, 230–236.

Smallwood, J., McSpadden, M., Luus, B., & Schooler, J. (2008). Segmenting the stream of consciousness: The psychological correlates of temporal structures in the time series data of a continuous performance task. Brain and Cognition, 66, 50–56.

Smallwood, J., & Schooler, J. W. (2006). The restless mind. Psychological Bulletin, 132, 946–958.

Stine, M. M., O’Connor, R. J., Yatko, B. R., Grunberg, N. E., & Klein, L. C. (2002). Evidence for a relationship between daily caffeine consumption and accuracy of time estimation.

Human Psychopharmacology: Clinical and Experimental, 17, 361–367.

Sumwalt, R. (2003, August). Cockpit monitoring: Using proce- dures to enhance crew vigilance. Professional Pilot, pp. 2–6.

Wiener, E. L. (1988). Cockpit automation. In E. L. Wiener & D. C. Nagel (Eds.), Human factors in aviation (pp. 433–459). San Diego, CA: Academic Press.

Wiener, E. L. (1989). Human factors of advanced technology (“glass cockpit”) transport aircraft (NASA Contractor Report 177528). Moffett Field, CA: NASA Ames Research Center. Stephen M. Casner is a research psychologist at the NASA Ames Research Center. He holds a PhD from the Intelligent Systems Program at the University of Pittsburgh, and an FAA Airline Transport Pilot cer - tificate. His research focuses on the impact of tech- nology on humans at work.

Jonathan W. Schooler is a professor of psychological and brain sciences at the University of California, Santa Barbara. He earned his PhD in cognitive psy- chology from the University of Washington. His research focuses on the nature of mind wandering and its impact on cognitive functioning.

Date received: December 27, 2012 Date accepted: July 14, 2013