Develop Exercise Evaluation Guide ( EEG) for Tabletop Exercise”TTX”. please be familiar with TTX

EXERCISE DESIGN Independent Study IS139 March, 2003E M E R G E N C Y M A N A G E M E N T I N S T I T U T E FEMA TABLE OF CONTENTS Exercise Design Page i Page Course Overview .................................... .............................................................................1 Unit 1: Introduction to Exercise Design Introduction ........................................ ..........................................................................1.1 Unit 1 Objectives ................................... .......................................................................1.1 Why Exercise? ........................................ .....................................................................1.1 Regulatory Requirements ............................ .................................................................1.6 Functions........................................... ...........................................................................1.7 Activity: Needs Assessment .............................. ..........................................................1.9 Summary and Transition .............................. .............................................................. 1.14 For More Information ............................... ................................................................... 1.14 Knowledge Check ..................................... ................................................................. 1.15 Unit 2: Comprehensive Exercise Program Introduction ........................................ ..........................................................................2.1 Unit 2 Objectives ................................... .......................................................................2.1 Progressive Exercising ................................. ................................................................2.1 Who Participates? .................................... ....................................................................2.3 What Activities Are Included? ......................... ..............................................................2.5 Comparing the Five Activities ....................... .............................................................. 2.17 Building an Exercise Program ......................... ........................................................... 2.20 Activity: Develop a Comprehensive Exercise Program Pla n ...................................... 2.27 Summary and Transition .............................. .............................................................. 2.31 For More Information ............................... ................................................................... 2.31 Knowledge Check ..................................... ................................................................. 2.32 Unit 3: The Exercise Process Introduction ........................................ ..........................................................................3.1 Unit 3 Objectives ................................... .......................................................................3.1 The Big Picture..................................... ........................................................................3.2 Accomplishment 1: Establishing the Base ................. ..................................................3.5 Accomplishment 2: Exercise Development ................. .................................................3.6 Accomplishment 3: Exercise Conduct ...................... ...................................................3.8 Accomplishment 4: Exercise Evaluation and Critique ..... .............................................3.9 Accomplishment 5: Exercise Followup .................... .................................................. 3.10 Reviewing the Current Plan ......................... ............................................................... 3.11 Assessing Capability to Conduct an Exercise ............... .............................................. 3.12 Addressing Costs and Liabilities ....................... .......................................................... 3.13 Activity: Getting Ready for Exercise Design ........... ...................................................3.14 Gaining Support .................................... ..................................................................... 3.18 Assembling a Design Team.............................. .......................................................... 3.21 Activity: Identify Design Team Members.............. ...................................................... 3.29 Exercise Documents..................................... .............................................................. 3.30 Summary and Transition .............................. .............................................................. 3.33 For More Information ............................... ................................................................... 3.33 Knowledge Check ..................................... ................................................................. 3.34 TABLE OF CONTENTS Page ii Exercise Design Page Unit 4: Exercise Design Steps Introduction ........................................ ..........................................................................4.1 Unit 4 Objective .................................... ........................................................................4.1 Step 1: Assess Needs .................................. ...............................................................4.2 Step 2: Define the Scope........................... ..................................................................4.4 Step 3: Write a Statement of Purpose ............... ..........................................................4.7 Activity: Define Exercise Scope and Purpose ............ ................................................ 4.10 Step 4: Define Objectives .......................... ................................................................ 4.14 Activity: Analyze an Objective...................... .............................................................. 4.17 Activity: Good or Poor Objectives ................... ........................................................... 4.21 Activity: Develop Objectives ........................ .............................................................. 4.24 Step 5: Compose a Narrative ........................ ............................................................ 4.25 Activity: Outline a Narrative ...................... ................................................................. 4.29 Step 6: Write Major and Detailed Events ........... ........................................................ 4.30 Step 7: List Expected Actions ........................ ............................................................ 4.33 Activity: Write Major and Detailed Events .......... ........................................................ 4.36 Step 8: Prepare Messages ............................ ............................................................ 4.38 Activity: Compose a Message ........................... ........................................................ 4.44 Pulling It Together: The Master Scenario of Events Li st ............................................ 4.45 Alternatives to Self-Developed Exercises .............. ..................................................... 4.46 Summary and Transition .............................. .............................................................. 4.47 For More Information ............................... ................................................................... 4.47 Knowledge Check ..................................... ................................................................. 4.48 Unit 5: The Tabletop Exercise Introduction ........................................ ..........................................................................5.1 Unit 5 Objectives ................................... .......................................................................5.1 Characteristics of the Tabletop Exercise ................. .....................................................5.1 How a Tabletop Works ............................... ..................................................................5.3 Facilitating a Tabletop Exercise ...................... ..............................................................5.4 Activity: Your Ideas for Facilitating a Tabletop ..... ........................................................5.8 Designing a Tabletop Exercise ......................... ............................................................5.9 Activity: Develop Problem Statements ............... ........................................................ 5.13 Activity: Develop a Tabletop Exercise Message .......... .............................................. 5.17 Summary and Transition .............................. .............................................................. 5.20 For More Information ............................... ................................................................... 5.20 Knowledge Check ..................................... ................................................................. 5.21 Exercise Design Page iii Page Unit 6: The Functional Exercise Introduction ........................................ ..........................................................................6.1 Unit 6 Objectives ................................... .......................................................................6.1 What Is a Functional Exercise? ......................... ...........................................................6.2 Activity: Compare Tabletop and Functional Exercises ... ..............................................6.4 Participant Roles ................................... .......................................................................6.6 How a Functional Exercise Works ....................... ....................................................... 6.11 Facilities and Materials ............................. .................................................................. 6.18 Designing a Functional Exercise ........................ ........................................................ 6.22 Activity: Identify Functional Exercise Responsibilities . ............................................... 6.25 Summary and Transition .............................. .............................................................. 6.27 For More Information ............................... ................................................................... 6.27 Knowledge Check ..................................... ................................................................. 6.28 Unit 7: The Full-Scale Exercise Introduction ........................................ ..........................................................................7.1 Unit 7 Objectives ................................... .......................................................................7.1 What Is a Full-Scale Exercise?......................... ............................................................7.2 Activity: Know Your Regulatory Requirements ........ ....................................................7.3 Activity: Compare Functional and Full-Scale Exercises .. .............................................7.6 Full-Scale Exercise Roles ............................. ................................................................7.8 How the Full-Scale Exercise Works..................... .........................................................7.9 Exercise Locations .................................... ................................................................. 7.11 Designing the Full-Scale Exercise ...................... ........................................................ 7.12 Special Considerations ............................... ................................................................ 7.14 Activity: Plan Ahead for the Full-Scale Exercise ...... .................................................. 7.22 Summary and Transition .............................. .............................................................. 7.24 For More Information ............................... ................................................................... 7.24 Knowledge Check ..................................... ................................................................. 7.25 TABLE OF CONTENTS Page iv Exercise Design Page Unit 8: Exercise Evaluation Introduction ........................................ ..........................................................................8.1 Unit 8 Objectives ................................... .......................................................................8.1 Integrating Evaluation Throughout Development ...... ....................................................8.2 Why Evaluate the Exercise?........................... ..............................................................8.3 The Evaluation Team ................................ ...................................................................8.4 Evaluation Methodology ............................. ..................................................................8.7 Evaluator Checklist.................................. ................................................................... 8.12 Narrative Summary .................................. .................................................................. 8.13 Key Event Response Form .............................. ........................................................... 8.15 Problem Log ........................................ ....................................................................... 8.16 Postexercise Meetings ................................. .............................................................. 8.17 Exercise Debriefing Log .............................. ............................................................... 8.19 Exercise Critique Form ............................... ................................................................ 8.20 After Action Report ................................. .................................................................... 8.22 Implementing Change ................................ ................................................................ 8.24 Activity: Plan the Evaluation ...................... ................................................................ 8.25 Summary and Transition .............................. .............................................................. 8.27 For More Information ............................... ................................................................... 8.27 Knowledge Check ..................................... ................................................................. 8.28 Unit 9: Exercise Enhancements Introduction ........................................ ..........................................................................9.1 Unit 9 Objectives ................................... .......................................................................9.1 Why Use Enhancements? ................................. ...........................................................9.2 Communications Equipment............................. ............................................................9.3 Visuals............................................. .............................................................................9.4 People and Props ................................... ......................................................................9.7 Enhancement Resources ................................. ............................................................9.8 Enhancement Logistics ................................. ...............................................................9.9 Activity: Enhance a Scenario ......................... ............................................................ 9.10 Activity: Plan Enhancements for Your Exercise .......... ............................................... 9.13 Summary and Transition .............................. .............................................................. 9.15 For More Information ............................... ................................................................... 9.15 Knowledge Check ..................................... ................................................................. 9.16 Exercise Design Page v Page Unit 10: Designing a Functional Exercise Introduction ........................................ ........................................................................ 10.1 Unit 10 Objective ................................... ..................................................................... 10.2 Select Your Exercise .................................. ................................................................ 10.2 Step 1: Assess Needs .................................. ............................................................. 10.3 Step 2: Define the Scope........................... ................................................................ 10.8 Step 3: Write a Statement of Purpose ............... ...................................................... 10.11 Step 4: Define the Objectives ...................... ............................................................ 10.12 Step 5: Compose a Narrative ........................ .......................................................... 10.13 Step 6: Write Major and Detailed Events ........... ...................................................... 10.16 Step 7: List Expected Actions ........................ .......................................................... 10.16 Step 8: Prepare Messages ............................ .......................................................... 10.18 Constructing the Master Scenario of Events List .......... ............................................ 10.23 Planning the Exercise Evaluation ..................... ........................................................ 10.25 Planning the Exercise Enhancements .................... .................................................. 10.27 Optional Activity: Exercise Design Documents ........... ............................................. 10.29 Summary and Transition .............................. ............................................................ 10.30 Unit 11: Course Summary Introduction ........................................ ........................................................................ 11.1 Reasons to Exercise.................................... ............................................................... 11.1 Comprehensive Exercise Programs ....................... .................................................... 11.1 The Exercise Process.................................... ............................................................. 11.3 Exercise Design Steps ................................. .............................................................. 11.4 The Tabletop Exercise................................ ................................................................ 11.5 The Functional Exercise ............................... .............................................................. 11.6 The Full-Scale Exercise............................... ............................................................... 11.7 Exercise Evaluation .................................. .................................................................. 11.9 Exercise Enhancements ................................. .......................................................... 11.10 Next Steps......................................... ....................................................................... 11.11 Appendix A: Job Aids Appendix B: Acronym List Appendix C: Exercise Tool Box Course Overview COURSE O VERVIEW Exercise Design Page 1 About This Course Emergencies happen. Emergencies can be limited in scope o r they can reach disaster proportions, sweeping through an entire co mmunity or multiple communities. Being prepared to respond to an d recover from emergencies is everyone’s challenge. Whether your organ ization is a government agency tasked with a particular response role , a volunteer agency that responds to the community’s needs, or a priva te sector entity that may be faced with an emergency situation, you hav e an important role in that preparation. As an outcome of your community’s or organization’s emer gency planning process, plans should be in place that specify how you prepare for emergencies, how you will respond if an emergency o ccurs, how you will mitigate the potential effects of emergencies, and how you will recover. Practice is an important aspect of the preparat ion process. Experience and data show that exercises are a practical, e fficient, and cost-effective way for organizations in the government, nonprofit, and private sectors to prepare for emergency response and reco very. This course is based on one important premise: Emergency exercises are worth the effort . Exercises identify areas that are proficient and those that need improvement. Lessons learned from exercises can be used to revise operational plans and provide a basis for traini ng to improve proficiency in executing those plans. This course is designed to introduce you to the fundament als of exercise design and to prepare you to design and conduct a small functional exercise for your organization. It addresses: The value of conducting exercises.

The components of a comprehensive exercise program.

The exercise development process development tasks, organization of the design team, exercise documentation , and the steps in designing an exercise.

COURSE OVERVIEW Page 2 Exercise Design About This Course (Continued) This course will cover the purpose, characteristics, and req uirements of three main types of exercises: Tabletop exercise Functional exercise Full-scale exercise In addition this course will cover: Exercise evaluation.

Exercise enhancements.

Designing a functional exercise. This course is one of several courses in exercise design offer ed through the Emergency Management Institute (EMI). To get inf ormation about other courses, check online at www.fema.gov/emi .

FEMA’s Independent Study Program The Federal Emergency Management Agency’s (FEMA’s) Inde pendent Study Program is one of the delivery channels that EMI uses to provide training to the general public and specific audiences. T his course is part of FEMA’s Independent Study Program. In addition to this course, the Independent Study Program includes other courses in the Professional Development Series (PDS), as well as courses in floodplai n management, radiological emergency management, the ro le of the emergency manager, hazardous materials, disaster assistance, the role of the Emergency Operations Center (EOC), and an orie ntation to community disaster exercises. FEMA’s independent study courses are available at no char ge and include a final examination. You may apply individu ally or through group enrollment. When enrolling for a course, you must incl ude your name, mailing address, social security number, and the title of the course in which you wish to enroll. COURSE OVERVIEW Exercise Design Page 3 FEMA’s Independent Study Program (Continued) If you need assistance with enrollment, or if you have questions about how to enroll, contact the Independent Study Program Administrative Office at: FEMA Independent Study Program Administrative Office Emergency Management Institute 16825 South Seton Avenue Emmitsburg, MD 21727 (301) 447-1200 Information about FEMA’s Independent Study Program a lso is available on the Internet at: http://training.fema.gov/EMIWeb/IS .

Each request will be reviewed and directed to the appr opriate course manager or program office for assistance. Course Prerequisites Exercise Design has no prerequisites. Final Examination This course includes a written final examination, which y ou must complete and return to FEMA’s Independent Study Office for scoring. To obtain credit for taking this course, you must successfully co mplete this examination with a score of 75 percent or above. You may take the final examination as many times as necessary. When you have completed all of the units, you may comp lete the final examination online or use the answer sheet (if one is p rovided in your course packet). If you choose to use the answer sheet, you must return it to the FEMA Independent Study Office at the addre ss listed on the previous page. EMI will score your test and notify you of the results.

Course Completion The course completion deadline for all FEMA Independen t Study courses is one year from the date of enrollment. The date o f enrollment is the date that the EMI Independent Study Office will use f or completion of all required course work, including the final examination. If you do not complete this course, including the final examination, w ithin that timeframe, your enrollment will be terminated.

COURSE OVERVIEW Page 4 Exercise Design How to Take This Course This Independent Study course is designed so that you can complete it on your own at your own pace. Take a break after ea ch unit, and give yourself time to think about the material, particular ly how it relates to your work in the exercise design function of emergency manage ment.

On the other hand, this course may be done in a group setting guided by an instructor. This Independent Study can also be used i n such groups.

Exercise Design has 11 units. Each of these units is described below.

Unit 1, Introduction to Exercise Design, explores the benefits organizations derive from exercising and leads you thro ugh a preliminary needs assessment. Unit 2, Comprehensive Exercise Program, provides an overview of five main types of exercise activities that make up a com prehensive exercise program. Unit 3, The Exercise Process, presents an overview of the process used to plan and implement a single exercise within a co mprehensive program. It also introduces four key design documents. In this unit, you will take some preliminary “groundwork” steps, incl uding a resource self-assessment and identifying potential design t eam members. Unit 4, Exercise Design Steps, takes you through an eight-step process for designing an exercise and provides a variety o f job aids to use in applying that process. You will practice each st ep in an abbreviated fashion as you progress through the unit. Unit 5, The Tabletop Exercise, takes a closer look at the tabletop exercise, including key characteristics, best uses, guidelines for facilitation, and special design considerations. Unit 6, The Functional Exercise, examines the functional exercise more closely again focusing on key characteristics, best uses, guidelines for control, and special design considerations. Unit 7, The Full-Scale Exercise, takes a similar look at full-scale exercises and how they differ from the other types of e xercises.

Unit 8, Exercise Evaluation, briefly discusses key aspects of evaluation methodology and the evaluation tasks that must take place before, during, and after an exercise. COURSE OVERVIEW Exercise Design Page 5 How to Take This Course (Continued) Unit 9, Exercise Enhancements, presents ideas for enhancing an exercise through visuals, equipment, props, and people. Unit 10, Designing a Functional Exercise, walks you through applying the eight design steps using the provided job aids in developing a simple functional exercise. Unit 11, Course Summary, prepares you for the final exam by presenting a brief review of the key points covered in the course.

Activities This course will involve you actively as a learner by i ncluding activities that highlight basic concepts. These activities emphasize d ifferent learning points, so be sure to complete all of them. C ompare your answers to the answers provided following each activity. If your answers are correct, continue on with the material. If any of your answers is incorrect, go back and review the material before conti nuing. Knowledge Checks To help you know when to proceed to the next unit, U nits 1 through 10 are followed by a Knowledge Check that asks you to answ er questions that pertain the unit content. Answers are provided f ollowing each Knowledge Check. When you finish each Knowledge Check, compare you answers to those provided, and review the parts of the text that you do not understand. It would be to your benefit to b e sure that you have mastered the current unit before proceeding to the nex t unit.

Exercise Design Tool Box A Tool Box is also included with the course materials. T he Exercise Design Tool Box contains a variety of exercise examples a nd templates to supplement the instruction. The course materials will instruct you when to use the Tool Box. The Tool Box can be accessed a t http://training.fema.gov/EMIWeb/IS/is139lst.asp .

COURSE OVERVIEW Page 6 Exercise Design How to Take This Course (Continued) Job Aids Throughout the course, you will find job aids designed to supplement the text. You can use the job aids during the course, and y ou will find them useful later, after you have completed the course. Cop ies of all job aids are assembled in Appendix A, for easy reference. Appendixes At the back of this course are appendixes that include, i n addition to the job aids, an acronym list. Equipment Requirements To complete this course, you will need the following eq uipment:

A computer with the following or greater capability:

Pentium processor Windows 95+ or NT 32 megabytes of RAM SVGA monitor, displaying 16-bit color, 800 x 600 reso lution Microsoft or Microsoft-compatible mouse or other pointin g device CD-ROM Drive Access to the internet is optional but will enable you t o acquire additional relevant information. COURSE OVERVIEW Exercise Design Page 7 How to Take This Course (Continued) Sample Learning Schedule Complete this course at your own pace. You should be ab le to finish the entire course including pretest, units, knowledge checks, and final examination in approximately 10 hours. The following learning schedule is only an example, intended to show relative times devoted to each unit.

Unit Suggested Time Unit 1: Introduction to Exercise Design Unit 2: Comprehensive Exercise Program Unit 3: The Exercise Process Unit 4: Exercise Design Steps Unit 5: The Tabletop Exercise Unit 6: The Functional Exercise Unit 7: The Full-Scale Exercise Unit 8: Exercise Evaluation Unit 9: Exercise Enhancements Unit 10: Designing a Functional Exercise Unit 11: Course Summary 20 minutes 1 hour 1 hour 2 hours 40 minutes 40 minutes 40 minutes 40 minutes 40 minutes 2 hours 10 minutes Unit 1: Introduction to Exercise Design UNIT 1 INTRODUCTION TO E XERCISE DESIGN Exercise Design Page 1.1 Introduction An exercise is a focused practice activity that places the participants in a simulated situation requiring them to function in the capacity that would be expected of them in a real event . Its purpose is to promote preparedness by testing policies and plans and training p ersonnel. In this unit, you will explore some of the benefits that organizations derive from exercising.

Unit 1 Objectives After completing this unit, you should be able to: Explain the value of exercises to improve the four pha ses of comprehensive emergency management. Identify the major reasons to conduct exercises.

Why Exercise?

Exercises are conducted to evaluate an organization’s capa bility to execute one or more portions of its response plan or cont ingency plan. Many successful responses to emergencies over the years have demonstrated that exercising pays huge dividends when an emergency occurs.

UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.2 Exercise Design Why Exercise? (Continued) Sioux City, 1989 The 1989 plane crash in Sioux City, Iowa, provided a clear demonstration of the value of exercises. In July of that year, Unite d Airlines Flight 232 crashed in flames after a failed emergency landing atte mpt. Although 109 lives were lost in this terrible disaster, 186 passeng ers survived. How were so many saved? Their survival was due mainly to t hree factors:

Response of the flight crew before the crash Trained rescue units waiting on the ground Centralized communications among all response agencies These factors were present because of training, and the h igh level of training was no coincidence. Years before the crash, a Di saster Services Center was established. Representatives from 40 local ag encies met regularly to review emergency procedures and plan real istic exercises. Two years before the crash, the community conducted a ful l-scale exercise based on a commercial plane crash. This simulation revealed several problems, including confusion in communications and inadequate numbers of ambulances and other equipment at the scene. An after- exercise plan was developed to address these problems. The day after the crash of United Flight 232, the Assist ant Fire Chief said, “We made mistakes [in the simulation]. The mistakes we made then did not materialize yesterday.” UNIT 1: INTRODUCTION TO EXERCISE DESIGN Exercise Design Page 1.3 Why Exercise? (Continued) Other Emergencies Other emergencies and disasters before and since the 1989 crash also have demonstrated the importance of exercising. In 1989, FEMA coordinated a full-scale response exercise i n Sacramento, California just two months before the Loma Prieta earthquake struck Northern California. The exercise wa s credited with improving the response to that disaster by both Ca lifornia and the Federal government.

The After-Action Report following the 1995 bombing o f the Alfred P.

Murrah Federal Building in Oklahoma City made the f ollowing recommendation regarding problems that were identifie d in integrating the response efforts of multiple agencies:

“Planning, training, and exercising are the only feasi ble recommenda- tions. If an integrated emergency management system is to be utilized and effective in future disasters, all levels o f government must be on the same page of the book. Effective coordinati on cannot be achieved during the chaos following any disaster. Relat ionships must be established, plans written and tested, and procedures agreed upon. Regardless of what our particular role is, our m ission is the same… ‘to provide effective response and recovery through coordinated logistics, communications, and information supp ort systems.’” UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.4 Exercise Design Why Exercise? (Continued) In 2000, Urban Search and Rescue (US&R) Task Forces parti cipated in two major exercises. One was an earthquake-based scena rio staged at Ames/Moffett Airfield in California. The o ther took advantage of the planned demolition of a sports arena in Denver, Colorado, by running a building collapse scenario. Of the latter exercise, a Denver Fire Department captain said, “We’re preparing for the event we hope never happens.” Ironically, that was exactly the case. In 2001, some of the same US&R Task Forces that participated in these exercises were sent to New York City to search for victims after the terrorist attack that resulted in the collapse of the World Trade Center tow ers.

Many communities across the nation have had similar expe riences that show the value of previous exercise training. Research h as shown that people generally respond to an emergency in the way t hat they have trained. It only makes sense for government, voluntee r, and private organizations to exercise their plans and procedures so th at they are better prepared to respond to and recover from an eme rgency.

UNIT 1: INTRODUCTION TO EXERCISE DESIGN Exercise Design Page 1.5 Why Exercise? (Continued) Reasons to Exercise The goal in exercise design is to establish a comprehensiv e exercise program one based on a long-term, carefully constructed plan. I n a comprehensive program, exercises build upon one another to meet specific operational goals. The aim is to provide compet ence in all emergency functions. There are two main benefits of an exercise program: Individual training: Exercising enables people to practice their roles and gain experience in those roles. System improvement: Exercising improves the organization’s system for managing emergencies.

These benefits arise not just from exercising, but from e valuating the exercise and acting upon the recommendations. An exercise has value only when it leads to improvement. To summarize, there are a number of reasons to perfor m exercises. Through exercises, organizations can: Test and evaluate plans, policies, and procedures.

Reveal planning weaknesses.

Reveal gaps in resources.

Improve organizational coordination and communications. Clarify roles and responsibilities.

Train personnel in roles and responsibilities.

Improve individual performance.

Gain program recognition and support of officials.

Satisfy regulatory requirements.

The focus of an exercise should always be on locating and eliminating problems before an actual emergency occurs. Corrective actions are an important part of exercise design, evaluation, and fol lowup.

UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.6 Exercise Design Regulatory Requirements Because the human and monetary costs of emergencies and d isasters are so high, governments, agencies, and many corporate e ntities and other governing bodies have mandated preparedness trai ning and exercising. Several examples of mandated exercises are l isted below.

State and local governments receiving Federal funds may have to comply with certain exercise requirements. FEMA’s requir ements change periodically, but the program is normally structu red around a four-year cycle. Nuclear Power Plants must exercise their plans yearly, con ducting a full-scale exercise every two years. This exercise is evalu ated by the Nuclear Regulatory Commission (NRC). Agencies or facilities falling under the coverage of the Superfund Amendment Reauthorization Act of 1986 (SARA) Title I II (e.g., local emergency planning committees; facilities where chemicals a re produced, used, or stored) must conduct a yearly exercise a nd evaluate their hazardous materials response and recovery plan.

Airports, hospitals, and other healthcare facilities must conduct a full- scale exercise once every 2 years to maintain their certif ication or license to operate. Many employers are required by the Occupational Safety and Health Administration (OSHA) to develop an emergency action p lan. OSHA recommends that such plans be exercised at least annually. UNIT 1: INTRODUCTION TO EXERCISE DESIGN Exercise Design Page 1.7 Functions In planning exercises, the emphasis is on functions rather than on types of emergencies, because preparedness in those functions is com mon to all emergencies. Functions are actions or operations required in emergency response or recovery .

FEMA defines 13 functions in its Emergency Management Ex ercise Reporting System: Alert Notification (Emergency Response) Warning (Public) Communications Coordination and Control Emergency Public Information Damage Assessment Health and Medical Individual/Family Assistance Public Safety Public Works/Engineering Transportation Resource Management Continuity of Government UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.8 Exercise Design Functions (Continued) If your organization is a private or volunteer orga nization, you may find it practical to exercise a somewhat different or more limited set of functions. Each of the functions listed above has a set of subfunction s related to it, and your group may focus on some of those. For example , your emergency response focus may relate to efforts such as: The management and distribution of donations.

The logistics of providing needed resources.

The temporary conversion of a manufacturing process to pr ovide emergency supplies.

How to coordinate with other organizations to provide mass care.

How your employees respond to an internal emergency. The variations are, of course, endless. However your en tity is organized, the point is that your exercise program should identify the applicable functions and emphasize testing the operational procedur es within those functions regardless of the type of emergency.

UNIT 1: INTRODUCTION TO EXERCISE DESIGN Exercise Design Page 1.9 Activity Activity: Needs Assessment Below is a job aid to help you assess your organization’s e xercise needs. Use this tool to analyze where you may wish to focus your or ganization’s exercise design efforts. In completing this needs assessment, you may wish to consult such resources as planning documents, demograp hic or corporate data, maps, and training records. ( Note: This assessment form also appears as Job Aid 1 in Appendix A.) 1. Hazards List the various hazards in your community or organizat ion. What risks are you most likely to face? You can use the following checklist as a starting point. Note: If your community has already conducted a hazard analysis, that is the best resource.

Airplane crash Sustained power failure Dam failure Terrorism Drought Tornado Epidemic (biological attack) Train derailment Earthquake Tsunami Fire/Firestorm Volcanic eruption Flood Wildfire Hazardous material spill/release Winter storm Hostage/Shooting Workplace violence Hurricane Other ______________________ Landslide/Mudslide Other ______________________ Mass fatality incident Other ______________________ Radiological release Other ______________________ UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.10 Exercise Design Activity: Needs Assessment (Continued) 2. Secondary Hazards What secondary effects from those hazards are likely to i mpact your organization?

Communication system breakdown Power outages Transportation blockages Business interruptions Mass evacuations/displaced population Overwhelmed medical/mortuary services Other ________________________ Other ________________________ Other ________________________ Other ________________________ Other ________________________ 3. Hazard Priority What are the highest priority hazards? Consider such fa ctors as:

Frequency of occurrence.

Relative likelihood of occurrence.

Magnitude and intensity.

Location (affecting critical areas or infrastructure).

Spatial extent.

Speed of onset and availability of warning.

Potential severity of consequences to people, critical faci lities, community functions, and property.

Potential cascading events (e.g., damage to chemical proce ssing plant, dam failure).

#1 Priority hazard: #2 Priority hazard: #3 Priority hazard:

UNIT 1: INTRODUCTION TO EXERCISE DESIGN Exercise Design Page 1.11 Activity: Needs Assessment (Continued) 4. Area What geographic area(s) or facility location(s) is(are) m ost vulnerable to the high priority hazards? 5. Plans and Procedures What plans and procedures emergency response plan, contingency plan, operational plan, standard operating procedures (SOPs) will guide your organization’s response to an emergency? 6. Functions What emergency management functions are most in need of rehearsal? (e.g., What functions have not been exercised recently? Where have difficulties occurred in the past?) You can use the following checklist as a starting p oint.

Alert Notification (Emergency Response) Public Safety Warning (Public) Public Works/Engineering Communications Transportation Coordination and Control Resource Management Emergency Public Information (EPI) Continuity of Government or Operations Damage Assessment Other ___________________________ Health and Medical Other ___________________________ Individual/Family Assistance Other ___________________________ UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.12 Exercise Design Activity: Needs Assessment (Continued) 7. Participants Who (agencies, departments, operational units, personnel ) needs to participate in an exercise? For example: Have any entities updated their plans and procedures? Have any changed policies or staff? Who is designated for emergency management responsibilit y in your plans and procedures? With whom does your organization need to coordinate i n an emergency? What do your regulatory requirements call for?

What personnel can you reasonably expect to devote to d eveloping an exercise?

8. Program Areas Mark the status of your emergency program in these and other areas to identify those most in need of exercising. New Updated Exercised Used in Emergency N/A Emergency Plan Plan Annex(es) Standard Operating Procedures Resource List Maps, Displays Reporting Requirements Notification Procedures Mutual Aid Pacts Policy-Making Officials Coordinating Personnel Operations Staff Volunteer Organizations EOC/Command Center Communication Facility UNIT 1: INTRODUCTION TO EXERCISE DESIGN Exercise Design Page 1.13 Activity: Needs Assessment (Continued) New Updated Exercised Used in Emergency N/A W arning Systems Utility Emergency Preparedness Industrial Emergency Preparedness Damage Assessment Techniques Other:

9. Past Exercises If your organization has participated in exercises befor e, what did you learn from them, and what do the results indicate about future exercise n eeds? For example, consider the following questions:

Who participated in the exercise, and who did not?

To what extent were the exercise objectives achieved?

What lessons were learned?

What problems were revealed, and what is needed to re solve them?

What improvements were made following past exercises, an d have they been tested?

UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.14 Exercise Design Summary and Transition This unit provided an overview of the benefits organ izations derive from exercising. Unit 2 discusses the elements of a comprehensiv e exercise program and the progressive activities that make up a co mprehensive program. Reference Library For More Information FEMA Preparedness, Training, and Exercises, Exercise FAQs: www.fema.gov/pte/faq.htm .

Environmental Protection Agency’s (EPA) Preparedness for Response Exercise Program: www.epa.gov/superfund/programs/er/training/prep/prepe xer.htm .

U.S. Department of Justice (DOJ), Office for Domestic Prep aredness, Exercises: www.ojp.usdoj.gov/odp/exercises/state.htm .

Oak Ridge Associated Universities (in conjunction with U.S .

Department of Energy (DOE)), Emergency Management La boratory, What Would You Do If It Happened to You?: www.orau.gov/eml/exercise.htm .

Occupational Safety and Health Administration, Emergen cy Preparedness and Response: www.osha.gov/SLTC/smallbusiness/sec10.html .

UNIT 1: INTRODUCTION TO EXERCISE DESIGN Exercise Design Page 1.15 Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer you have chosen. 1. Research has shown that: a. Exercises are the best way to teach employees new skill s.

b. Extended lectures are an effective alternative to e xercises.

c. When possible, it is more effective to use real (rathe r than simulated) emergencies to test response procedures. d. People usually respond to an emergency in the way t hat they have trained.

2. Through exercising, organizations can reveal plannin g weaknesses and gaps in resources.

a. True b. False 3. A good reason to exercise is to reduce the need for o rganizational coordination and communication.

a. True b. False 4. The focus of an exercise should be on eliminating pro blems before an actual emergency occurs. a. True b. False 5. An exercise has value only when it leads to improvem ent.

a. True b. False 6. In planning exercises, the main emphasis should be on the organization’s:

a. Vulnerability to natural hazards.

b. Vulnerability to manmade hazards (e.g., terrorism) .

c. Ability to effectively carry out response and recovery functions.

d. Ability to respond to specific types of emergencies. UNIT 1: INTRODUCTION TO EXERCISE DESIGN Page 1.16 Exercise Design Answers Knowledge Check (Continued) 1. d 2. a 3. b 4. a 5. a 6. c Unit 2: Comprehensive Exercise Program UNIT 2 COMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.1 Introduction In any discussion of emergency preparedness, the emphasis is on a comprehensive exercise program, made up of progressively complex exercises, each one building on the previous one, until the exercises are as close to reality as possible. This unit provides an ove rview of five main types of exercise activities that make up a comprehe nsive exercise program. Unit 2 Objectives After completing this unit, you should be able to: Identify the basic components of a comprehensive exercise p rogram. Explain the importance of designing a comprehensive and progressive exercise program to meet the needs of your o rganization or community.

Progressive Exercising A progressive program has several important characteristi cs:

The exercise program involves the efforts and participat ion of various entities whether departments, organizations, or agencies. Throu gh the involvement of multiple entities, the program al lows the involved organizations to test, not only their implementation of emergency management procedures, but their coordination with each other in the process.

The program is carefully planned to achieve identified goals.

It is made up of a series of increasingly complex exercises. Each successive exercise builds upon the previous one until m astery is achieved. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.2 Exercise Design Progressive Exercising (Continued) Broad Commitment When a community or an organization engages in a prog ressive exercise program, the program needs to be comprehensive. A comm unity program must consider every type of responding agency an d organization in the community. Communities are composed of more than police, fire, and public works. The following entities have requirements to exercise as well:

Hospitals Airports, chemical, and nuclear facilities, and other reg ulated organizations Volunteer agencies and organizations in the private sect or that contribute services, materials, and personnel to the respo nse and recovery effort Work sites public or private that may be vulnerable to significant emergency events A progressive exercise program, therefore, requires a com mitment from various agencies and organizations to participate in incr easingly challenging exercises over a period of time, to address t he larger emergency management system rather than a single proble m.

The same is true when a single organization engages in a progressive exercise program. It must consider the role of each depa rtment and each function that will be involved in responding to and r ecovering from an emergency event, and it must secure the commitment of al l of those elements to a sequence of progressive internal and exter nal exercises that will build a coordinated, effective response. Careful Planning Exercises require careful planning around clearly identi fied goals. Only through identifying exercise goals, then designing, dev eloping, conducting, and analyzing the results can those who are r esponsible for emergency operations be sure of what works—and what does not.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.3 Progressive Exercising (Continued) Increasing Complexity Exercises should be organized to increase in complexity for example, from tabletop discussions to functional exercises to a full- scale exercise. Each type of exercise builds on previous exercises using mor e sophisticated simulation techniques and requiring more pr eparation time, personnel, and planning. Rushing into a full-scale exercise too quickly can open th e door to potential failure because shortfalls have not been iden tified through less complicated and less expensive exercises. Success Breeds Success An important advantage of building incrementally to a full-scale exercise is that successful exercise experiences breed new successes:

Officials and stakeholders are more willing to commit re sources.

Personnel are more motivated and look forward to the next exercise.

Confidence increases.

Operating skills improve.

Who Participates?

For a community-wide exercise program, the jurisdiction determines what agencies, organizations, and stakeholders participate in each exercise. Participants are further determined by the nature and size of the exercise. Larger exercises would include all of the participants wh o would have responsibilities in a real emergency. Smaller exercises, which focus on a limited aspect of the emergency plan, would limit the number of participants. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.4 Exercise Design Who Participates? (Continued) The same is true of exercises conducted by a particular or ganization. For example, let’s look at the case of a beer or soft drink bottler that during disasters converts its production line over to bottling wa ter for mass care. This organization might design exercises to test procedure s for:

Coordination with jurisdiction officials.

Managerial decision making on when to convert, how much of the line to convert, and when to convert back.

Internal notifications.

Line personnel responsibilities.

Temporary facility changes.

Coordination with suppliers.

Product distribution.

Transitioning back to regular production.

Documentation.

In either case whether the exercises involve an entire community or a more limited population the nature of the exercise somewhat determines the participants. On one end of the spectrum, a tablet op exercise might involve only key decision makers. An exercise to test pa rticular functions would limit its participants to those functions. And a f ull-scale exercise might involve the entire community or the whole facil ity.

Some types of exercises have additional participant req uirements. For example, a functional exercise involves not only player s but simulators, controllers, and evaluators. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.5 What Activities Are Included?

There are five main types of activities in a comprehen sive exercise program: Orientation seminar Drill Tabletop exercise Functional exercise Full-scale exercise As we have discussed, these activities build from simple to complex, from narrow to broad, from least expensive to most costly to implement, from theoretical to realistic. When carefully planned to ach ieve specified objectives and goals, this progression of exercise activitie s provides an important element of an integrated emergency prepare dness system.

Orientation Seminar Purpose As the name suggests, the orientation seminar is an overview or introduction. Its purpose is to familiarize participant s with roles, plans, procedures, or equipment. It can also be used to resolve questions of coordination and assignment of responsibilities. Characteristics Key characteristics of the orientation seminar are summar ized in the table on the next page. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.6 Exercise Design What Activities Are Included? (Continued) Orientation Seminar Characteristics Format The orientation seminar is a very low-stress event, usually presented as an informal discussion in a group setting. There is little or no simulation. (For this reason, orientations do not qualify as FEMA-recognized exer cises.) A variety of seminar formats can be used, including: Lecture.

Discussion.

Slide or video presentation.

Computer demonstration.

Panel discussion.

Guest lecturers.

Applications The orientation seminar can be used for a wide vari ety of purposes, including:

Discussing a topic or problem in a group setting.

Introducing something new (e.g., policies and plans ).

Explaining existing plans to new people (e.g., newl y elected officials or executives need an explanation of the EOP and their role at th e EOC; new employees need an orientation to operational plans as they relate to emergencies).

Introducing a cycle of exercises or preparing parti cipants for success in more complex exercises.

Motivating people for participation in subsequent e xercises.

Leadership Orientations are led by a facilitator, who presents information and guides discussion. The facilitator should have some leadership skills, but very little other training is required.

Participants A seminar may be cross-functional involving one or two participants for each function or service being discussed (e.g., management, polic y, coordination, and operations staff). Or, it may be geared to several people fro m a single agency or department.

Facilities A conference room or any other fixed facility may b e used, depending on the purposes of the orientation. Time Orientations should last a maximum of 1 to 2 hours.

Preparation An orientation is quite simple to prepare (two week s’ preparation time is usually sufficient) and conduct. Participants need no prev ious training.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.7 What Activities Are Included? (Continued) Conducting an Orientation There are no cut-and-dried rules for an effective ori entation; its purpose will determine its format. Below are a few helpful guidelines for conducting a seminar. Orientation Seminar Guidelines Be creative. You can use various discussion and presentation methods. Think of interesting classes that you have atte nded in other subjects, and borrow the techniques of good teacher s and presenters. For example, you might call on people one by one to give ideas, plan a panel discussion, hold a brainstorming session, present case studies for problem solving, or give an illu strated lecture.

Get organized and plan ahead. Even though orientation seminars are less complex than other activities, it is no time to “wing it.” Be ready to facilitate a successful orientation sem inar. Discourage long tirades, keep exchanges crisp and to the p oint, focus on the subject at hand, and help everyone feel go od about being there. Drills Purpose A drill is a coordinated, supervised exercise activity, normally used to test a single specific operation or function . With a drill, there is no attempt to coordinate organizations or fully activate the EOC. I ts role in an exercise program is to practice and perfect one small part of the response plan and help prepare for more extensive exercises, in which several functions will be coordinated and tested. The effectiveness of a drill is its focus on a single, relatively limited portion of the overall emergency management system. It makes possible a tight focus on a potential pr oblem area. Characteristics Key characteristics of drills are summarized in the table on the next page.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.8 Exercise Design What Activities Are Included? (Continued) Drill Characteristics Format A drill involves actual field or facility response for an EOC operation. It should be as realistic as possible, employing any equipment or a pparatus for the function being drilled. Applications Drills are used to test a specific operation. They are also used to provide training with new equipment, to develop new policies or procedure s, or to practice and maintain current skills. Drills are a routine part of the d aily job and organizational training in the field, in a facility, or at the EOC. Some examples of drills run by different organizations are listed below: EOC: Call down procedures Public works: Locating and placing road barriers u nder time constraints Public health and safety: Site assessment and samp ling Red Cross: Locating specific types of blood within a time constraint Military: Activation and mobilization drill Airport: Fire Department response to the furthest part of a runway within a given time Chemical plant: Evacuation and isolation of spill area and valve system shutoff Private sector resource provider: W arehouse readin ess drill Leadership A drill can be led by a manager, supervisor, depart ment head, or exercise designer. Staff must have a good understanding of the single function being tested.

Participants The number of participants depends on the function being tested. Coordination, operations, and response personnel could be include d.

Facilities Drills can be conducted within a facility, in the f ield, or at the EOC or other operating center. Time ½ to 2 hours is usually required. Preparation Drills are one of the easiest kinds of exercise act ivities to design. Preparation may take about a month. Participants usually need a short o rientation beforehand.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.9 What Activities Are Included? (Continued) Conducting a Drill How a drill is conducted varies according to the type of drill ranging from simple operational procedures to more elaborate communi cation and command post drills. For example, a command post drill would require participants to report to the drill site, where a “visu al narrative” would be displayed in the form of a mock emergency. Equipment, such as vans, command boards, and other needed supplies would be made available.

Given the variety of functions that may be drilled, t here is no set way to run a drill. However, some general guidelines are gi ven below.

Drill Guidelines Prepare. If operational procedures are to be tested, review t hem beforehand. Review safety precautions.

Set the stage. It’s always good to begin with a general briefing, which sets the scene and reviews the drill purpose and obj ectives. Some designers like to set the scene using films, slides, or videotapes.

Monitor the action. After a drill has been started, it will usually continue under its own steam. If you find that somethi ng you wanted to happen is not happening, however, you might want to insert a message to trigger that action. (We’ll learn more abo ut messages later on.) UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.10 Exercise Design What Activities Are Included? (Continued) Tabletop Exercise Purpose A tabletop exercise is a facilitated analysis of an emergency situation in an informal, stress-free environment . It is designed to elicit constructive discussion as participants examine and resolve problems based on existing operational plans and identify where those pl ans need to be refined. The success of the exercise is largely determined by group participation in the identification of problem areas. Characteristics There is minimal attempt at simulation in a tabletop exercise. Equipment is not used, resources are not deployed, and time pressure s are not introduced. Key characteristics of the tabletop exercise a re summarized in the table on the next page. We will discuss how to conduct a tabletop exercise in Unit 5.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.11 What Activities Are Included? (Continued) Tabletop Exercise Characteristics Format The exercise begins with the reading of a short nar rative, which sets the stage for the hypothetical emergency. Then, the facilitator may stimulate discussion in two ways: Problem statements: Problem statements (describing major or detailed events) may be addressed either to individual participants or to participating departments or agencies. Recipients of problem statements then di scuss the actions they might take in response.

Simulated messages: These messages are more specif ic than problem statements. Again, the recipients discuss their responses.

In either case, the discussion generated by the pro blem focuses on roles (how the participants would respond in a real emergency), pl ans, coordination, the effect of decisions on other organizations, and similar conce rns. Often maps, charts, and packets of materials are used to add to the realism of the exercise.

Applications Tabletop exercises have several important applicati ons. They:

Lend themselves to low-stress discussion of coordin ation and policy. Provide a good environment for problem solving. Provide an opportunity for key agencies and stakeho lders to become acquainted with one another, their interrelated roles, and the ir respective responsibilities.

Provide good preparation for a functional exercise. Leadership A facilitator leads the tabletop discussion. This person decides who gets a message or problem statement, calls on others to participate, asks questions, and guides the participants toward sound decisions. Participants The objectives of the exercise dictate who should p articipate. The exercise can involve many people and many organizations essentially anyone who can learn from or contribute to the planned discussion items. This m ay include all entities that have a policy, planning, or response role. Facilities A tabletop exercise requires a large conference roo m where participants can surround a table. Time A tabletop exercise usually lasts from 1 to 4 hours but can be longer. Discussion times are open-ended, and participants are encouraged to take their time in arriving at in-depth decisions without time pressure. W hen the time is up, the a ctivity is concluded. Although the facilitator maintains an awareness of time allocation for each area of discussion, the group does not have to complete eve ry item for the exercise to be a success. Preparation It typically takes about a month to prepare for a t abletop exercise. Preparation also usually requires at least one orientation and somet imes one or more drills.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.12 Exercise Design What Activities Are Included? (Continued) Functional Exercises Purpose A functional exercise is a fully simulated interactive exercise that tests the capability of an organization to respond to a simul ated event. The exercise tests multiple functions of the organization’s op erational plan. It is a coordinated response to a situation in a time-pressur ed, realistic simulation. Characteristics A functional exercise focuses on the coordination, integr ation, and interaction of an organization’s policies, procedures, ro les, and responsibilities before, during, or after the simulated event. Key characteristics of a functional exercise are summarized in t he following table. We will discuss how to conduct a functional exercise in Unit 6.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.13 What Activities Are Included? (Continued) Functional Exercise Characteristics Format This is an interactive exercise—similar to a full-scale exercise without t he equipment. It simulates an incident in the most realistic manner possible short of moving resources to an actual site. A functional exercise is: Geared for policy, coordination, and operations per sonnel the “players” in the exercise—who practice responding in a realistic way to carefully planned and sequenced messages given to them by “simulators.” The messages reflect ongoing events and problems that might actually occur in a real emergency.

A stressful exercise because players respond in real time, with on-the-spot decisions and actions. All of the participants’ decisions an d actions generate real responses and consequences from other players. Complex— Messages must be carefully scripted to cause partic ipants to make decisions and act on them. This complexity makes t he functional exercise difficult to design. Applications Functional exercises make it possible to test sever al functions and exercise several agencies or departments without incurring the cost of a full-scale exercise. A functional exercise is always a prerequisite to a full-scale e xercise. In some instances, taking part in a functional exer cise may serve as a full-scale exercise for a participating organization (e.g., a hospital may conduct its own full-scale exercise as part of a community-based functional exercise). Leadership and Participants Functional exercises are complex in their organizat ion of leadership and the assignment of roles. The following general roles are used: Controller: Manages and directs the exercise Players: Participants who respond as they would in a real em ergency (Players should include policy makers; may include coordinat ors and operational personnel directing field activities.) Simulators: Assume external roles and deliver planned message s to the players Evaluators: Observers who assess performance Facilities It is usually conducted in the EOC or other operati ng center. Ideally, people gather where they would actually operate in an emergency. Players and simulators are often seated in separate areas or rooms. Realism is achi eved by the use of telephones, radios, televisions, and maps. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.14 Exercise Design What Activities Are Included? (Continued) Functional Exercise Characteristics (Continued) Time A functional exercise requires from 3 to 8 hours, a lthough it can run a full day or even longer. Preparation Plan on 6 to 18 months or more to prepare for a fun ctional exercise, for several reasons: Staff members need considerable experience with the functions being tested. The exercise should be preceded by lower-level exer cises, as needed.

The controller, evaluators, and simulators require training.

The exercise may require a significant allocation o f resources and a major commitment from organizational leaders.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.15 What Activities Are Included? (Continued) Full-Scale Exercise Purpose A full-scale exercise simulates a real event as closely as possible . It is an exercise designed to evaluate the operational capabi lity of emergency management systems in a highly stressful environment that simulates actual response conditions. To accomplish this realism, it r equires the mobilization and actual movement of emergency personne l, equipment, and resources. Ideally, the full-scale exercise should test and evaluate most functions of the emergency management plan or oper ational plan.

Characteristics A full-scale exercise differs from a drill in that it co ordinates the actions of several entities, tests several emergency functions, and act ivates the EOC or other operating center. Realism is achieved thr ough:

On-scene actions and decisions.

Simulated “victims.” Search and rescue requirements.

Communication devices.

Equipment deployment.

Actual resource and personnel allocation.

Key characteristics of full-scale exercises are summarized in the table on the next page. We will discuss how to conduct a full-scale exercise in Unit 7.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.16 Exercise Design What Activities Are Included? (Continued) Full-Scale Exercise Characteristics Format The exercise begins with a description of the event , communicated to responders in the same manner as would occur in a real event. Person nel conducting the field component must proceed to their assigned locations, where the y see a “visual narrative” in the form of a mock emergency (e.g., a plane crash with victi ms, a “burning” building, a simulated chemical spill on a highway, or a terrorist attack) . From then on, actions taken at the scene serve as input to the simulation taking place at the EOC or operating center.

Applications Full-scale exercises are the ultimate in the testin g of functions the “trial by fire.” Because they are expensive and time consuming, it i s important that they be reserved for the highest priority hazards and functions. Leadership and Participants One or more controllers manage the exercise, and ev aluators are required. All levels of personnel take part in a full-scale exercise: Policy personnel Coordination personnel Operations personnel Field personnel Facilities The event unfolds in a realistic setting (e.g., a p lane crash at an airport, a terrorist attack at a public venue). The EOC or other operating cen ter is activated, and field command posts may be established. Time A full-scale exercise may be designed to be as shor t as 2 to 4 hours, or to last as long as 1 or more days. Preparation Preparation for a full-scale exercise requires an e xtensive investment of time, effort, and resources—1 to 1½ years to develop a complete exerci se package. This timeframe includes multiple drills and preparatory tabletop a nd functional exercises. In addition, personnel and equipment from participating agencies must be committed for a prolonged period of time. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.17 Comparing the Five Activities Each of the five activities just described plays an impor tant part in the overall exercise program. The following chart lists some of reasons for conducting each type of activity. Key characteristics of ea ch type of exercise are shown in the table on the next page. Reasons to Conduct Exercise Program Activities Orientation Drill Tabletop Exercise Functional Exercise Full-Scale Exercise No previous exercise Assess equipment capabilities Practice group problem solving Evaluate a function Assess and improve information analysis No recent operations Test response time Promote executive familiarity with emergency management plan Observe physical facilities use Assess and improve interagency cooperation New plan Personnel training Assess plan coverage for a specific case study Reinforce established policies and procedures Support policy formulation New procedures Assess interagency cooperation Assess plan coverage for a specific risk area Assess hospital preparedness Assess negotiation procedures New staff, leadership Verify resource and staffing capabilities Examine staffing contingencies Test seldom-used resources Test resource and personnel allocation New nuclear facility Test group message interpretation Measure resource adequacy Direct media attention New industrial risk Assess interagency or interdepartmental coordination Assess and strengthen inter- jurisdictional or interorganizational relations Assess and strengthen interjurisdictional or interorganizational relations Observe information sharing Assess personnel and equipment locations Train personnel in negotiation Test equipment capabilities UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.18 Exercise Design Comparing the Five Activities (Continued) Key Characteristics The following table briefly compares the key character istics of the five types of exercise program activities. Comparison of Key Activity Characteristics Orientation Drill Tabletop Exercise Functional Exercise Full-Scale Exercise Format Informal discussion in group setting Various presentation methods Actual field or facility response Actual equipment Narrative presentation Problem statements or simulated messages Group discussion No time pressures Interactive, complex Players respond to messages (events/problems) provided by simulators. Realistic but no actual equipment. Conducted in real time; stressful Realistic event announcement Personnel gather at assigned site Visual narrative (enactment) Actions at scene serve as input to EOC simulation Leaders Facilitator Manager, supervisor, department head, or designer Facilitator Controller Controller(s) Participants Single agency/ department, or cross- functional Personnel for the function being tested May include coordination, operations, response personnel Anyone with a policy, planning, or response role for the type of situation used Players (policy, coordination, and operations personnel) Simulators Evaluators All levels of personnel (policy, coordination, operations, field) Evaluators Facilities Conference room Facility, field, or EOC Large conference room EOC or other operating center (multiple rooms) Realistic setting EOC or other operating center Time 1–2 hours ½–2 hours 1–4 hours or longer 3–8 hours or longer 2 hours to 1 or more days UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.19 Comparing the Five Activities (Continued) Comparison of Key Activity Characteristics (Continu ed) Orientation Drill Tabletop Exercise Functional Exercise Full-Scale Exercise Preparation Simple preparation, 2 weeks Easy to design, 1 month Participants need orientation 1 month preparation Preceded by orientation and 1 or more drills Complex, 6–18 months preparation Preceded by simpler exercises Significant allocation of resources Extensive time, effort, resources 1–1½ years development Including preparatory drills, tabletops, functional exercises UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.20 Exercise Design Building an Exercise Program As you have seen, a progressive exercise program involve s the combined efforts of many agencies, departments, or other entitie s in a series of activities that increase in complexity until mastery is ach ieved.

Building an exercise program is a little like planning a single exercise except that the activities take place on a much larger sca le. Plans are developed by a team and are based on a caref ul examination of the operating plan. The development of an exercise program has many facets, including:

Analysis of capabilities and costs.

Scheduling of tasks.

Public relations efforts.

Development of a long-term plan.

Careful work on the long-term plan will carry over i nto the design of individual exercises. The Planning Team A comprehensive exercise plan requires the combined effo rts of many people. For a community program, the team should consi st of representatives from every major government agency in the jurisdiction and from private and volunteer organizations large e nough to have exercise mandates: Fire department Sheriff Public works Hospital Airport Schools Communications Volunteer organizations UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.21 Building an Exercise Program (Continued) In a private or volunteer organization that is plan ning an exercise program, the team would be similarly organized, with representatives of all major functions and departments. The emergency manager and other emergency personnel ( or the person responsible for the organization’s emergency response eff ort) would take the lead, and the representatives would then meet to analyze what they need to do to support one another. Often organizati ons can meet the exercise needs of more than one agency at a time. This teamwork can help establish important relationships among participati ng organizations.

Later, members of the team can also serve on exercise de sign teams to design individual exercises. Goal Setting Because a comprehensive exercise program usually extends o ver several months, it is important to set long-term goals or devel op a mission statement. Without this, the program is likely to lack focus and continuity.

Schedule and Sequence When these preliminary steps (organizing the team and establishing a mission statement and goals) have been taken, the hard work of drawing up a plan can take place. Developing the exercise prog ram plan involves:

1. Laying out a series of exercises that can meet the nee ds of the various participating entities. 2. Organizing them into a workable sequence and time schedule.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.22 Exercise Design Building an Exercise Program (Continued) Plan Format An exercise program plan can use any format, but it sho uld include the following elements: A timeframe A problem statement Long-range goal(s) Functional objectives A schedule Exercise descriptions, including:

Type of exercise.

Participants.

Purpose.

Rationale.

A sample plan format is shown on the next page. This i s a hypothetical example of one community’s exercise plan. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.23 Building an Exercise Program (Continued) Plan Format Sample Plan: Comprehensive Exercise Pr ogram Note: During the previous year, several tabletops and functional exercises were held to test weaknesses in Communications, Alert an d Notification, and Individual/Family Assistance. The series of exerci ses might take less time in some communities. Timeframe The exercise program extends over an 18-month perio d.

Present Problems This program has been formulated to address problem s arising as a result of rapid population growth. According to experts, pos sibilities for a mass casualty incident are increasing. Personnel involved in the functional areas listed below have not been tested in the last year. Long-Range Goal To work toward a full-scale exercise testing all im portant functions in the context of a mass casualty incident. This will satisfy FEM A requirements and full-scale exercise requirements for the hospital and airport, by involving these agencies.

Functions to be Tested Health and Medical, Public Information, Coordinatio n and Control (EOC Operations, Incident Command) To determine the adequacy of plans and procedures w ithin the following functional areas to handle a mass casualty incident : Health and Medical, Public Information, Coordination and Control (EOC O perations, Incident command).

To test the ability of the above-named functional a reas to communicate and coordinate their response efforts during a mass cas ualty incident.

To test the ability to obtain adequate resources (l ocally and through mutual aid agreements) in the above-named functional areas to handle a mass casualty incident.

First Month Exercise: Orientation For: Emergency Management staff and heads of variou s agencies: Mental Health Association, State Funeral Director, County Coroner, County Fire, County Police Purpose: To review new plans and procedures for dea ling with mass casualty incidents. Rationale: Inform those who are unaware of plans an d gain support and additional input from department leaders. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.24 Exercise Design Building an Exercise Program (Continued) Plan Format Sample Plan: Comprehensive Exercise Pr ogram Second Month Exercise: Orientation For: Emergency management staff and heads of variou s agencies: fire, police staff, county Public Information Offic er (PIO) Purpose: To review new plans for mass casualty inc idents with responders. Rationale: Gain support and additional input from first responders and acquaint them with leadership’s plans. Fourth Month Exercise: Training course with functional exercise.

For: Responders and incident commanders; Emergency management staff; various chiefs, captains, lieuten ants from fire and police; Emergency Medical Services (EMS), menta l health, Radio Amateur Civil Emergency Services (RACES), fun eral directors, county coroner, county PIO Purpose: To provide training in field mass casualty incident response.

Rationale: This is a training session in the FEMA F ield Mass Casualty Incident Response course. This course provides an excellent overview of specific needs related to a mass fatali ty incident. The course culminates in a functional activity. Seventh Month Exercise: Drill For: Fire, police, EMS, coroner, funeral directors Purpose: To set up the Incident Command System (ICS ) for responding to mass fatality incidents. Rationale: Establish ICS to support needed function s and tasks.

Eighth Month Exercise: Drill For: PIO, fire, police, Emergency Manager Purpose: To set up Joint Information Center (JIC) Rationale: Acquaint participants with the PIO funct ion and JIC operations, test equipment and lines of communication. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.25 Building an Exercise Program (Continued) Plan Format Sample Plan: Comprehensive Exercise Pr ogram Ninth Month Exercise: Drill For: Mental health, funeral directors, PIO, clergy, Emergency Manager Purpose: To set up a family assistance center.

Rationale: Acquaint participants with the office eq uipment and test role as support to the victims’ families. Eleventh Month Exercise: Tabletop exercise For: Incident Command, PIO, police, fire, EMS Purpose: To pull together the three functions teste d in the previous drills in the context of a mass casualty incident as the resu lt of a hotel fire. Rationale: Address and resolve potential communication and coo rdination problems among the Incident Command, PIO, police, f ire, and EMS before the functional exercise. Fourteenth Month Exercise: Functional exercise For: Communications, coordination and control, ICS and EOC, PIO, health and medical Purpose: To test additional functions for mass fata lity in the context of a plane crash: Emergency public information effectiv eness, health and medical mass casualty, coordination and control , ICS, and EOC operations. Rationale: Identify preliminary shortfalls and test overall coordination before full-scale exercise. Fifteenth Month Exercise: Tabletop exercise For: Communications, coordination and control, ICS and EOC, PIO, health and medical Purpose: To correct and retest problems identified in preceding functional exercise. Rationale: W ork out potential problems discovered i n the previous functional exercise and make adjustments necessary before the full-scale exercise. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.26 Exercise Design Building an Exercise Program (Continued) Plan Format Sample Plan: Comprehensive Exercise Pr ogram Eighteenth Month Exercise: Full-scale exercise: Airplane crash For: All agencies: heads of agencies and responder s Purpose: To test all functions in the context of a mass casualty airplane crash. Rationale: The exercise fulfills full-scale require ments for FEMA, Federal Aviation Administration (FAA) requirements for airp orts, and Joint Commission on Accreditation of Healthcare Organizat ions (JCAHO) Certification for the hospital. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.27 Activity Activity: Develop a Comprehensive Exercise Program Plan Working from the needs assessment you completed for your jur isdiction or organization in the previous unit, develop a plan f or a comprehensive exercise program to address those needs. Include the key ele ments discussed in the last section. You can use the following wo rksheet (which also appears as Job Aid 2 in Appendix A) in develop ing your plan. If this format doesn’t work for you, change it to meet yo ur needs. Comprehensive Exercise Program Planning Worksheet Timeframe: Present Problems: Long-Range Goal: Functional Objectives: Month: Exercise: For: Purpose: Rationale: UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.28 Exercise Design Activity: Develop a Comprehensive Exercise Program Plan (Continued) Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.29 Activity: Develop a Comprehensive Exercise Program Plan (Continued) Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.30 Exercise Design Activity: Develop a Comprehensive Exercise Program Plan (Continued) Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.31 Summary and Transition Unit 2 provided an overview of the five main types of exercise activities that make up a comprehensive exercise program. Unit 3 provides general information on the exercise development process and illustrates how the activities in that process relate to one another .

Reference Library For More Information The FEMA Comprehensive Exercise Program: www.fema.gov/pte/section3.htm .

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.32 Exercise Design Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer you have chosen. 1. In a progressive exercise program, the exercises: a. Have a consistent format but are conducted with increa sing frequency.

b. Are organized to increase in complexity. c. Are organized to decrease in complexity.

d. Are sponsored on a rotating basis by different organ izations.

2. A likely cause of exercise failure is: a. Running too many drills and functional exercises.

b. Conducting orientations and drills before functional exercises.

c. Rushing into a full-scale exercise before the organiza tion is ready.

d. Basing the selection of participants on the nature of the exercise.

3. Which statement is true of an orientation? a. It requires field sites and actual equipment.

b. It may be used to introduce or explain plans and po licies.

c. It involves a controller, simulators, and evaluators.

d. It is used to test a specific operation. 4. Which statement is true of a drill? a. It is best conducted in a conference room.

b. It involves a controller, simulators, and evaluators. c. It is used to test a specific operation.

d. It is aimed primarily at policy makers and decision m akers.

5. Which statement is true of a tabletop exercise? a. It involves a highly realistic simulation.

b. It involves a controller, simulators, and evaluators. c. It requires field sites and actual equipment deployme nt.

d. It is a facilitated analysis of an emergency situation .

6. Which statement is true of a functional exercise? a. It involves a controller, simulators, and evaluators. b. It is simple, informal, and stress-free.

c. It requires field sites and actual equipment deployme nt.

d. It may be used to introduce or explain plans and po licies.

UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Exercise Design Page 2.33 Knowledge Check (Continued) 7. Which statement is true of a full-scale exercise? a. It involves a highly realistic simulation. b. It is aimed primarily at policy makers and decision m akers.

c. It requires field sites but actual equipment remains i n the shed.

d. It is used to test a specific operation. 8. Development of an exercise program includes analysis o f capabilities and costs and scheduling of tasks.

a. True b. False 9. The planning for an exercise program should be done primarily by the emergency manager or whoever is responsible for the organization’s emerge ncy response effort.

a. True b. False 10. Which statement is NOT true of progressive exercise programs?

a. They allow participating organizations to test both implementation of procedures and coordination with each other.

b. Each successive exercise builds upon the previous one unti l mastery is achieved.

c. The entire program is planned to achieve identified goals.

d. Very little commitment is required from participati ng agencies because they can opt in or out at any time. UNIT 2: C OMPREHENSIVE EXERCISE PROGRAM Page 2.34 Exercise Design Answers Knowledge Check (Continued) 1. b 2. c 3. b 4. c 5. d 6. a 7. a 8. a 9. b 10. d Unit 3: The Exercise Process UNIT 3 THE EXERCISE PROCESS Exercise Design Page 3.1 Introduction In the previous unit, we took the broad view the planning of an entire comprehensive exercise program. In this unit, we will t ake a step closer and look at what goes into planning and implementing a single exercise within that program. When an exercise proceeds smoothly, it all looks so easy! But there is far more to it than the time spent in the exercise itse lf. A great deal of thought and planning preceded the exercise, and more w ork will follow.

An exercise is not an independent activity with clearly marked beginning and ending points. Rather, it can be seen as part of a complex process that involves a number of accomplishments and tasks. All of those tasks are interrelated; they affect not only the success of the current exercise but the design and success of future exercises. It will be much easier to understand and visualize the individual activities if you can first see how they fit into the overall pro cess. This unit presents an overview of the main exercise activities and their relationships to one another. It also discusses some import ant aspects of laying the groundwork for an exercise. In later unit s, we will focus more closely on particular aspects of the design process. Unit 3 Objectives After completing this unit, you should be able to: Identify the five major accomplishments in designing and implementing an exercise.

Describe the organization of an exercise design team.

Define the purpose of the four exercise documents.

UNIT 3: T HE EXERCISE PROCESS Page 3.2 Exercise Design The Big Picture In preparation for launching an exercise program and designing individual exercises, it is important to have a clear vision of the entire exercise process. There are a number of ways to visualize the ex ercise process. Let’s look briefly at three graphic representations of the process: Organized by task sequence, organized by task categories and phase, and organized by major accomplishments. Each of them he lps clarify the overall process. Sequence of Main Tasks In the following graphic, the main tasks are shown in their approximate sequence. This chart may help you get a good mental pi cture of the entire sequence. It’s also a good place to start in creat ing a more detailed schedule of tasks. Sequence of Tasks for a Successful Exercise Review PlanReview Plan Develop Schedule Develop Schedule Determine Exercise Scope DetermineExercise Scope Assess Capability Assess Capability Conduct Needs Assessment Conduct Needs Assessment Write Purpose Statement Write Purpose Statement Send Exercise Directive Send Exercise Directive Organize Evaluation Team Organize Evaluation Team Organize DesignTeam Organize DesignTeam Prepare Objectives Prepare Objectives Write Narrative Write Narrative Write Messages Write Messages Develop Evaluation Format Develop Evaluation Format List Major & DetailedEvents List Major & DetailedEvents Finalize Exercise E nhancements Finalize Exercise E nhancements Determine Expected Actions Determine Expected Actions Conduct ExerciseConduct Exercise Conduct Post-Exercise MeetingConduct Postexercise Meeting Conduct Follow-Up Activities Conduct Followup Activities Write After-Action Report Write After-Action Report Before During After Review PlanReview Plan Develop Schedule Develop Schedule Determine Exercise Scope DetermineExercise Scope Assess Capability Assess Capability Conduct Needs Assessment Conduct Needs Assessment Write Purpose Statement Write Purpose Statement Send Exercise Directive Send Exercise Directive Organize Evaluation Team Organize Evaluation Team Organize DesignTeam Organize DesignTeam Prepare Objectives Prepare Objectives Write Narrative Write Narrative Write Messages Write Messages Develop Evaluation Format Develop Evaluation Format List Major & DetailedEvents List Major & DetailedEvents Finalize Exercise E nhancements Finalize Exercise E nhancements Determine Expected Actions Determine Expected Actions Conduct ExerciseConduct Exercise Conduct Post-Exercise MeetingConduct Postexercise Meeting Conduct Follow-Up Activities Conduct Followup Activities Write After-Action Report Write After-Action Report Before During After Review PlanReview Plan Develop Schedule Develop Schedule Determine Exercise Scope DetermineExercise Scope Assess Capability Assess Capability Conduct Needs Assessment Conduct Needs Assessment Write Purpose Statement Write Purpose Statement Send Exercise Directive Send Exercise Directive Organize Evaluation Team Organize Evaluation Team Organize DesignTeam Organize DesignTeam Prepare Objectives Prepare Objectives Write Narrative Write Narrative Write Messages Write Messages Develop Evaluation Format Develop Evaluation Format List Major & DetailedEvents List Major & DetailedEvents Finalize Exercise E nhancements Finalize Exercise E nhancements Determine Expected Actions Determine Expected Actions Conduct ExerciseConduct Exercise Conduct Post-Exercise MeetingConduct Postexercise Meeting Conduct Follow-Up Activities Conduct Followup Activities Write After-Action Report Write After-Action Report Before During After UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.3 The Big Picture (Continued) Categories of Tasks Another way to look at the exercise process is by organi zing the tasks into two dimensions: Exercise phase (preexercise, exercise, and postexercise) Type of task (those related to design and those related to evaluation) The following matrix illustrates this type of organiza tion.

Task Categories Preexercise Phase Exercise Phase Postexercise Phas e Design Review plan Assess capability Address costs and liabilities Gain support/issue exercise directive Organize design team Draw up a schedule Design exercise (8 design steps) Prepare facility Assemble props and other enhancements Brief participants Conduct exercise Evaluation Select evaluation team leader Develop evaluation methodology Select and organize evaluation team Train evaluators Observe assigned objectives Document actions Assess achievement of objectives Participate in postexercise meetings Prepare evaluation report Participate in followup activities UNIT 3: T HE EXERCISE PROCESS Page 3.4 Exercise Design The Big Picture (Continued) Major Task Accomplishments One of the simplest ways to envision the exercise process is by major accomplishments. As shown in the graphic below, the process can be factored into five major accomplishments that make up th e design cycle: 1. Establishing the base 2. Exercise development 3. Exercise conduct 4. Exercise critique and evaluation 5. Exercise followup Each accomplishment is the outgrowth of a set of specific t asks and subtasks (similar to those listed in the earlier models), which we’ll review shortly. The process is circular, with the results of one e xercise providing input for the next. 1. Establishing the base 5. Exercise followup 4. Exercise critique and evaluation 3. Exercise conduct 2. Exercise development UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.5 The Big Picture (Continued) Flexibility Is Key The exercise process applies no matter what level of exe rcise is being designed and no matter what size organization is using it. Whether you are located in a large jurisdiction with extensive resou rces, a smaller community with meager resources, or a private or volunt eer organization with a particular vested interest in emergency preparat ions, you can use this process. The underlying premise is that this process must be flexib le enough to meet the unique exercise needs of the organization usin g it. Therefore, as you consider each task, it is important to remember th is:

Each task must be designed, tailored, and applied in a manner that suits your organization’s specific objectives and capabilities.

Accomplishment 1: Establishing the Base Exercises are designed to motivate personnel to think or act as they would in a real event. Establishing the base is basicall y laying the groundwork for the exercise to ensure that motivation occurs. Getting ready for the exercise involves: 1. Reviewing the current plan: What does it tell us about ideal performance i.e., how are we supposed to implement policies and procedures in the event of an emergency? 2. Conducting a needs assessment: What are our risks and vulnerabilities, and where do we need to focus our tra ining efforts?

3. Assessing the jurisdiction’s capability to condu ct an exercise: What resources can we draw from to design and implement an exercise? 4. Defining the exercise scope: How should we limit this exercise? 5. Selecting the exercise type: What type of exercise best meets our training needs within the available resources? 6. Addressing the costs and liabilities: What will the exercise cost in terms of funding, human resources, and organizational l iability?

7. Developing a statement of purpose: What do we expect to gain from the exercise? UNIT 3: T HE EXERCISE PROCESS Page 3.6 Exercise Design Accomplishment 1: Establishing the Base (Continued ) 8. Gain support and announce the exercise: How can we obtain the support of those in authority and then use that support to garner support among participants? Additional “groundwork” tasks include organizing a desi gn team and developing a work plan and schedule. Establishing the base need not take a long time and can often be done largely at your desk or with the help of a few other people. We’ll revisit some of these tasks shortly. Accomplishment 2: Exercise Development Exercises particularly tabletop, functional, and full-scale exerci ses are developed by following an eight-step process: 1. Assess needs. 2. Define scope. 3. Write a statement of purpose. 4. Define objectives. 5. Compose a narrative. 6. Write major and detailed events. 7. List expected actions.

8. Prepare messages. UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.7 Accomplishment 2: Exercise Development (Continued) These eight steps will be the focus of Unit 4.

Duplicate Steps? Performing a needs assessment, defining scope, and statemen t of purpose are important aspects of developing the overall exercise program. They crop up again when establishing the base for an individual exercise. And here they appear as part of exercise development. Does this mean we just keep repeating th e same steps?

Certainly not. Sometimes (as in a simple exercise), the effort you put into these steps in the early stages will suffice: You’ll already have done the work when you get to development. Other times ( as with a more complex exercise), you may need to come back to the need s assessment (or the scope or the purpose statement) and buil d on it or refine it further for the particular exercise. The mo re complex the exercise, the more detailed the planning and developm ent tend to be.

UNIT 3: T HE EXERCISE PROCESS Page 3.8 Exercise Design Accomplishment 3: Exercise Conduct The day of the exercise is the culmination of your plan ning. What can you do to help ensure that the exercise is successful? A fe w suggestions are listed below. Conducting a Successful Exercise Be clear. The success of an exercise depends largely on the participants having a clear and consistent understanding o f what is expected of them. Many exercises fail because the ground rules or simulation techniques to be used during the exercise are inadequately explained.

Sustain action. Sustaining action through messages is one way to meet the key objectives of the scenario. Messages help ke ep all participants active throughout the entire exercise. Thi s requires careful monitoring and control of the message flow thro ughout the exercise.

Foster realism. Participants should be encouraged to treat the simulated emergency situations the communications outages, damage, equipment failure, logistical limitations, per sonnel losses, etc. as if they were actually occurring.

Establish timelines. The establishment of valid timelines helps keep what is occurring in its proper order.

Review emergency call-off procedures. To ensure that all participants understand the procedure for a real emerge ncy call-off, they should be briefed before the exercise begins. Capitalize on problem situations. Situations that can cause stoppage in an exercise also have value because they place added stress on the system. Often they will test more effective ly the organization’s capability to cope in times of real emer gencies.

UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.9 Accomplishment 4: Exercise Evaluation and Critique An important part of any exercise is evaluating how w ell the objectives have been achieved. The objectives might relate to such issues as:

Needed improvements in the plan.

Needed improvements in the emergency management system. Personnel training.

Overcoming staffing deficiencies.

The extent and depth of the evaluation are determin ed by the participating organizations. Controllers’ evaluations and observations may suffice for some exercises whereas additional analysis b y objective observers may be needed for others. Critiques and reports analyze and explain the attainm ent or nonattainment of the exercise objectives and provide re commendations for addressing any deficiencies. Evaluators should be tho roughly familiar with the community’s EOP and the area that they are e valuating.

Strategies for evaluating an exercise will be the focus of Unit 8.

Evaluator Checklist ______________________ ______________________ ______________________ ______________________ UNIT 3: T HE EXERCISE PROCESS Page 3.10 Exercise Design Accomplishment 5: Exercise Followup Followup is one of the most neglected areas of the exe rcise process. An exercise without evaluation and recommendations makes fo r an incomplete exercise. Similarly, recommendations without followup will keep you from getting the full benefit of the exerci se. How do you follow up on an exercise? Selected strategi es are provided below. Exercise Followup Strategies Assign responsibility. Clearly assign tasks and schedules, and designate responsibility for each recommended improvemen t.

Monitor. Establish a monitoring plan to track the progress of implementing recommended improvements.

Complete the cycle. Build the testing of improvements into the next exercise. This is perhaps the surest way to make certain t hat they are implemented. It is probably not necessary to retest every objective fu lly. Rather, select a few recommendations that would illustrate impro vements and include those in a future exercise. This overview has presented the exercise process in a nutshe ll. As mentioned earlier, more detailed discussions of many aspe cts of this process will follow in later units. Now, let’s take a cl oser look at some of the key tasks in establishing the base for an exercise. ( Other groundwork tasks those that also become part of the exercise development process will be the subjects of subsequent units.) UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.11 Reviewing the Current Plan The EOP describes how personnel in the organization sho uld respond in the event of an emergency. It answers the question, “W hat resources, personnel, and procedures will be used to resolve proble ms created by an emergency?” Your plan may be called by various names perhaps one of the following: EOP Area Contingency Plan Operational or operating plan For simplicity, we will refer to all of these plans as emergency plans, and the plan is the place to begin. Examining the plan will help you identify problems, select an exercise, define its purpose, and formulate objectives. While ex ercises test performance, it is the plan that describes the ideal per formance. To use your plan properly as the base document, you must first become familiar with it. What to Look for in the Emergency Plan While reviewing the plan, ask yourself key questions: What responses are currently planned (i.e., what are th e hazards that the plan is intended to address)? What resources, personnel, and procedures will be used to resolve problems?

Are they different for various types of emergencies?

Do roles vary according to the type of emergency?

What training have response personnel experienced?

What training is necessary? UNIT 3: T HE EXERCISE PROCESS Page 3.12 Exercise Design Assessing Capability to Conduct an Exercise To design an exercise that simulates a real emergency, y ou must know what responses are planned and assess what capabilities are needed to meet those responses. Then you must ask, “Are we at a po int where we can conduct an exercise?” Before you launch into an exercise, it’s important to f ind out if you have the resources skills, funding, personnel, time, facilities, and support . Deficiencies in any of these areas will have to be conside red in the design of the exercise. When sheer enthusiasm says, “Let’s do a f ull-scale exercise,” sometimes the capability assessment will say, “We can only do a tabletop right now.” Questions to Ask About Capability and Resources The questions below, though very general, can help you assess your organization’s level of capability. For example, you may find that before you consider planning an exercise, you will need to dev elop support and train people. 1. When was your organization’s last exercise? 2. What exercise experience is available in your communi ty or organization? What is your own experience? What is y our staff’s experience?

3. How much preparation time can you reasonably expect to have allocated to developing an exercise?

4. How much time can people devote to developing an e xercise?

5. What skills can those people provide? 6. What physical facilities do you use when you conduct an emergency operation? Will they be available for an exercise?

7. What communication facilities and systems do you use in a real emergency? Will they be available for an exercise?

8. What attitudes do you expect of the chief executive and emergency service directors or other organizational leaders toward the exercise? UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.13 Addressing Costs and Liabilities Costs and liabilities should be addressed early on. Liabilities A problem inherent in many exercises particularly drills and full-scale exercises is the possibility of personal injury or damage to equi pment. Before planning the exercise, check the organization’s o r jurisdiction’s insurance coverage. Costs Costs (both apparent costs and hidden costs) are incurred at every stage of exercise development. Plan ahead to be sure that y ou have the resources. Some cost considerations to look at are listed b elow.

Cost Considerations Plan for a wide variety of costs. The following are a few examples; actual cost items will depend on the exercise and your or ganization’s policies.

Staff salaries Contract services Equipment and materials Fuel to run equipment and transport volunteers Miscellaneous items (e.g., coffee, pencils) Ask some key questions to avoid committing more resources t han are available. For example: Will reimbursement for overtime be required if the exercise takes place on a weekend or evening? If the exercise supports a hospital certification exercise, who will cover the costs?

Employees should have their emergency management responsibilities reflected in their job descriptions. Time for participation in training, planning, and ex ercising should be set aside by each employee who has an emergency manageme nt responsibility. Costs for routine participation in exercises should be reco gnized by agency or organization officials. UNIT 3: T HE EXERCISE PROCESS Page 3.14 Exercise Design Activity Activity: Getting Ready for Exercise Design The questions below relate to your organization’s readi ness for the exercise process. Write brief answers to the following que stions in the spaces provided. ( Note: This form also appears as Job Aid 3 in Appendix A.) Self-Assessment: Resources and Costs 1. Plans How familiar are you with the emergency plans, policie s, and procedures of your organization or jurisdiction? Very familiar Only general familiarity Familiar with only a portion Need to review plans, policies, and procedures thorough ly 2. Time a. How far in advance would your organization realist ically have to schedule to plan and design each of the following exercise activities effective ly?

Orientation ________________________________ Drill ________________________________ Tabletop exercise ________________________________ Functional exercise ________________________________ Full-scale exercise ________________________________ b. How much preparation time can reasonably be allocate d to developing an exercise?

Actual person days:

Elapsed time to exercise:

UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.15 Activity: Getting Ready for Exercise Design (Conti nued) Self-Assessment: Resources and Costs (Continued) 3. Experience a. When was your organization’s last exercise? b. What is your previous experience with exercises? (Che ck all that apply.) Orientation: Presenter Participant Drill: Controller Participant Tabletop exercise: Facilitator Participant Functional exercise: Controller Simulator Player Evaluator Full-scale exercise: Controller Responder Evaluator Victim Took part in postexercise debrief Helped write an evaluation report c. What other exercise-related experience is available i n your organization?

4. Facilities What physical facilities do you use when conducting an eme rgency operation?

Will they be required for this exercise? Yes No Will they be available for this exercise? Yes No UNIT 3: T HE EXERCISE PROCESS Page 3.16 Exercise Design Activity: Getting Ready for Exercise Design (Conti nued) Self-Assessment: Resources and Costs (Continued) 5. Communications : What communications facilities and systems do you use in a r eal emergency? Will they be required for this exercise? Yes No Will they be available for this exercise? Yes No 6. Barriers : Are there any resource barriers that need to be overcom e to carry out this exercise? Yes No If so, what are the barriers and how can they be overco me?

UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.17 Activity: Getting Ready for Exercise Design (Conti nued) Self-Assessment: Resources and Costs (Continued) 7. Costs a. What types of costs might be incurred for these exercise s in your organization? (Do not list exact figures just types of expenses, such as wages and salaries, transporta tion, etc.) For an orientation: For a drill: For a tabletop exercise: For a functional exercise: For a full-scale exercise: b. Are there ways that different organizations can red uce costs (e.g., by combining exercises, cost-sharing, resource-sharing) and still fulfill program requirements? Explain.

UNIT 3: T HE EXERCISE PROCESS Page 3.18 Exercise Design Gaining Support It is important, from the beginning, to establish aut hority for conducting the exercise. This means gaining the support of the hig hest possible official in your jurisdiction or organization. Even if the chief executive does not participate in the exercise, his or her authority and support are essential. The top official can get nearly instant and complete cooperation from those who will participat e in the exercise. It can be very difficult without that authority and su pport to put on an exercise involving departments, organizations, or agenci es over which you personally have no authority. Gaining support of the executive is not always easy, but the following approaches will help: Gain support for the entire exercise program. Build a comprehensive, progressive exercise program. The executiv e will be more receptive to an exercise that is part of a proven, consistent, and goal-oriented program than to an isolated exercise.

Protect the organization. Make a conscientious effort to protect the organization or jurisdiction from lawsuits.

“Sell” the process. Your needs assessment, capability analysis, purpose statement, and objectives are important buildin g blocks for the exercise. Beyond that, they provide a valuable t ool for selling the idea professionally first to your boss and later to the chief executive of your organization.

Presenting the Concept Having the results of your early preparation activitie s in hand will add to your credibility when presenting the exercise concept to those in authority. Your presentation should include brief exp lanations of:

The need for the exercise.

Organizational capability (experience, personnel, costs). The type of exercise.

The scope of the exercise.

The purpose of the exercise. UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.19 Gaining Support (Continued) Announce the exercise. Broad support for the exercise may be gained in some instances by sending out an announcement. The announcement often in the form of an exercise directive should come from the chief executive, but you should be prepar ed to write it. The directive serves the purpose of authorizing you to conduct the exercise and giving you the clout you need to gain supp ort from others.

Writing the Exercise Directive The exercise directive will closely resemble the purpose st atement (which will be discussed in more detail in Unit 4). The directive should contain the: Purpose.

List of participating agencies, organizations, or depart ments.

Personnel responsible for designing the exercise.

Exercise date (or approximate dates).

Point of contact for additional information.

Notice that the nature of the emergency and the locati on of the exercise are not revealed. Regarding dates: Totally unannounced exercises are not recommended. However, whether you specify the exact da te will depend on the degree of surprise that you intend. At a minimum, a range of dates should be given. In selecting the exerci se date, check the community calendar to avoid conflicting with a majo r event (e.g., an athletic event). An example of an exercise directive is given on the nex t page. Although this example illustrates an Emergency Management Office announcement, a similar approach would be appropriate for an organization in the nonprofit or private sector. UNIT 3: T HE EXERCISE PROCESS Page 3.20 Exercise Design Gaining Support (Continued) Sample Exercise Directive February 24, 20XX TO: All Agency Directors FROM: CHARLENE W. WILLIAMS Chief Administrative Officer SUBJECT: Emergency Exercise A simulated emergency exercise involving a terrorist in cident has been scheduled for sometime during the week of May 12–18, 20xx.

The purpose of the proposed exercise is to improve the f ollowing emergency operations:

1. Rapid assessment.

2. Notification and alert.

3. Scene isolation and perimeter control.

4. Mass casualty triage.

It is important that your agency participate in this exercise. We encourage involvement at the highest level.

I believe we all realize the importance of emergency exercises as a means to community preparedness. I fully support this exercise and intend t o join with you in participating.

The Emergency Management Office will be coordinating t he exercise. They will be contacting you to make necessary arrangements for the development and conduct of the exercise. For purposes of realism and interest, details of the exercise situation will not be made known prior to the exercise.

For further information, call Tom Smith at EXT 1234.

UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.21 Assembling a Design Team Planning an exercise requires a multitude of tasks, from designing the exercise to arranging detailed administrative matters. These tasks require the effort of a dedicated team and team lead er.

Exercise Design Team Leader The exercise design team leader is responsible for the e xercise throughout the entire development process—and for manag ing all administrative and logistical matters. In an exercise in volving multiple organizations or jurisdictions, the team leader needs assi stants or liaison persons from each entity to help coordinate the many de tails.

Who Should Lead ?

Because the leader is so important, it is essential that t his person be experienced and capable. The exercise design team leade r should be someone who: Can devote a considerable amount of time throughout t he exercise cycle. Is familiar with the emergency plan and has a sound und erstanding of the response organizations that will be participating. Is not a key operational member of one of these organizati ons. All key members should participate in the exercise, and they might not be able to participate fully if they are involved in the design.

What About the Emergency Manager? Generally speaking, the person with chief responsibility for managing emergency events should be a player in the exercise. Th erefore, instead of heading the design team, the Emergency Manager shou ld assign someone else to develop and conduct the exercise. Prior to the exercise, the Emergency Manager should be given the same informa tion about the scenario as other department heads but no more.

When staffing does not permit the Emergency Manager to assign someone else, he or she will need to play a dual role: Assume responsibility for developing the exercise and get people from other agencies to help. UNIT 3: T HE EXERCISE PROCESS Page 3.22 Exercise Design Assembling a Design Team (Continued) Participate in the exercise, but on a limited basis. De spite being familiar with the exercise design, the Emergency Manage r can still take a number of actions without compromising the reali sm for example, negotiate and coordinate, support the decision s of the chief executive, and approve resource allocation.

Exercise Design Team Exercise design is a complex task that should not be done by a single individual. The exercise design team assists the team lea der in developing exercise content and procedures. Design Team Responsibilities Design team members: Determine the exercise objectives.

Tailor the scenario.

Develop the sequence of events and associated messages.

Assist in the development and distribution of preexercise materials.

Help conduct preexercise training sessions.

Ultimately, team members will be good candidates to act as simulators or controllers in a functional exercise. Design Team Candidates Ideally, the team should include a representative from each of the participating jurisdictions in a multijurisdictional exer cise. In a single- jurisdictional or single-organization exercise, it should include representatives from the key departments. If this resul ts in an unwieldy team, however, select a small core team that can draw on other members as needed. UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.23 Assembling a Design Team (Continued) Backgrounds Selecting team members with varied backgrounds helps wit h coordination and stimulates creativity. Some additional technical an d administrative support may be required for typing, printing, and ot her mechanics involved in materials preparation. Organizing the Design Team Design teams are so varied in number of members, team makeup (multiagency or single organization), available skills, political climate, scope of the exercise program to be developed, and many other factors that there is no single approach to organizing a design team.

Generally, the team leader should use the kinds of te amwork strategies that make any team successful. Team leadership is too lar ge a topic to address in this course, but there are many resources (courses, written documents, Internet resources) you can access to learn more a bout teamwork. A few suggestions for organizing the work o f a design team are listed on the next page. UNIT 3: T HE EXERCISE PROCESS Page 3.24 Exercise Design Assembling a Design Team (Continued) Strategies for Organizing the Team Establish clear goals.

Agree on a plan of action, with specific responsibilities and due dates, to meet the established goals.

Reach consensus on a realistic schedule for completing milesto ne activities.

Meet regularly to monitor progress.

Work together to share expertise and resources to get th e job done.

Use the team interaction to learn more about each othe r’s organizations. The more you know about other stakeho lders, the better you will be able to work together to manage emergencies when they arise.

Keep lines of communication open about new ideas, poten tial pitfalls, successes, problems, and needs. Shared informatio n and creative problem solving are huge assets in emergency management. Use job aids to make the schedule, responsibilities, and p rogress visible and to keep everyone “on the same page.” Some examples are provided in the following pages. R emember, these are only examples: Any job aid you use should be adapted to the needs of your team. UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.25 Assembling a Design Team (Continued) Checklist. A simple “to-do list” similar to the following can be u sed to provide an overview of the process and ensure that all main tasks are completed. (Note: This checklist appears as Job Aid 4 in Appendix A.) Sample Exercise Development Checklist Mission Scenario Needs Assessment Narrative Scope Major/Detailed Events Statement of Purpose Expected Actions Objectives Messages Personnel Logistics Design Team Safety Controller or Facilitator Scheduling Players Rooms/Location Simulators Equipment Evaluators Communications Management Phones Safety Radio Observers Computers Enhancements Information Maps Directives Charts Media Other:

Public Announcements Invitations Evaluation Community Support Methodology Management Support Locations Timeline Requirements Evaluation Forms Postexercise Debrief Training/Briefings After Action Documentation/ Train Simulators, Evaluators, Recommendations Controllers Evaluation Meeting Players’ Preexercise Briefing Evaluation Report Followup Ideas for Next Exercise UNIT 3: T HE EXERCISE PROCESS Page 3.26 Exercise Design Assembling a Design Team (Continued) Activities Schedule. For relatively simple exercises, a basic schedule may be used to show major activities of the team leader and the team and completion deadlines for each. An example is shown below. Deadlines should be realistic, b ased on team resources. (Note: See Job Aid 5 in Appendix A for a blank version of this sche dule form.) Sample Activities Schedule Deadline for Completion Leader Activities Team Acti vities 3 months prior Hold initial planning meeting 2½ months prior Brief government officials Arrange for facilities Determine simulation structure Convene and brief design team Attend team briefing 2 months prior Review and finalize scenario Develop/review exercise procedures Arrange simulation Arrange participation Review exercise scenario 1½ months prior Obtain exercise materials Prepare ideas for scripted messages Prepare participant information packet Prepare operational data 1 month prior Review messages with team Review messages with leader Review evaluation forms Print forms Prepare scripted messages 3 weeks prior Prepare briefing for participants 2 weeks prior Integrate messages into time schedule Develop training sessions 1 week prior Prepare exercise facility 2–4 days prior Conduct training session Train supervisors Assist in training sessions UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.27 Activities Schedule Sample Activities Schedule (Continued) Deadline for Completion Leader Activities Team Acti vities Day of Exercise Conduct participant briefing Perform preexercise check Supervise the exercise Assist with preexercise check 1 week after Help prepare draft of final report Review final report and make suggestions 2 weeks after Revise and submit report 3 weeks after Submit recommendations UNIT 3: T HE EXERCISE PROCESS Page 3.28 Exercise Design Assembling a Design Team (Continued) A more detailed plan is needed for a major exercise, which must be planned with the thoroughness of any major organizati onal effort. A Gantt chart, as shown in the example below, is a useful scheduling tool for such detailed planning. A Gantt chart displays time across the top and a sequence o f tasks down the left-hand side. Time can be given in days, weeks, or months. The duration of time devoted to each activity is represente d by bars extending across the time lines. The example shows task groupings (with supporting subtasks scheduled) and staff allocations indicated by initials in the left column.

Sample Gantt Chart Tasks January February 1 7 14 21 28 4 11 18 25 Refine Objectives Scenario Development Narrative (EW , CF) Major Events (EW , CF) Detailed Events (EW ) Messages (EW ) Materials Development Maps (SD) Handouts (SD) (etc.) UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.29 Activity Activity: Identify Design Team Members Who can assist in developing exercises? Consider your own staff, other departments or agencies, volunteer organizations, the S tate Emergency Office, or others as appropriate. Consider the followi ng types of skills that may be needed for designing exercises: Planning Logistics Promotion Materials Scenarios/messages Other Using the worksheet below (also provided as Job Aid 6 in A ppendix A), identify potential candidates for a design team to work on the exercise program that you outlined in Unit 2 and decide who cou ld be the leader. For each person, identify what function or organization they represent and what they would bring to the design process (e.g., p articular skills, experience, knowledge, perspective). Design Team Worksheet Name Agency Represented Contributions/Qualification s Leader Members UNIT 3: T HE EXERCISE PROCESS Page 3.30 Exercise Design Exercise Documents Four major documents are developed during the exercise design process:

Exercise Plan Control Plan Evaluation Plan Player Handbook These documents are basically handbooks for specific audience s and serve as useful tools during exercise development, conduct, and evaluation. Exercise Plan The exercise plan contains information that everyone n eeds and serves many purposes. For example, it can be used: To provide general information about the exercise for everyone concerned, including exercise overview, parameters, and t imelines.

As a guide for developers and participants.

To assist participants in enhancing exercise play.

As a promotional tool by the exercise director.

Control Plan The control plan is for controllers and simulators. It is not made available to players. It provides information about controller and simulator requirements and explains the exercise concept as it relat es to controllers and simulators. This plan: Establishes the basis for control and simulation of the ex ercise.

Explains the procedures, responsibilities, assignments, and support for exercise control and simulation.

Defines the communications, logistics, and administrative st ructure needed to support control and simulation during the ex ercise.

UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.31 Exercise Documents (Continued) Evaluation Plan The evaluation plan provides exercise evaluators, as we ll as controllers and simulators, with guidance concerning exercise evaluati on procedures, responsibilities, and support. This plan: Explains the exercise concept as it relates to the evaluat ion process.

Establishes the basis for evaluation.

Defines the communications, logistics, and administrative st ructure needed to support evaluation before, during, and aft er the exercise.

Player Handbook The Player Handbook provides exercise players with inf ormation needed to participate effectively in the exercise. This inform ation is also discussed at the player briefings conducted prior to the st art of the exercise. Specific information included in each document is listed i n the table on the next page. Templates of these documents are provided i n the Exercise Design course Tool Box, available at http://training.fema.gov/emiweb/IS/is139lst.asp .

UNIT 3: T HE EXERCISE PROCESS Page 3.32 Exercise Design Exercise Documents (Continued) Contents of Exercise Design Documents Exercise Plan Control Plan Evaluation Plan Player H andbook Exercise type and purpose Scenario narrative Scope References Assumptions, artificialities, and simulations Objectives Concept of operations Exercise management structure and responsibilities Safety and security Administration and logistics Exercise management job aids Exercise concept Preexercise player activity Assumptions, artificialities, and simulations Concept for management, control, and simulation Control team staffing Control team training Control team staff responsibilities Control team procedures Communications, logistics, administrative, and other support Exercise concept Preexercise player activity Assumptions, artificialities, and simulations Concept for evaluation management Evaluation team staffing Evaluation team training Evaluation team staff responsibilities Evaluation team procedures Support for the evaluation team Exercise scope (concept of play, assumptions, artificialities, and simulations) Scenario narrative Player procedures and responsibilities Safety and security Communications Reporting Administrative systems Recommended preexercise training events Schedule of player exercise briefings Provisions to review community plans, policies, and procedures EOC (or other operating center) procedures UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.33 Summary and Transition Unit 3 presented an overview of the main exercise acti vities and their relationships to one another. It also discussed some impor tant aspects of laying the groundwork for an exercise, including: Reviewing the current plan.

Assessing capability to conduct an exercise.

Addressing costs and liabilities.

Gaining support.

Assembling and organizing the design team.

Finally, this unit provided an overview of th e four main design documents: the exercise plan, control plan, evaluation plan, and player handbook. Unit 4 explains the eight steps in the design process. Reference Library For More Information FEMA Internet Exercise Toolkit: http://training.gov/emiweb/IS/is139lst.asp FEMA Comprehensive Exercise Program:

http://www.fema.gov/pte/cepnew.htm FEMA National Exercise Schedule: http://www.fema.gov/pte/nes.htm FEMA Regional Liaisons: http://www.fema.gov/pte/pdb.htm FEMA courses related to team leadership:

Leadership and Influence (G240) Decision making and Problem-Solving (G241) Effective Communication (G242) UNIT 3: T HE EXERCISE PROCESS Page 3.34 Exercise Design Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer that you have chosen. 1. The major task accomplishments in the exercise process are :

a. Conducting tabletop, functional, and full-scale exer cises.

b. Developing, conducting, and evaluating the exercise. c. Establishing the base, exercise development, exercise con duct, critique and evaluation, and followup.

d. Conducting a needs assessment, writing objectives, and e valuating whether the objectives were achieved. 2. In conducting an exercise, unexpected problem situatio ns are likely to lead to failure of the exercise. a. True b. False 3. Critiques and reports analyze how well objectives we re achieved. They should also provide recommendations for addressing any deficiencies.

a. True b. False 4. An important followup strategy is to implement nee ded improvements revealed in the exercise and to test those improvements in the next exer cise.

a. True b. False 5. Availability of resources: a. Is irrelevant to the exercise process.

b. Should not affect exercise design.

c. Should be considered in scheduling and planning an ex ercise.

d. Is seldom an issue for exercise design because exercises req uire few resources.

6. The exercise design team leader should be: a. The Emergency Manager.

b. Someone who is familiar with the emergency plan.

c. A key operational member of a participating organi zation.

d. The chief official of the jurisdiction or organizati on.

UNIT 3: T HE EXERCISE PROCESS Exercise Design Page 3.35 Knowledge Check (Continued) 7. The design team should consist of: a. As many members as possible from all agencies or depart ments.

b. Two or three members of the chief official’s key staf f. c. Members with varied backgrounds, representing key par ticipating entities.

d. The organization’s training department. 8. The primary audience of the Exercise Plan is: a. Controllers and simulators.

b. Participants.

c. The facilitator.

d. Everyone involved in the exercise. 9. The Control Plan explains the procedures, responsibil ities, assignments, and support for exercise control and simulation.

a. True b. False 10. The Control Plan should not be seen by __________ ____.

a. Controllers b. Simulators c. Evaluators d. Players 11. The ______________ can be used as a promotional to ol by the exercise director.

a. Exercise Plan b. Control Plan c. Evaluation Plan d. Player Handbook UNIT 3: T HE EXERCISE PROCESS Page 3.36 Exercise Design Answers Knowledge Check (Continued) 1. c 2. b 3. a 4. a 5. c 6. b 7. c 8. d 9. a 10. d 11. a Unit 4: Exercise Design Steps UNIT 4 EXERCISE DESIGN STEPS Exercise Design Page 4.1 Introduction Exercise design is much like scripting a play to make sure that all of the players perform the correct actions and make the right d ecisions at the appropriate time. Tabletop, functional, and full-scal e exercises are based on a design process that includes the eight steps introduced in the previous unit: 1. Assess needs. 2. Define scope. 3. Write a statement of purpose. 4. Define objectives. 5. Compose a narrative. 6. Write major and detailed events. 7. List expected actions. 8. Prepare messages. This unit describes what is involved in completing each of the eight steps.

Unit 4 Objective After completing this unit, you should be able to list and explain the eight exercise design steps. UNIT 4: E XERCISE DESIGN STEPS Page 4.2 Exercise Design Step 1: Assess Needs Some organizations plan exercises as a response to pressure or a “gut feeling.” For example, someone may suddenly decide to do a full-scale exercise based on some dramatic disaster, because full-scale e xercises generate a lot of excitement. Such hasty decisions usuall y result in failure and embarrassment. The best way to determine whether you need an exercise and what kind of exercise is needed is to study your situation systematically. Conducting a needs assessment will help you define the pr oblems, establish the reasons to do an exercise, and identify the functions to be exercised. Begin With Your Plan A needs assessment should begin with a review of the emer gency plan and should address: Hazards—the risks that you are most likely to face and th e priority levels of those hazards.

Area(s) most vulnerable.

Functions most in need of rehearsal.

Potential participants (agencies, organizations, departm ents, personnel).

Past exercises.

Exercise requirements.

Capabilities.

If you have assessed your organization’s exercise needs whe n planning a comprehensive exercise program, you have a good basis alr eady. (For example, the assessment you completed in Unit 1 touched o n many key issues.) Consulting and updating that assessment will be a n important step whenever a new exercise is considered for developmen t.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.3 Step 1: Assess Needs (Continued) Lessons Learned In doing a needs assessment for a single exercise, an obvio us starting point is the evaluations of past exercises: Who participated in the exercise and who did not?

To what extent were the exercise objectives achieved?

What lessons were learned?

What problems were revealed, and what is needed to re solve them?

What improvements were made following past exercises, an d have they been tested?

Needs Assessment Results In summary, your needs assessment should reveal the follow ing types of issues if they exist: Primary and secondary hazards that the organization fa ces Problems that need to be resolved Problems that recur Skills that need to be practiced Functions that are weak Improvements implemented after previous exercises, which now need to be tested New facilities, personnel, or equipment that have not been included in an exercise Weaknesses (such as gaps, conflicting policies, or vague procedures) in the emergency plan or the SOPs The need for role clarification The need for a certain type of exercise UNIT 4: E XERCISE DESIGN STEPS Page 4.4 Exercise Design Step 1: Assess Needs (Continued) A needs assessment form appears as Job Aid 1 in Appendix A. Now that you know a little more about exercising, you may wish to review your assessment from Unit 1 and update it if needed. Step 2: Define the Scope Defining the scope of an exercise means putting realistic limits on the issues that you identified in the needs assessment.

Why Define the Scope ?

A needs assessment may reveal a wide array of concerns. Cl early you can’t design an exercise that effectively practices: All functions. . .

In the context of all hazards. . .

Using all agencies, organizations, or departments. . .

In all exercise formats. . .

Employing all resources.

You will need to set priorities and make choices. It is important that the scope be clearly and narrowly defined.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.5 Step 2: Define the Scope (Continued) How Is Scope Determined? Many factors influence which areas of concern will be inclu ded in an exercise and which will not. Sometimes one decision will influence another (for example, the functions that are selected w ill determine who plays in the exercise). Other factors that help define the scope of an exercise include: Expense.

Availability of personnel and other resources.

Seriousness of the problem.

Capability of the exercise to address the problem.

Skills and experience of the designers.

Length of the exercise. What Does Scope Include?

There are five key elements of scope: Type of emergen cy, location, functions, participants, and exercise type. Type of emergency: An exercise is usually limited to one major event, although others especially secondary events might develop as the scenario develops. Hazards may be chosen for severa l reasons, including: The emergencies that will generate the types of action s that need to be practiced.

The highest priority hazards that the organization f aces.

The hazards that haven’t been exercised recently.

Problems that have just recently developed.

Location: Identify the location (a specific address) where the simulated event will occur. For tabletop and functiona l exercises, select a place where the hazard could realistically occur. For a full- scale exercise, traffic problems or safety issues may make it necessary to compromise on an area similar to the ideal l ocation.

UNIT 4: E XERCISE DESIGN STEPS Page 4.6 Exercise Design Step 2: Define the Scope (Continued) Functions: List the operations that the participants will practice. Be sure that the procedures within a certain function are clear and narrowly defined . Example To exercise a community’s alert warning system, the follo wing actions might be part of a response function: Exercise Alert Warning System Notify the warning agency.

Turn on sirens.

Notify fire or police to use loud speakers in area.

Notify Emergency Alert System (EAS) to interrupt prog ramming with message.

Participants: After the most important functions or needs have been identified, you can narrow the list of participating o rganizations and individuals to those that are required to carry out the actions.

Ask yourself:

Which organizations need to be involved to carry out t he function(s) being tested?

Which representatives from the identified organization s should be there?

For example, in an EOC or other operations center, yo u would typically want policy makers, coordinators, and operatio ns personnel. In an Incident Command Post, you would most likely wan t personnel who are knowledgeable in field operations and have so me on-scene decision-making authority. UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.7 Step 2: Define the Scope (Continued) Exercise type: Finally, a decision must be made on the type of exercise. Ask yourself:

What exercises are most needed?

What experience have personnel had with the various ty pes of exercises?

What stress level do we want?

What types of exercises are mandated by regulatory requirements?

After these issues have been settled, it is time to formu late them into a statement of purpose.

Step 3: Write a Statement of Purpose The purpose statement is a broad statement of the exercise goal . It focuses and controls the whole exercise. The purpose statem ent:

Governs the selection of the objectives, which in turn go vern subsequent steps. Clarifies for the chief executive and potential partici pants why the exercise is being conducted. Is useful in communicating plans to the media and communi ty leaders.

Developing the Purpose Statement A purpose statement is easily constructed. One approach is simply to incorporate the scope decisions (type of emergency, locatio n, functions, organizations, and exercise type) into a single sentence. A date is usually added. Two sample formats are provided on the following page s.

UNIT 4: E XERCISE DESIGN STEPS Page 4.8 Exercise Design Step 3: Write a Statement of Purpose (Continued) Using the Purpose Statement as an Exercise Directiv e Statement of Purpose: Sample 1 The purpose of the proposed emergency management exerci se is to improve the following emergency operations:

a. Flood stage monitoring b. Evacuation warning c. Relocation of school children d. Shelter management by involving the following agencies:

a. Emergency Management b. Fire Department c. Public Works d. Health Department e. Red Cross f. Public Schools in a functional exercise simulating a flash flood at Planter’s Street Bridge to Route I-740 on April 2.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.9 Step 3: Write a Statement of Purpose (Continued) Statement of Purpose: Sample 2 The purpose of the proposed emergency management exerci se is to coordinate the activities of city and county government, volunteer organizations, and private industry in their response to a major incident; to provide training to staff; to test and evaluate the Alert and Warning, Evacuatio n and Shelter/Mass Care Annexes; and to enhance interagency coordination and coo peration by involving the following department or agency heads: 1. County Commissioner or Chief Administrative Officer 2. Mayor 3. Emergency Manager 4. Emergency Manager 5. Fire Chief 6. Law Enforcement 7. PIO 8. HazMat Team Liaison 9. Chemical Expert 10. Poison Control Center 11. The American Red Cross 12. Liberty City Hospital Justice County City of Liberty City of Liberty Justice County Justice County Justice County Sheriff Liberty City Gazette Representative Justice County Team #3 Arrow Chemical Company Dr. Smith, Disaster Director Emergency Room Director These entities will be tested on July 15, in a simulated exercise involving a hazardous materials transportation accident at SW Mail R oad near SW Johnston Boulevard, approximately 300 yards from the L iberty City Hospital.

The exercise directive, as discussed in Unit 3, is a memo fr om the chief official in your organization or jurisdiction, sent to agencies or departments whose support you need. The directive is esse ntially a restatement of the purpose statement. When using the statement of purpose as the basis for the directive, the following information should be added: Contact person and telephone number Hours the exercise will be conducted Exercise location (may be omitted to retain an element of surprise) UNIT 4: E XERCISE DESIGN STEPS Page 4.10 Exercise Design Activity Activity: Define Exercise Scope and Purpose In Unit 2, you outlined a comprehensive exercise program f or your organization. Select a tabletop or functional exercise from that outline. Then, define the scope and write a statement of purpose using the worksheet (also provided as Job Aids 7 and 8 in Appendix A) .

1. Highest priority hazards (major and secondary): 2. Geographic areas/locations of greatest vulnerability to these hazards:

3. Agencies/departments/organizational units: List belo w the entities that have a significant role in emergency management/response. Then, enter che ck marks in any columns that apply. Agency/Organization Limited experience with major emergencies New plans, staff, or organizational structure not yet exercised Problems revealed in prior exercises UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.11 Activity: Define Exercise Scope and Purpose (Conti nued) 4. Types/levels of personnel that you want to have in the exercise:

Policy making (elected officials, chief operating office rs, department heads) Coordination (managers, EOC representatives, departme nt deputies) Operations (field personnel, headquarters staff level) Public representatives (media, PIOs, general public) Other: _____________________________________ 5. Types of operations/functions that you want participa nts to engage in (e.g., notification, evacuation): 6. Degree of stress, complexity, time pressure that the e xercise should have:

High Medium Low Stress Complexity Time pressure UNIT 4: E XERCISE DESIGN STEPS Page 4.12 Exercise Design Activity: Define Exercise Scope and Purpose (Conti nued) Exercise: Scope: Type of Emergency: Location: Functions: Organizations and Personnel: Exercise Type: UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.13 Activity: Define Exercise Scope and Purpose (Conti nued) Statement of Purpose: UNIT 4: E XERCISE DESIGN STEPS Page 4.14 Exercise Design Step 4: Define Objectives Early in the development of an exercise, you must decid e what the exercise is intended to accomplish. These outcomes, or obje ctives, must be specified clearly. An objective is a description of the performance you expect from participants to demonstrate competence . Objectives go hand in hand with the purpose statement but are more specific and per formance based.

Why Define Objectives ?

Objectives are essential during the four stages of the ex ercise process: Design process: Objectives are the pivot point in the design process:

The needs assessment, scope, and purpose statement lead to the formulation of objectives.

The success of later actions and decisions begins with careful ly written objectives.

The narrative, the major and detailed events, expect ed actions, and messages are all based on the objectives. In one sense , the objectives can be thought of as general statements of exp ected actions.

Exercise conduct: During the exercise itself, elements of the exercise should be conducted according to the objectives to make sure that it stays on track. Evaluation: Writing objectives is the beginning of the exercise evaluation process. During the exercise, observers use the objectives to evaluate performance. After the exercise, the evaluation report is based upon those objectives. The p rocess of identifying evaluation criteria takes place at the tim e objectives are written. Followup: During the followup period, participants retrain, pla n, and practice to address objectives that were not fulfilled. UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.15 Step 4: Define Objectives (Continued) How Are Objectives Determined ?

Many objectives become evident at the time of the need s assessment, when designers identify problem areas. These needs can u sually be translated into a statement of objectives. Objectives are also arrived at by breaking down a purp ose statement into its logical components. Example: Suppose your last exercise showed weaknesses in alert and notification specifically a failure on the part of the EOC to analyze and implement call-down procedures. One of th e resulting objectives would be to verify that the EOC is now able to notify the proper agencies according to the plan.

How Many Objectives ?

There can be as few as two or three objectives in a small exercise, or as many as 100 in a large national exercise including many Federal, State, and local jurisdictions. For an average exercise, 10 or fewer objectives are recommended. In larger exercises, each participating organization shou ld be responsible for developing its own specific objectives, which are then incorporated into one exercise package by the design team.

What Makes a “Good” Objective ?

The main thing to remember about objectives is that th ey must be clear, concise, and focused on participant performance. They shou ld contain:

An action, stated in observable terms.

The conditions under which the action will be performed .

Standards (or level) of performance.

In other words, an objective should state who should do what under what conditions according to what standards .

UNIT 4: E XERCISE DESIGN STEPS Page 4.16 Exercise Design Step 4: Define Objectives (Continued) Here’s an example:

Within 15 minutes after the evacuation notice is given, members of the EOC will complete notification procedures to school administrators. Who Specific action Conditions Standards Writing SMART Objectives A useful guideline for writing objectives is the SMART system, which is described below. This system is easy to use and easy to rem ember.

SMART Guidelines for Useful Objectives Simple A good objective is simply and clearly phrased. It is brief and easy to understand.

Measurable The objective should set the level of performance, so that results are observable, and you can tell when the objective has been achiev ed. This doesn’t mean that you have to set a quantifiable standard. It just means that people can agree on whether they succeeded. Achievable The objective should not be too difficult to achiev e. For example, achieving it should be within the resources that the organization is ab le to commit to an exercise.

Realistic The objective should present a realistic expectatio n for the situation. Even though an objective might be achievable, it might not be r ealistic for the exercise.

Task Oriented The objective should focus on a behavior or procedu re. W ith respect to exercise design, each objective should focus on an individua l emergency function.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.17 Activity Activity: Analyze an Objective How good is the following objective? Use the provided r ating scale to analyze the objective. Objective: To demonstrate an understanding of the procedures nece ssary in protecting responder health and safety. Is this objective … Simple? No Needs W ork Yes Measurable? No Needs W ork Yes Achievable? No Needs W ork Yes Realistic? No Needs W ork Yes Task Oriented? No Needs W ork Yes Is this a useful objective? Why? UNIT 4: E XERCISE DESIGN STEPS Page 4.18 Exercise Design Activity: Analyze an Objective (Continued) Suggested Response: The writer of this objective may have known exactly wh at he or she meant, but the statement doesn’t work as an objective because it doesn’t clearly com municate its meaning to anyone else. We are left with many unanswered questions, such a s:

What does “demonstrate” mean?

What would the participant do to properly “demonstrat e?” Who is the person who will demonstrate this understandin g?

What does “complete understanding” mean, and what stand ards would the participant have to reach to prove competence? What specific “procedures” does the writer have in mind?

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.19 Step 4: Define Objectives (Continued) Points of Review Another way to ensure that the objective will provide useful measures is to include specific points of review very specific items to be observed by an evaluator. For example, the objective below is a little too vagu e to guide an evaluation team: Objective: Demonstrate the adequacy of displays and other materials to support emergency operations.

If the objective is supported by points of review simila r to those shown below, it will be adequate, however:

Display Yes No 1. Status boards available in facility 2. Status boards used 3. Status boards kept updated by ________ 4. Maps available 5. Maps up to date UNIT 4: E XERCISE DESIGN STEPS Page 4.20 Exercise Design Step 4: Define Objectives (Continued) Helpful Hints: Word Choice Use concrete words. One way to avoid vagueness is to use concrete words. Pay particular attention to the verb that descr ibes participant performance.

Avoid vague verbs , such as:

Know.

Understand.

Appreciate.

Show the ability to.

Be aware of.

Use action words , such as:

Assess.

Clarify.

Define.

Determine.

Demonstrate.

Establish.

Evaluate.

Examine.

Explain.

Identify.

Inspect.

List.

Notify.

Operate.

Prepare.

Record.

Report.

Show.

Test.

Most objectives written for exercises describe performance that is, the actions that a person or organization will carry out. Occasionally (especially in orientation seminars and some tabletops) t he objective describes the understanding of a concept or a change in at titude.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.21 Activity Activity: Good or Poor Objectives Some examples of objectives are listed below. Review each example. Is it a good objective that will be useful to designers, pl ayers, and evaluators? Or is it a poor objective that needs improvem ent? Mark your answers in the provided boxes. Good Poor 1. Responders will establish an incident command post in t he lobby of the high-rise structure within 15 minutes after the ini tial call for service. 2. To get all of the emergency services to sit down in t he same room together and talk. 3. Demonstrate the ability to field a nuclear response team (fully equipped and assembled) within 30 minutes following a terrorist incident. 4. Identify and activate an alternate communication syst em to be used as a backup within 30 minutes of failure of the primar y communication system. 5. To get the agencies to improve their disaster operat ions.

6. The Mental Health Coordinator in the EOC will con tact and deploy crisis intervention teams to the incident site after noti fication of request. 7. To determine the capabilities of the fire/rescue dep artment to effectively perform fire fighting, rescue, hazardous ma terials containment, and similar hazard abatement duties durin g a major emergency. 8. The warehouse manager will inventory and report t o Central Processing the available stores of protective building sup plies within 2 hours of notification by city officials. 9. To identify the primary reason for slow response of ambulance units. 10. Volunteers will be utilized. 11. Emergency management staff will initiate and compl ete a callback of EOC personnel as prescribed in the emergency plan. UNIT 4: E XERCISE DESIGN STEPS Page 4.22 Exercise Design Activity: Good or Poor Objectives (Continued) Good Poor 12. Demonstrate the ability of the Emmit Internation al Airport Fire Brigade to respond to the farthest portion of the run way within 3 minutes of an alarm. 13. All facility personnel will respond properly to a chemical spill.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.23 Activity Activity: Good or Poor Objectives (Continued) Compare your answers with the following suggested respo nses.

Good Poor 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. UNIT 4: E XERCISE DESIGN STEPS Page 4.24 Exercise Design Activity Activity: Develop Objectives For the same tabletop or functional exercise for which you developed a statement of purpose earlier in this unit, write three objectives and identify the responsible organization. Use the SMART syst em, and remember to include: Action, stated in observable terms.

Conditions.

Standards.

Exercise Objectives Objective #1: Organization:

Objective #2: Organization:

Objective #3: Organization:

(Note: A similar worksheet appears as Job Aid 9 in Appendix A .) UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.25 Step 5: Compose a Narrative An exercise is a scenario that simulates an emergency . Part of this scenario is the narrative, which is a brief description of the events that have occurred up to the minute the exercise begins. The narrative has two important functions. First, it sets the mood for the exercise. Participants need to be motivated to participate. The narrative captures their attentio n and makes them want to go on. Second, the narrative sets the stage for later action by providing information that the participants will need during th e exercise.

Characteristics of a Narrative A good narrative: Is usually one to five paragraphs long.

Is very specific.

Is phrased in present tense.

Is written in short sentences to lend immediacy and tensio n.

May develop the situation chronologically (event with warning time).

May emphasize the emergency environment.

For an emergency with warning time (e.g., a hurricane ), the narrative often outlines the developing situation chronologically . For an unexpected event (e.g., a chemical spill or terr orist bomb attack), the narrative may be shorter. Or, it may devote mor e detail to the environment of the emergency (e.g., a nearby school, o ther chemicals stored, rush hour approaching) to create intensity of fe eling.

UNIT 4: E XERCISE DESIGN STEPS Page 4.26 Exercise Design Step 5: Compose a Narrative (Continued) Hints: Outlining a Narrative You can outline a narrative by jotting down short resp onses (one or two words) to the following questions: What event?

How fast, strong, deep, dangerous?

How did you find out?

What response has been made?

What damage has been reported?

What is the sequence of events?

What time?

Was there advance warning?

Where does it take place?

What are the relevant weather conditions?

What other factors would influence emergency procedures?

What is predicted for the future?

Then, when you’re ready to write the text of the na rrative, just take each of the key words and turn it into a brief sentence. Two sample narratives are provided on the following p ages. As you read the narratives, notice where the questions listed above are answered.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.27 Step 5: Compose a Narrative (Continued) Sample Narrative: Hurricane The National Weather Service’s National Hurricane Cent er issues news on the formation of a storm off the southern U.S. coast that appears to have hurricane potential. Tropical storm Anne is upgraded to Hurricane Anne and NWS issues a Hurricane Watch for a three-state area along the U.S. coast. Wind velocity and northwest movement over the last day have decreased, but an overnight change in direction to a steady northwest line calls for an immediate Hurricane Warning for five coastal counties of the state. Winds of 120 m.p.h. are predicted during the incoming tide, with high water expected to reach 12–15 feet over high tide. Low lying, newly developed resort areas and heavy influx of visiting weekend campers have been advised to evacuate the area. Access bridges to barrier islands are n arrow and could become impassable with 15-foot water heights.

Hurricane Anne, considered a very dangerous hurricane wi th high winds and an accompanying storm surge, will hit the coastal communities along Stevens Bay and farther inland, a population area of between 5,000 and 25,000.

Following the hurricane watch, emergency service personne l notified elected officials and agency heads within the watch area. News media were also alerted and encouraged to broadcast the notice. When the warning of landfall within 24 hours was given, the Emergency Manager placed her staff on alert but did not activate the EOC. She has asked all appropriate emergency service personnel to meet at 07:30, approxima tely four hours after the warning was given. On its present course, the hurricane will make landfall at approximately 23:30. Flood stage from rising tides and tidal surge could, however, impact bridges by 16:00. All appropriate staff and emergency personnel are now gathered in the EOC.

UNIT 4: E XERCISE DESIGN STEPS Page 4.28 Exercise Design Step 5: Compose a Narrative (Continued) Sample Narrative: Air Crash A Boeing 747, en route from Panama to San Francisco, i s experiencing in-flight engine problems and will have to make an emergency landing. Plans have been made to land at a large airport 200 miles north. However, the latest communication with the pilot indicates that the plane has lost engine power and is losing altitude too quickly to reach the large airport. Even though your city airport is too small to handle a 747, you are the only hope for the 350 passengers and 10 crew members.

Conditions at your airport are clear, and the surround ing area is dry. A hot, dry wind is blowing from the north.

The main runway lies along a relatively unpopulated suburban area. However, the likelihood of the pilots being able to control the huge plane and stay within the landing space is slim. The approach passes over populated suburban housing developme nts.

The airport control tower alerts its own Crash/Fire Rescue units and requests that the local emergency services provide backup assistance in fire, police, medical, welfare, and search and rescue capabilities.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.29 Activity Activity: Outline a Narrative For the same tabletop or functional exercise as before, o utline the key points of a narrative using the worksheet below (also pro vided as Job Aid 10 in Appendix A). Just list key words; you do not need to develop the full text at this time. Narrative Outline Event: How fast, strong, deep, dangerous: How you found out: Response made: Damage reported: Sequence of events: Current time: Advance warning: Location: Relevant weather conditions: Other factors that would influence emergency proced ures:

Predictions: UNIT 4: E XERCISE DESIGN STEPS Page 4.30 Exercise Design Step 6: Write Major and Detailed Events Developing an exercise scenario is much like writing a pl ay. In developing a play, the playwright organizes events in to acts and scenes. Similarly, an exercise designer organizes events into ma jor and detailed events. Major and detailed events are occurrences large or small that take place after and as a result of the emergency described in the narrative. It may be helpful to think of them as problems requiring a realistic action that will meet exercise objectives. The goal in developing events is to provide a structure that will:

Link the simulated event to the actions that you want people to take.

Provide unity to the exercise. Without the overall o rganization provided by major events, the exercise could dissolve int o random actions.

Careful scripting is very important if you are going t o produce a convincing, unified scenario rather than a series of unre lated, miscellaneous occurrences. It is also necessary for creating an exercise that is governed by objectives.

Developing Major Events Major events are big problems resulting from the emergency . They should be likely events, based on case studies or operatio nal plans, that call for realistic action. Usually, the best way to arrive at a list of major eve nts is to take it in two stages: First, identify several major occurrences the high points in a sequence that might follow the narrative events.

Second, decide which of these events might generate situa tions that would test the objectives. Then concentrate on those tha t best support the objectives.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.31 Step 6: Write Major and Detailed Events (Continued ) The major events in the following example were devel oped from the sample airplane crash narrative presented in Step 5.

Sample Major Events Sequence for Air Crash Scenario 1. Fuselage breaks apart and hits buildings below. 2. Jet fuel ignites several homes in the area. 3. About 60 survivors are thought to be trapped in th e front section of the aircraft.

4. Several bystanders are injured on the ground. 5. A crowd convenes around the crash site. 6. Family members of victims begin to gather at the cra sh site.

7. Estimates of passenger casualties rise between 200 and 2 20 deaths and at least 70 severe burn victims. Most events require action from one or more organizati ons. Because your goal is to develop an exercise that will test certa in functions and organizations, the major events should be developed fr om your purpose statement.

Example: Event # 1 above tests damage assessment and command and control. Event # 2 tests deployment of fire, police, and medical services. Event # 3 tests search and rescue and medical personnel procedures. UNIT 4: E XERCISE DESIGN STEPS Page 4.32 Exercise Design Step 6: Write Major and Detailed Events (Continued ) Developing Detailed Events Detailed events are specific problem situations to which p ersonnel must respond. Each detailed event should be designed to pro mpt one or more expected actions for one or more organizations that are participating in the exercise. When the design task is small, it may not be necessary to distinguish between major and detailed events. For fu nctional exercises, however, it is easier to write messages if you can base th em on a list of detailed events. There are several ways to develop detailed events. Fo r example, you can:

Plan the detailed events and expected actions at the sam e time. Work backwards first identifying an action that you want players to perform, then listing a problem (a detailed event re lated to a major event) that would motivate the action. Make a list of specific problems that are likely to occur in connection with each major event, then identify actions that woul d be expected as a result.

Whatever method you use, the result should be a list of specific events that are closely linked with actions that you want the participants to perform. The detailed events in the following example were de veloped from event # 7 in the Sample Major Events Sequence (“Estimates of passenger casualties rise between 200 and 220 deaths and at least 7 0 severe burn victims”).

Sample Detailed Events a. The mortuary is unable to accept the large numbers o f remains resulting from the crash.

b. Local hospitals lack specialized facilities and personnel to treat large numbers of severe burn victims.

c. The American Red Cross has agreed to set up a family i nformation center to link victims and their families. UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.33 Step 7: List Expected Actions Expected actions are the actions or decisi ons that you want participants to carry out to demonstrate competenc e. It is necessary to identify expected actions to: Write messages. Because the point of the exercise is to get the participants to think and react in certain ways, the scrip t must be carefully developed to ensure that the messages get the planned results. Your list of expected actions will enable you t o write effective messages. Determine what should be evaluated. The exercise evaluation will focus on whether the participants respond appropriately in an emergency. The list of actions will become the core of t hat evaluation. Types of Actions There are four types of actions that the participants ma y carry out:

Verification: Gather or verify information.

Consideration: Consider information, discuss among players, negotiate, consult plan.

Deferral: Defer action to later, put action on priority list.

Decision: Deploy or deny resources.

How do you know what actions are appropriate in respon se to a given event? Refer to the emergency plan. UNIT 4: E XERCISE DESIGN STEPS Page 4.34 Exercise Design Step 7: List Expected Actions (Continued) Relationship to Objectives Expected actions are closely tied to objectives. Objective s state general desired actions. Expected actions are a breakdown of objectives the actions that would be taken by an organization or an in dividual to meet the objective. The following example illustrates this relationship.

Example: Objective and Expected Actions Function Coordination and communication among the airport an d the jurisdiction’s emergency systems. Objective Upon notification that a crash is imminent, respons e units will stage within 3 minutes, according to SOPs. Event Landing of disabled aircraft is imminent.

Expected Actions Airport Control Tower : Notify police, fire, medical personnel to proceed t o airport.

Alert hospitals of potential mass casualty incident .

Dispatch Center :

Alert police, fire, and medical supervisors.

Hospital :

Notify other medical facilities as appropriate.

Crash Fire Rescue :

Initiate Incident Command System.

Notify dispatch of command post and staging locatio ns.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.35 Step 7: List Expected Actions (Continued) Generally speaking, every detailed event results in on e or more expected actions from various organizations. When you prepare a list of expected actions: List only those that involve the participating organiz ations (those identified in the exercise scope and statement of purpose ).

List expected actions for all exercise participants. (It i s not necessary that each detailed event generate responses from all pa rticipants.) UNIT 4: E XERCISE DESIGN STEPS Page 4.36 Exercise Design Activity Activity: Write Major and Detailed Events First, review the objectives that you developed earlier. With these objectives in mind, write two major events. Then, comple te the Expected Actions Planning Sheet that follows. (This sheet is also provided as Job Aid 11.) Events Major Event #1: Detailed Events:

1. 2. Major Event #2: Detailed Events:

1. 2. Note: Leave the objectives # column blank for now.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.37 Activity: Write Major and Detailed Events (Continu ed) For each detailed event listed on the previous page, ide ntify the expected action, the organization responsible, and the objectives it addresses. Expected Actions Planning Sheet Detailed Event Expected Action Organization Object ives # ( Note: See Job Aid 10 in Appendix A for a similar planning sheet.) UNIT 4: E XERCISE DESIGN STEPS Page 4.38 Exercise Design Step 8: Prepare Messages Messages are used to communicate detailed events to exercise participants. One message may represent an event, or sev eral messages may be needed to notify the participants of the event. Messages serve one purpose: To evoke a response that is, to cause exercise participants to make decisions and take actions tha t meet the exercise objectives. In a full-scale exercise, the controller may input prescri pted messages into the action. Participants receiving the messages make decisions or take action as they would in a real emergency. Transmitting Messages Messages can be transmitted in various ways, including: Landline telephone.

Cellular telephone.

Radio.

In person.

Written note.

Fax.

When transmitting messages in functional or full-scale exe rcises, try to use the method of transmission that would be most likely in an actual emergency. Credibility However messages are delivered, they must come from a cre dible source and be delivered through credible channels. Example: If the participants are in the EOC, a message from Joh n Q.

Public to the EOC from an unlisted number would not b e credible. However, the call could go to dispatch or to someone’s secr etary and the information could then be relayed to the EOC. UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.39 Step 8: Prepare Messages (Continued) Relationship to Expected Actions Messages have a direct relationship to expected actions. E ach message is designed to generate one or more expected actions. T he following example, begun earlier, lists some ideas for messages that could achieve the expected actions.

Example: Objective and Expected Actions Function Coordination and communication among the airport an d the jurisdiction’s emergency systems. Objective Upon notification that a crash is imminent, respons e units will stage within 3 minutes, according to SOPs. Event Landing of disabled aircraft is imminent.

Expected Actions Airport Control Tower : Notify police, fire, medical personnel to proceed t o airport.

Alert hospitals of potential mass casualty incident .

Dispatch Center :

Alert police, fire, and medical supervisors.

Hospital :

Notify other medical facilities as appropriate.

Crash Fire Rescue :

Initiate Incident Command System.

Notify dispatch of command post and staging locatio ns.

Possible Messages Radio call from plane to tower.

Tower calls police, fire, rescue.

Plane requests runway be designated.

Call from hospital requesting information.

Calls to dispatch from media.

Degrading radio communications with plane.

Pilot feels major vibrations/noise on the plane. (Note: Job Aid 12 in Appendix A provides a worksheet for pl anning messages to generate expected actions.) UNIT 4: E XERCISE DESIGN STEPS Page 4.40 Exercise Design Step 8: Prepare Messages (Continued) Message Variables Messages whether simple or complex have four main variables, which are listed in the table below. Not all variables wil l be explicitly stated in every message, but they should be kept in mind as you w rite because these variables form the classic definition of communicatio n:

Who sends what to whom , with what effect.

Message Variables Message source (WHO): W ho sends the message (must be a credible source). Transmission method (HOW): How the message is transmitted (must be a credible means of transmission).

Message content (WHAT): Information conveyed. (Does the message contain the information needed by the recipient to make a decision?) Recipient (TO WHOM): W ho should receive the message. (W ho would credibly receive it, and who ultimately needs to receive it to take action?) All of these variables will influence the action ta ken (to what EFFECT).

Some message examples are given on the next page.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.41 Step 8: Prepare Messages (Continued) Message Examples From: Police To: EOC Police Staff Cab door of trailer truck has been forced open and dri ver has been removed. Shipping papers indicate hydrochloric acid being transpor ted. Acid flowing into sewers. Attempts to open rear trailer door ongoing. From: Environmental Protection (Field) To: EOC Environmental Staff Resident managers of apartments in area request informa tion concerning safety of drinking water, water in swimming pools, and dwell ings after evacuees are allowed to return to homes. From: EOC Fire Staff To: Fire/Rescue Communications Weather Service reports winds in an east-northeast direct ion at 10 to 15 mph with gusts of 20 to 25 mph. Forecast for continued rain with possible thunderstorms with strong gusting winds of up to 45 mph during the storm. Current temperature: 82 degrees. From: Chemical Facility Safety Officer To: Fire Dept. HazMat Team Leader The supervisor responsible for shutting off chemical valve s in a plant area near the fire has not reported in. It is unknown whether the valves have been shut off and whether the person has evacuated. From: Betsy Ames To: Township Fire PIO My name is Betsy Ames. I’m a reporter for the Daily E xpress News. Can you give me a flood situation report for Hamilton, Jordan , Kemper, and the lakes in this area? UNIT 4: E XERCISE DESIGN STEPS Page 4.42 Exercise Design Step 8: Prepare Messages (Continued) Message Format Exercise designers often use a standardized form with space s for the four variables: To, From, Method, and Content. A standar d message form may also have spaces for message number, the time that th e message is to be delivered, and the action to be taken. A sampl e messages format is shown below and appears as Job Aid 13 in Appendix A.

EMERGENCY EXERCISE TO: METHOD FROM:

NO: TIME: CONTENT: _________________________________________ _______ ___________________________________________________ ________ ___________________________________________________________ __________________________________________________________ ACTION TAKEN: ____________________________________ ________ ___________________________________________________ _______ ___________________________________________________ ________ ___________________________________________________________ UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.43 Step 8: Prepare Messages (Continued) Spontaneous Messages The majority of exercise messages in a functional exercise will be prescripted. And in the course of designing a functional exercise, it is a good idea to try to anticipate things that might go w rong and to provide the controller and simulators with ideas about ways to h andle those situations. However, in an actual exercise activity, the participant s don’t always respond as you expect them to. Then, it becomes necessary for the controller and simulators to improvise. Usually, the con troller will decide on an appropriate response. But if the action is inten se, the simulators may also have to make quick decisions and ad lib. If th e controller and simulators are familiar with the scenario and objectives, their spontaneous messages can still fulfill the purposes of the e xercise.

Composing a Message In composing a message, begin with an expected action. Think about who could send a message and what that perso n could say to motivate the expected action. Think about the four message variables:

Who would credibly send the message?

How will the message be transmitted?

Who will receive the message? If not the decision make r, where would the message be redirected?

Does the message provide all of the information neede d to make a decision?

Keep it realistic.

Practice with the messages. Read them through with someo ne who is familiar with the organization involved. Does the m essage motivate the expected action? If it does, then you probably have a successful message.

UNIT 4: E XERCISE DESIGN STEPS Page 4.44 Exercise Design Activity Activity: Compose a Message Select one of the expected actions you listed in the last a ctivity and compose a message that would generate the selected action. You can use the message form provided below. In your message, comp lete the following items: To Method From Content EMERGENCY EXERCISE TO: METHOD FROM:

NO: TIME: CONTENT: _________________________________________ _______ ___________________________________________________ ________ ___________________________________________________________ __________________________________________________________ ACTION TAKEN: ____________________________________ ________ ___________________________________________________ _______ ___________________________________________________ ________ ___________________________________________________________ UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.45 Pulling It Together: The Master Scenario of Events List During a functional exercise, a Master Scenario of Event s list is often used to monitor the progress of the exercise to keep it on schedule and on track. This chart (a list of events, time of occurrence , and expected actions) provides a picture of the whole exercise that is essential to the controller and helpful to simulators. The list should N OT be shown to the participants. ( Note: Job Aid 14 in Appendix A provides an MSEL worksheet. A partial example is shown below.) Sample Master Scenario of Events List Time Message/Event Expected Actions 07:35 Plane radios tower: losing engine power and altitude. 1. Tower notifies dispatch center. 2. Dispatch alerts police, fire, medical to procee d to airport. 07:40–07:50 Pilot reports major vibrations/noise; requests runway designation. 1. Tower designates runway; notifies dispatch of runway and potential for mass casualty incident.

2. Dispatch relays runway to police, fire, medical. 3. Dispatch notifies hospitals.

4. Crash fire rescue initiates ICS; notifies dispatch of CP and staging locations.

5. Dispatch relays CP and staging locations to police, fire, medical. 07:55 Hospital calls dispatch requesting more information. 1. Dispatch obtains potential number of casualties and relays to hospital.

2. Hospital notifies other medical facilities. 08:00 Media call dispatch requesting information. (etc.) UNIT 4: E XERCISE DESIGN STEPS Page 4.46 Exercise Design Alternatives to Self-Developed Exercises In addition to developing exercises as described in this co urse, there are two other ways to prepare for an emergency management exercise:

Exercise-based training courses. FEMA sponsors the Integrated Emergency Management Course (IEMC). Over a period of several days, this course accomplishes at least five important emerg ency management objectives: Agency-specific briefings; minipre paratory exercises; multiagency, real-time exercise; creation of a teamwork environment; and participant critique. Prepackaged exercises. FEMA and many State and local governments have developed a wide variety of exercise packages that may reduce the design time and effort. The avai lable exercise incidents include natural (e.g., weather-related) emer gencies, hazardous materials incidents, terrorist scenarios, and oth ers. These materials must be tailored to fit the specific needs and characteristics of your jurisdiction or organization. Therefore, you should review them carefully and adapt them before using them in yo ur community.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.47 Summary and Transition Unit 4 explained the eight-step design process. The ne xt three units will provide more detailed information about designing an d implementing specific types of exercises: Tabletop, functional, and ful l-scale exercises. Reference Library For More Information Integrated Emergency Management Course: http://training.fema.gov/emiweb/iemc.htm Regional and state FEMA offices:

http://www.fema.gov/about/regoff.htm UNIT 4: E XERCISE DESIGN STEPS Page 4.48 Exercise Design Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer that you have chosen. 1. Which of the following is an example of a good exe rcise objective?

a. To improve the number of accurate messages transmitted by the police to the communications center.

b. At the time the evacuation notice is received, the E OC policy and coordination groups will examine the needs of schools and other special facili ties and organize notification according to SOPs.

c. Proper procedures to declare a disaster or ask for outsi de aid will be taken.

d. Interaction with other jurisdictions will be demonstr ated.

2. An objective should specify the level of performance, so that results are observable.

a. True b. False 3. Which of the following verbs would be most useful in an objective?

a. Know b. Be aware of c. Appreciate d. Notify 4. The narrative: a. Sets the stage for later action.

b. Lists all of the events that will occur in the exercise .

c. Provides the master scenario of events list and expected actions.

d. Explains the objectives of the exercise. 5. The scope: a. Sets the mood for the exercise.

b. Establishes the limits of the exercise.

c. States the objectives of the exercise.

d. Lists the financial and human resources required to ru n the exercise.

UNIT 4: E XERCISE DESIGN STEPS Exercise Design Page 4.49 Knowledge Check (Continued) 6. A convincing, unified scenario requires: a. 10 major events and approximately 100 detailed ev ents.

b. Messages written in a way that will not allow unexp ected responses to occur.

c. The players having knowledge of the events in advan ce.

d. Careful scripting of events. 7. Every major event should require an action from al l participants involved in the exercise.

a. True b. False 8. Deferring action until later might be a valid acti on to take in response to a message.

a. True b. False 9. In most exercises, the best way to deliver messages is in writing, using a standard form.

a. True b. False 10. A ________________ is a useful tool to help the co ntroller keep the exercise on track and on schedule.

a. Message form b. Narrative c. Master scenario of events list d. Exercise directive UNIT 4: E XERCISE DESIGN STEPS Page 4.50 Exercise Design Answers Knowledge Check (Continued) 1. b 2. a 3. d 4. a 5. b 6. d 7. b 8. a 9. b 10. c Unit 5: The Tabletop Exercise UNIT 5 THE TABLETOP E XERCISE Exercise Design Page 5.1 Introduction Now that you have the “big picture” of the exercise pr ocess and the steps in designing an exercise, you’re ready to take a closer look at specific kinds of exercises. This unit focuses on the tabletop exer cise. First, we’ll review how a tabletop exercise works and t he role of the facilitator. At some point you will probably be calle d upon to serve in this role, so we will discuss some guidelines for successfully facili tating a tabletop exercise. Finally, we’ll examine how the de sign steps you learned in the previous unit are applied to this type of exercise.

Unit 5 Objectives After completing this unit, you should be able to: Describe the purposes and characteristics of a tabletop exer cise.

Describe the steps in facilitating a tabletop exercise.

Characteristics of the Tabletop Exercise As learned in Unit 2, a tabletop exercise simulates an e mergency situation in an informal, stress-free environment. The participants usually people on a decision-making level gather around a table to discuss general problems and procedures in the context of a n emergency scenario. The focus is on training and familiarization with roles, procedures, or responsibilities.

UNIT 5: T HE TABLETOP EXERCISE Page 5.2 Exercise Design Characteristics of the Tabletop Exercise (Continued ) Purpose The tabletop is largely a discussion guided by a facilita tor (or sometimes two facilitators who share responsibilities). Its purpose is to solve problems as a group. There are no simulators and no at tempts to arrange elaborate facilities or communications. One or two eval uators may be selected to observe proceedings and progress toward the ob jectives.

The success of a tabletop exercise is determined by feedback from participants and the impact this feedback has on the eval uation and revision of policies, plans, and procedures. Advantages and Disadvantages The tabletop exercise is a very useful training tool th at has both advantages and disadvantages, as summarized in the follo wing table.

Advantages and Disadvantages of Tabletop Exercises Advantages Requires only a modest commitment in terms of time, cost, and resources.

Is an effective method for reviewing plans, procedu res, and policies.

Is a good way to acquaint key personnel with emerge ncy responsibilities, procedures, and one another. Disadvantages Lacks realism and thus does not provide a true test of an emergency management system’s capabilities.

Provides only a superficial exercise of plans, proc edures, and staff capabilities.

Does not provide a practical way to demonstrate sys tem overload.

UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.3 How a Tabletop Works In many respects, a tabletop exercise is like a problem-so lving or brainstorming session. Unlike a functional exercise, prob lems are tackled one at a time and talked through without stress. Problem Statements and Messages A tabletop is not tightly structured, so problem stateme nts can be handled in various ways: The facilitator can verbally present general problems, which are then discussed one at a time by the group. Problems can be verbally addressed to individuals first a nd then opened to the group. Written detailed events (problems) and related discussion questions can be given to individuals to answer from the perspecti ve of their own organization and role, then discussed in the group. Another approach is to deliver prescripted messages to pla yers. The facilitator presents them, one at a time, to individua l participants. The group then discusses the issues raised by the message, using th e EOP or other operating plan for guidance. The group determines what, if any, additional information is needed and r equests that information. They may take some action if appropriat e.

Occasionally, players receiving messages handle them indivi dually, making a decision for the organization they represent. Players then work together, seeking out information and coordinati ng decisions with each other.

Some facilitators like to combine approaches, beginning the exercise with general problems directed to key individuals and then passing out messages one at a time to the other players.

Handling Problems: It is usually wise to take the time to resolve problems, rather than hurry from one problem or messag e to the next, even though players sometimes will want to bypass the to ugh problems.

UNIT 5: T HE TABLETOP EXERCISE Page 5.4 Exercise Design How a Tabletop Works (Continued) Facilities and Materials It is recommended that the EOC or other operations cent er be used for the tabletop exercise, for two reasons: It provides the most realistic setting.

Needed plans, displays, and maps are available on the p remises.

Any conference facility that will comfortably accommodate the expected number of participants in a face-to-face setting will be adequate, however. The number of participants and the scenario will determ ine the number and arrangement of tables for the exercise. Some facil itators like to arrange small groups around separate tables. Others pre fer a U-shaped layout. Provided reference materials should include emergency pl ans, maps, and other reference materials that would normally be avai lable in the EOC.

Facilitating a Tabletop Exercise A tabletop exercise provides a relaxed environment of team problem solving. While functional and full-scale exercises are in teractive, a tabletop is managed by a facilitator. The facilitator has a number of responsibilities, including:

Setting the stage (e.g., introducing the narrative).

Involving everyone—stimulating discussion and drawing answ ers and solutions from the group (rather than supplying them). Facilitating in-depth problem solving.

Controlling the pace and flow of the exercise and distr ibuting messages.

UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.5 Facilitating a Tabletop Exercise (Continued) The facilitator must have good communication skills and b e well informed on local plans and organizational responsibilities. Alt hough the facilitator can be thought of as a discussion leader, the role can be much more broad. The following are some guidelines for facilita ting a tabletop exercise.

Setting the Stage The opening remarks and activities influence the whole experience. Players need to know what will happen and to feel com fortable about being there. Below are some guidelines for setting t he stage for a successful tabletop exercise. Guidelines for Setting the Stage Welcome. Begin by sincerely welcoming participants and putting them at ease. Briefing. Brief the participants about what will happen. This includes a clear explanation of:

Purposes and objectives.

Ground rules.

Procedures.

Narrative. Start the exercise by reading (or having someone read) the narrative and introducing the first problem or me ssage.

Ice Breaker. Try breaking the ice by beginning with a general question directed at one or two high-ranking officials or to the group as a whole. Later, other problem statements or messages can be addressed to other individuals or organizations. UNIT 5: T HE TABLETOP EXERCISE Page 5.6 Exercise Design Facilitating a Tabletop Exercise (Continued) Involving Everyone It is important that everyone participates and that no one person or organization dominates the discussion. Tips for involvin g all of the participants are summarized below. Ways to Involve All of the Participants Organize the messages so that all organizations must deal with a question or problem. Give extra encouragement to those who are a little re ticent.

Avoid the temptation to jump in with the right solut ions when players are struggling. This will often hamper the discussion. Instead, try to draw out the answers from the players. They will be m ore likely to participate if they feel people are listening intentl y and sympathetically. Model and encourage the behaviors you want from the p articipants: Give eye contact.

Acknowledge comments in a positive manner.

In-Depth Problem Solving The purpose of tabletop exercises is usually resolving pro blems or making plans as a group. That means going after real solutions not superficialities. Some facilitators make the mistake of trying to move t oo fast through the scenario, believing that they have to meet all of the objectives and get through all of the messages. However, that is not a go od approach if nothing gets settled. Remember: If you spend all the time on one big prob lem, maintain interest among players, and reach consensus, then the tabl etop is a success! Push the players past superficial solutions . A few carefully chosen, open-ended questions can keep the discussion going t o its logical conclusion. UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.7 Facilitating a Tabletop Exercise (Continued) Controlling and Sustaining Action To maintain a high level of interest and keep everyo ne involved, the facilitator needs to control and sustain the action. Th ere are several ways to do this. Ways to Control and Sustain Action Use multiple event stages. Develop the scenario narrative in event stages. (For example, the initial narrative may invo lve warning. A later one could deal with search and rescue.) Then, as d iscussion begins to fade on one issue, introduce the next segment. Vary the pace. Add or delete problem statements and messages to alter the speed of the action. Occasionally give two me ssages at the same time to increase pace and interest. Maintain a balance. Maintain a balance between overly talking about a problem to death and moving along so fast tha t nothing gets settled. Don’t hesitate to control the exercise tightly !

Watch for signs of frustration or conflict. Always remember that the tabletop is basically training, not testing. Peopl e may come with fragile egos and little exercise experience. If you see mounting frustration or conflict, stop the exercise. Reach into yo ur experience as a discussion leader to help the players resolve conflicts and feel comfortable. Keep it low-key. Avoid a bad experience by keeping in mind the low-key nature of the tabletop. UNIT 5: T HE TABLETOP EXERCISE Page 5.8 Exercise Design Activity Activity: Your Ideas for Facilitating a Tabletop You have read about many techniques for facilitating a t abletop exercise. But group facilitation styles and techniques are as varied as the facilitators who use them, and you may have some additiona l ideas about group techniques that would help you facilitate a successful tabletop exercise. Jot your ideas below. Setting the Stage: Involving Everyone: In-Depth Problem Solving: Controlling and Sustaining Action: UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.9 Designing a Tabletop Exercise The eight-step process outlined in Unit 4 is used to desig n a tabletop exercise:

1. Assess needs. 2. Define the scope. 3. Write a purpose statement. 4. Define objectives. 5. Compose a narrative. 6. Write major and detailed events. 7. List expected actions. 8. Prepare messages.

You can use the job aids provided in Unit 4. For a ta bletop exercise, however, the process can be somewhat simplified. Because a tabletop is only partially simulated, it requires little scripting. The only roles are the facilitator, the participants (who respond in their rea l-life roles), and one or two recorders. Recorders take minutes and record decisions and usually do not need formal evaluation forms.

Applying the Design Steps The first four steps are handled just as described in Unit 4. The remaining steps can be simplified as follows: Narrative: The tabletop narrative is sometimes shorter. It is ne arly always given to the players in printed form, although it can be presented on TV or radio. When the purpose of the ta bletop is to discuss general responses, the narrative can be presented in parts, with a discussion of problems after each part.

UNIT 5: T HE TABLETOP EXERCISE Page 5.10 Exercise Design Designing a Tabletop Exercise (Continued) Events: The events should be closely related to the objectives of the exercise. Most tabletop exercises require only a few maj or or detailed events, which then can easily be turned into p roblem statements. Expected actions: A list of expected actions is useful for developing both problem statements and messages. It is always import ant to be clear about what you want people to do. However, in a tabletop, sometimes the “expected action” will be a discussion that w ill eventually result in consensus or ideas for change.

Messages: A tabletop can succeed with just a few carefully written messages or problem statements. As always, messages should be closely tied to objectives and should be planned to give all participants the opportunity to take part. The messages might relate to a large problem (almost li ke an announcement of a major event) or a smaller problem, depending on the purpose of the exercise. Usually they are directed to a single person or organization, although others may be invite d to join in the discussion.

Message Example General Problem Statement: During a relocation process, what do you feel is needed for the support of your functions? Specific Message: A call from Southside Nursing Home: They do not have enough transportation for all of their patients. How many messages? It is a good idea to write a few more messages than you think you will need. If messages are ca refully thought through, however, they will create a rather lengthy discussion. It’s better to have 10 or 15 good messages tha n 20 or 30 hastily written ones. (Note: A job aid for designing and facilitating a tabletop exercise is provided on the next page and also appears as Job Aid 1 5 in Appendix A.) UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.11 Designing a Tabletop Exercise (Continued) Tabletop Exercise Checklist Design Needs assessment, scope, statement of purpose, and objectives developed Narrative:

May be shorter Presented all at once or incrementally Events:

Limited number Presented as problem statements Expected actions:

May involve identification of appropriate responses, identification of gaps in procedures, reaching group consensus, developing ideas for change, etc. Messages:

Limited number (e.g., 10-15) Involve everyone Tied to objectives Facilitation Welcome participants Briefing:

Purpose and objectives Ground rules and procedures Narrative presentation (printed, verbal, TV, radio) Ice breaker questions directed at high-ranking officers Messages organized to involve all organizations Strategies to encourage the reticent Facilitate don’t dominate Model positive behaviors (eye contact, positive reinforce ment) Aim for in-depth problem solving Strategies for sustaining action Multiple event stages Varied pace Balanced pace Conflict resolution Low-key atmosphere UNIT 5: T HE TABLETOP EXERCISE Page 5.12 Exercise Design Designing a Tabletop Exercise (Continued) Exercise Examples Examples of two approaches to tabletop exercises “scenario development” and “single narrative with messages” are provided in the activities that follow. Review the plans carefully for similarities and differences. A third approach is illustrated in the sample tabletop exercise provided in the Exercise Design Tool Box in the “Samples” Directory. In this exercise program, local governments in a jurisdiction conduct concur rent tabletop exercises and report their actions via situation reports, Emergency Management Exercise Reporting System (EMERS) forms, and Participant Narrative Summary forms. The local exercises are follow ed by a regional exercise based on outcomes from the local exercises.

UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.13 Activity Activity: Develop Problem Statements The following example illustrates one tabletop exercise d esign approach the “scenario development” approach. The goal of this exercise (based on an earthquake scenario) is to generate g eneral solutions to problems. To accomplish this, it presents the narrative in sections, with each section followed by a few problem state ments. The facilitator would have participants discuss each problem stat ement until reasonable solutions have been reached, then proceed to t he next narrative section and its accompanying problem statements. Review the sample plan. Problem statements for the thi rd narrative section have been omitted. In the spaces provided, develop two problem statements related to the third narrative section that w ould help achieve the objectives of the exercise.

Tabletop Plan: Example 1 (Scenario Development App roach) Organization: Department of Management Services Objectives: 1. Heighten awareness of city plan and standard ope rating procedures.

2. Identify and prioritize response activities. 3. Identify plans, policies, and procedures which a re specific for this department and its various divisions.

4. Identify resources available to the department t eam.

5. Identify critical operations. Participants: Department Level Department of Management Services; Department Direc tor Assistant to Mayor City Treasurer Division Level Division of Human Resources Management Division of Labor Relations Division of Accounting Division of Budget Treasurer Division of Purchasing City Recorder UNIT 5: T HE TABLETOP EXERCISE Page 5.14 Exercise Design Activity: Develop Problem Statements (Continued) Tabletop Plan: Example 1 (Continued) Scenario Development 1 (Narrative, Part 1) It is 9:00 a.m. on Tuesday, January 15. You have j ust felt a moderate earthquake as you worked at your desk in the City Office Building. Y ou hide under your desk as plaster and glass flew around your office. After the shaking h ad stopped, you left the building from the nearest exit. Everyone is shocked, but apparen tly there are no serious injuries. You see several cracks in the rock fascia and wonder wh ether it is safe to reenter the building. Your assistant comes up to you and tells you that he could not get a dial tone on his phone before exiting the building. Near pan ic, he asks these questions: “W hat do we do? How do we get help? How do we call the eme rgency command center? How bad do you think the quake is?” It occurs to you that these are questions that need to be answered.

Problem Statements: A. As Division heads, what are our primary and seco ndary priorities and responsibilities? W hat are the time constraints on these activities?

B. As a Department Command Team, what are our prior ities and responsibilities?

C. How is this department prepared to sustain divis ion support/activity throughout a lengthy response? W hat about shifts for the comman d post members?

Scenario Development 2 (Narrative, Part 2) Because of possible structural damage to the buildi ng, the group decides to relocate the Department Command Post to ______________. It appe ars that the earthquake was not too severe; however, it will require a serious effo rt to keep the department services available and respond to the city’s needs. Since t he City Command Post is expecting the department to be located at the City Office Buildin g in Roger Brown’s office, it is necessary to discuss what to do and how to do it. Problem Statements: A. If there is damage to this building, how do we s elect where we will relocate?

B. Roger Brown, at the City EOC, has to know of you r whereabouts. How do you contact him at the City EOC to let him know where y ou are relocating? If radios are used, where did they come from? Are they operation al?

C. During a relocation process, what do you feel as a team is needed for the support of your functions (administrative support, personnel, facilities, etc.)?

UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.15 Activity: Develop Problem Statements (Continued) Tabletop Plan: Example 1 (Continued) Scenario Development 3 (Narrative, Part 3) It is now 1½ days since the earthquake. None of yo u have been able to go home. Because of debris removal, all of the city response agencies, including public works, have been working almost nonstop. You hear of a st reet worker who was injured falling out of a tree while cutting down a broken branch. Also, last night a secretary complained that she was doing purchasing/contract procurement while only being p aid as a secretary. She joked with her supervisor, but Frank was alerted to this concern by the supervisor for policy guidance. Problem Statements: (Develop your problem statements in the spaces be low.) A. B. Note:

The scenario may continue through two or three more problem areas. The problem statements need not be addressed in sequenc e. UNIT 5: T HE TABLETOP EXERCISE Page 5.16 Exercise Design Activity: Develop Problem Statements (Continued) Suggested Answers A wide variety of problem statements could be develope d based on the provided narrative section. Below are two examples. Yours will probably be different, but they should reinforce the exercise ob jectives given at the beginning of the example. Examples of Problem Statements: A. Does this body have the power to make policy decision s with regard to contract violation, shift problems for extended respo nse periods, and injuries related to personnel performing functions outside their normal duties? Have liability issues been fully addresse d?

B. What plan is in place to assist city responders (this gro up included) with family information assistance to reduce the stress of employees not knowing how their families are?

UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.17 Activity Activity: Develop a Tabletop Exercise Message The following example illustrates another tabletop exer cise design approach a single narrative with messages. This plan focuses the participants on more specific responses to problems through the use of printed messages. The facilitator would present the ent ire narrative, then distribute the messages to specific players for discussion. Review the plan. Imagine that your organization is included among the players. In the space provided, develop a message related to a communications issue that would affect your organization’s ability to meet the stated objectives of the exercise. Tabletop Plan: Example 2 (Message Approach) Objectives: Participants will demonstrate the ability to: 1. Coordinate effective evacuation.

2. Establish and maintain shelters for evacuees.

3. Coordinate round-the-clock field operations.

4. Ascertain safety levels needed to allow reentry into affected areas.

Players: City/County Manager Emergency Manager Law Enforcement Representative Fire Services Representative Health/Environmental Representative Public W orks Director Public Information Officer Volunteer Shelter Coordinator Narrative: Steady rain has been falling in the region due to a moisture-laden low pressure system. Early into the storm, a Flood W atch was issued by t he National W eather Service (NW S). W ithin 10 hours after the rain began to fall, the N W S issued a Flood W arning for low-lying areas along small streams. Some roads had to be cl osed and a small number of residents had to be evacuated along overflowing str eams located northwest of the city.

Sixteen hours have elapsed since the rain began. B ased on reports from designated citizens with rain gauges, total rainfall across th e jurisdiction ranges from 4.1 to 4.7 inches.

The soil has reached the saturation point, causing extensive runoff. Designated river spotters, water level gauges, and/or electronic riv er rise warning devices indicate that the river has risen to a dangerous level. The 24-hour forecast calls for continued rainfall, with a clearing trend possible. Based on these data, the NW S issues a Flood W arning for area s along the river basin. Police report additional road closings due to high water, and a couple dozen more residents have been evacuated. W ater levels are rising steadily, as a driving rain continues to fall. It is now 3:00 p.m. The EOC has been opened and the staf f has been assembled. UNIT 5: T HE TABLETOP EXERCISE Page 5.18 Exercise Design Activity: Develop a Tabletop Exercise Message (Con tinued) Tabletop Plan: Example 2 (Continued) Messages: Message #__1__ Time 4:15 pm To: Public W orks From: W ater Treatment Plant W ater continues to rise. River approaching flood stage. Debris is piling up at bridges.

W ater is now to top step of plant. Message # 2_ Time 5:00 pm To: Police Chief From: Police Officer Motorists stranded in high water along Riverside Drive, south of bridge. They are on top of their cars in about 3 feet of water. The water is rising and moving faster. Message #__3__ Time 5:30 pm To: Fire Chief From: Battalion Chief House on Nelson Drive contains 6 children under age 4, babysitter is only 11. They are in an upstairs bedroom. W ater is over 2 feet deep in house. The sitter refuses to let the kids leave until the mother returns. Message # __4__ Time 6:45 pm To: Public W orks From: Public W orks Crew Heart attack victim in River Oaks. Ambulance cannot get to him because of high water. Message #__5__ Time 7:30 pm To: Dispatch From: Citizen Power line down near the Riverside City Park. One person appears dead, at least 3 others trapped in vehicles. W ater is rising rapidly. Message # __6__ Time 7:30 pm To: Emergency Manager From: Dispatch Switchboard and dispatch are overwhelmed. W e are short of replacement personnel. Message # __7__ Time 8:45 pm To: Volunteer Coordinator From: Incident Coordinator Emergency workers have been on job for many hours. They are in need of food and refreshments. Message #__8__ Time 11:00 pm To: Public Information Officer From: Public Utilities Many parts of the city have lost power and telephone service, including many city agencies. Utilities do not know which agencies need repair first. Setting priorities is difficult. UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.19 Activity: Develop a Tabletop Exercise Message (Con tinued) Tabletop Plan: Example 2 (Continued) Message #__9__ Time 11:30 pm To: City Manager From: Incident Commander Emergency personnel have been working for many hours and are exhausted. Message #__10__ Time 1:00 am To: Volunteer Shelter Coordinator From: Red Cross Shelters are running low on supplies, especially food. Evacuees have forgotten to bring vital medications. Message #__11__ Time 3:30 am To: Health/Environment Representatives From: Policeman Flood waters recede. Some residents are wanting to get back in their homes. Some lack proper ID. Some areas are still too dangerous.

Message #__12__ Time 9:00 am To: Health/Environment Representatives From: Citizen There is a shortage of potable water. Your Message: Message # ____________ Time: _________________ To: From: Message: UNIT 5: T HE TABLETOP EXERCISE Page 5.20 Exercise Design Summary and Transition This unit provided an overview of the tabletop exerci se: how it works, how to facilitate one, and how to design one using the eight design steps.

Unit 6 will provide a similar overview of the functio nal exercise. Reference Library For More Information FEMA’s Emergency Management Exercise Reporting System (EMERS) www.fema.gov/onp/emers FEMA: Descriptions of past tabletop exercises: Las Cruces counter-terrorism exercise: http://www.fema.go v/reg- vi/2001/r6_24.htm Pacific Northwest terrorism workshop: http://www.fema.go v/reg- x/2001/r10_36.htm Virtual tabletop exercise (Internet): http://www.fema.gov/pte/98365.htm Red River Basin flooding exercise: http://www.fema.gov /reg- viii/premac.htm Tabletop exercises participated in by FEMA: www.fema.gov . Type in “tabletop” under search FEMA.gov and click “Go”.

U.S. Department of Energy/Oak Ridge Associated Universi ties, Emergency Management Laboratory: Design course: Beyond the Traditional Tabletop Exercise : www.orau.gov/eml/manage.htm What Would You Do If It Happened to You? A Tableto p Exercise Gives You a Chance to Find Out: www.orau.gov/eml/exercise .htm UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.21 Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer that you have chosen. 1. A tabletop exercise: a. Involves a controller, players, and evaluators.

b. Is an informal discussion guided by a facilitator.

c. Requires a formal evaluation to determine its success.

d. Must achieve all of the stated objectives to be conside red a success.

2. A tabletop exercise: a. Provides a true test of an emergency management syste m’s capabilities.

b. Provides a practical way to demonstrate system overloa d.

c. Is an effective method for reviewing plans, procedure s, and policies.

d. Is better suited for field operations personnel than planners and policy makers.

3. In a tabletop exercise, problems can be addressed to i ndividuals, the group, or both.

a. True b. False 4. It is more important to engage in in-depth proble m solving than to meet all objectives and get through all of the problem statements.

a. True b. False 5. If possible, a tabletop exercise should be run: a. At several field locations to simulate a real emerge ncy. b. At a selected field location where an incident might occur.

c. At a central location such as an operations center.

d. As a teleconference. 6. An important part of the facilitator’s job is to: a. Maintain an even pace and consistent approach.

b. Adhere to a highly structured agenda.

c. Sustain action and keep everyone involved.

d. Make sure the entire set of problem statements is discu ssed.

UNIT 5: T HE TABLETOP EXERCISE Page 5.22 Exercise Design Knowledge Check (Continued) 7. Most tabletop exercises: a. Require 50 to 100 detailed events.

b. Are designed without identifying expected actions.

c. Use general discussion in place of problem statements and messages. d. Require only a few major or detailed events. 8. The first four steps of the eight-step design process can be omitted when developing tabletop exercises.

a. True b. False 9. Introducing multiple event stages is a good way to k eep the action moving in a tabletop exercise.

a. True b. False 10. A tabletop exercise should be: a. Low-key.

b. Highly stressful.

c. Formal and highly structured.

d. Narrowly focused on field operations. UNIT 5: T HE TABLETOP EXERCISE Exercise Design Page 5.23 Answers Knowledge Check (Continued) 1. b 2. c 3. a 4. a 5. c 6. c 7. d 8. b 9. a 10. a Unit 6: The Functional Exercise UNIT 6 THE FUNCTIONAL E XERCISE Exercise Design Page 6.1 Introduction This unit focuses on the functional exercise. We will loo k closely at the characteristics of the functional exercise how it differs from the tabletop, who participates, how it works, and key design considerat ions. This unit is pivotal, because later in the course you will develop a functional exercise based on what you have learned here. Unit 6 Objectives After completing this unit, you should be able to: Describe the purpose and characteristics of a functional exe rcise.

Explain how designing a functional exercise differs from designing a tabletop exercise.

Describe the physical requirements and participant roles i n a functional exercise.

UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.2 Exercise Design What Is a Functional Exercise?

The functional exercise simulates an emergency in the most realistic manner possible, short of moving real people and equip ment to an actual site. As the name suggests, its goal is to test or evaluat e the capability of one or more functions in the context of an emergency event.

It is important not to confuse “functional exercises” with emergency “functions.” All exercises (tabletop, functional, and fu ll-scale) test and evaluate functions contained in the Emergency Operation s Plan (EOP). In this course, “functions” refers to actions or operations required in emergency response or recovery. The thirteen functions r ecognized by FEMA were introduced earlier, in Unit 1, are:

Alert Notification (Emergency Response).

Warning (Public).

Communications.

Coordination and Control.

Emergency Public Information.

Damage Assessment.

Health and Medical Individual/Family Assistance.

Public Safety.

Public Works/Engineering.

Transportation.

Resource Management.

Continuity of Government.

The key characteristics of functional exercises were discussed i n Unit 2. You may wish to refer back to that discussion now. Below is a brief summary of the main points.

Key Characteristics Interactive exercise, designed to challenge the entire e mergency management system. Can test the same functions and respon ses as in a full-scale exercise without high costs or safety risks. Usual ly takes place in an EOC or other operating center.

Involves controller(s), players, simulators, and evaluato rs.

The atmosphere is stressful and tense because of real-time action and the realism of the problems.

Exercise is lengthy and complex; requires careful scripting , careful planning, and attention to detail.

Geared for policy, coordination, and operations personn el (the players).

Players practice their response to an emergency by respond ing in a realistic way to carefully planned and sequenced messages given to them by simulators.

Messages reflect a series of ongoing events and problems.

All decisions and actions by players occur in real time and generate real responses and consequences from other players. Guiding pri nciple: Imitate reality.

UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.3 What Is a Functional Exercise? (Continued) Best Uses The functional exercise makes it possible to test the same functions and responses as would be tested in a full-scale exercise, witho ut the high costs or safety risks. The functional exercise is well-suited to assess the:

Direction and control of emergency management.

Adequacy of plans, policies, procedures, and roles of indi vidual or multiple functions.

Individual and system performance.

Decision-making process.

Communication and information sharing among organizat ions.

Allocation of resources and personnel.

Overall adequacy of resources to meet the emergency situa tion.

UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.4 Exercise Design Activity Activity: Compare Tabletop and Functional Exercise s In the following table, compare tabletop and function al exercises by writing a brief description in each of the cells. Tabletop Functional Degree of Realism Format/Structure Atmosphere Who Takes Part Who Leads Where Held Equipment Deployed Test Coordination Test Adequacy of Resources Test Decision-Making Process Relative Complexity/Cost Formal Evaluation UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.5 Activity: Compare Tabletop and Functional Exercise s (Continued) Suggested Answers :

Tabletop Functional Degree of Realism Lacks realism As realistic as possible without deploying resources Format/Structure Group discussion, based on narrative and problem statements/messages Interactive; simulators deliver “problem” messages, players respond in real time Atmosphere Low-key, relaxed Tense, stressful Who Takes Part Facilitator, participants (decision- making level); may use recorders Controller, players (policy, coordination, and operations personnel), simulators, evaluators Who Leads Facilitator Controller Where Held EOC, other operations center, or conference room EOC or other operations center Equipment Deployed No No Test Coordination Yes, on a discussion level Yes Test Adequacy of Resources No Yes Test Decision-Making Process Yes Yes Relative Complexity/Cost Small group; simple format; modest cost Large scale; complex format; moderate cost to design and implement (higher than tabletop, lower than full-scale) Formal Evaluation No (self-assessment by participants) Yes UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.6 Exercise Design Participant Roles As noted earlier, the functional exercise involves playe rs, simulators, a controller, and evaluators. In a small jurisdiction or organization, one or two people may serve as controller, simulator, and eval uator. In larger jurisdictions, many more people will be necessary. Let’s take a closer look at what is involved in each rol e and how participants are selected.

Players The players in a functional exercise are people who hol d key decision- making or coordinating positions and would normally fu nction in the operations center. By operations center, we mean the central location that is designated in a real emergency for policy decisions, coordination, control , and overall planning. For a governmental jurisdiction, it would be the EOC; for a volunteer agency or private sector entity, it would be the central location from which key decision makers operate in an emergency si tuation. Decision makers. Key decision makers would normally include leaders in government and key responding organizations: The mayor or other chief executive, and chiefs and coordinators of emergency services such as Fire, Police, EMS, Public Information Officer (PIO), and so on. In a nongovernmental organization, the CEO and other org anizational leaders would participate. Coordination and operations. Serving in the coordination and operations groups are people from various departments w ho work with policy makers. In large exercises, a separate operations group carries out directives. In small exercises, the coordination and operations roles may be taken by the policy makers. The best guide in selecting who should participate in an exercise is the emergency plan. Duties. The only job of the players is to respond as they wou ld in a real emergency to the messages that they receive during the e xercise. All of the decisions and actions of the players take place in rea l time and generate real responses and consequences from other player s.

UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.7 Participant Roles (Continued) Simulators To create a real-life environment, simulators portray the organizations that would normally interact with the players in the operations center. They do this by delivering messages descriptions of events or problems which require players to act. Some messages are scripted in advance; others are spontaneo us responses to player decisions. They are input into the e xercise by means of radio or telephone, or by written notes simulating radio and telephone transmissions. Duties: Simulators are responsible for all actions taken by orga nizations or individuals outside of the EOC. They: Send the players prescripted messages representing private citizens, agencies, or other organizations, according to scheduled t imes in the sequence of events. Simulate all actions taken by an agency or other organ ization.

Ad lib spontaneous messages as needed. Examples of times w hen a simulator may need to respond spontaneously include:

When a member of the operations center issues a directiv e that results in events not anticipated in the scenario.

When a player asks for more information.

When a player decision is not logically linked to the next event in the scenario. Inform the controller of any deviations from the scenar io, or special problems.

When simulators are given directives, they are required to follow through and implement the directives in a professional manner.

UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.8 Exercise Design Participant Roles (Continued) Selection: Simulators must be able to ad lib intelligently in th e situations just described, so it is important that they be familiar with the organization(s) that they are simulating and with the sequence of events and messages. It is useful, therefore, to draw simulator s from the organizations that they will portray, and/or from th e design team.

Numbers: It is difficult to give a rule of thumb concerning specif ic numbers of simulators needed for an exercise. The numbe r of simulators will vary according to the:

Number of players.

Length of the exercise.

Knowledge and training of the simulators.

Communication channels available.

For best results, try to have at least one simulator per organization represented in the operations center, with extras to pl ay the part of citizens or other private organizations. Organizing: It is a good idea to group simulators according to funct ion, to simplify the exercise and reduce the number of simula tors needed. One approach is to organize them into three groups: Government agencies not participating in the exercise.

Participating organizations: Field units of organizat ions participating in the exercise (police, fire, public works, etc.) and pr ivate medical and support organizations.

Other private facilities and individuals: Citizens and nongovernment organizations.

The following table illustrates how this approach could be used for a community. UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.9 Participant Roles (Continued) Nonparticipating Government Entities Participating Organizations Other Private Facilities/ Individuals One or two persons simulating: Federal regulators.

State or state area EOC.

County EOC.

Other city EOC.

State/Federal officers.

Care and shelter.

Resources and support.

One person per organization simulating: City departments and agencies.

County departments.

Medical/health services.

Volunteer organizations. One or two persons simulating: Industries.

Commercial business.

Media.

Private citizens. Controller The controller supervises the simulation or overall condu ct of the exercise, making certain that it proceeds as planned and that objectives are reached. The controller must be able to view the exercise as a wh ole and to think quickly on his or her feet. Players often make unanti cipated decisions, and the controller must be able to respond to these. Duties: The main duties of the controller are to:

Ensure that the simulators and evaluators are properly trained before the exercise. Orient the participants to the exercise and present the narrative.

Monitor the sequence of events and supervise the input o f messages, using the Master Scenario of Events List as a guide. Adjust the pace of the exercise when needed inserting more messages when it drags and discarding messages when the pace is too frantic. Make decisions in the event of unanticipated actions or r esource requirements.

Maintain order and professionalism throughout the exer cise.

UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.10 Exercise Design Participant Roles (Continued) Selection: Controllers can usually be drawn from the exercise desi gn team. Because the team members are already familiar w ith the exercise, they are well suited to the task of keeping the exerci se moving toward the anticipated conclusion.

Preparation: To properly prepare for the event, the controller should have the following items available: List of objectives.

Master Scenario of Events List.

Messages.

List of players.

List of resources available to the jurisdiction or organi zation.

It is usually helpful to hold a briefing before the e xercise to orient the staff members. At the briefing, the controller should train the simulators, ensuring that they are familiar with the scenario, obj ectives, resources, and the messages they will be responsible for delivering . The evaluation team leader should provide similar training to the ev aluators, including exercise objectives, evaluator duties, and schedule. Evaluators The evaluators observe the actions and decisions of the pl ayers to report later what went well and what did not. To do this, evaluators need to be familiar with the objectives, the exercise scenario, and the jurisdiction or organization that is undertaking the exercise. Duties: Key duties of the evaluators include the following:

Observing exercise progress and recording observations (usua lly on provided evaluation forms), taking care to remain uno btrusive in the process. Noting how well the exercise is fulfilling objectives an d trying to identify problems if objectives are not met. Evaluating the actions of the players, not the players themselves. Documenting both positive and negative observations. UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.11 Participant Roles (Continued) Informing the controller during the exercise of any pr oblems.

Preparing brief written comments that can be included i n the final evaluation and recommendation report that will be pr epared by the emergency manager or other responsible party.

Unit 8 will provide more detailed information about the role of the evaluators.

How a Functional Exercise Works A brief review of how a functional exercise works is giv en below. Keep in mind, however, that you will gain a better understan ding of how a functional exercise works if you look for opportunities to observe one or better yet to participate in one.

The Beginning When a functional exercise begins will depend on its obj ectives. If testing the notification function is one of the objectives, then a “no-notice” exercise is useful. In this case, participants are given on ly the approximate timeframe scheduled for the exercise anywhere from one day to several weeks). The exact time when it begins w ill be a surprise, allowing the exercise evaluators to observe how effectiv ely notification and assembly at the command point take place. In exercises where notification is not an objective, the exercise time is usually announced in advance. UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.12 Exercise Design How a Functional Exercise Works (Continued) Briefing Exercise participants may arrive on the scene of a functio nal exercise with only a vague notion of what is to take place. T he exercise is much more likely to be successful if the participants receive a briefing that covers the following: Overview of objectives How the exercise will be carried out Time period to be simulated Ground rules and procedures Keep the Briefing Short. Avoid anything that distracts from the atmosphere of a real emergency. (For example, include a written announcement in the exercise materials to cover any admi nistrative details such as restrooms and break times.) Narrative The exercise formally begins with the presentation of t he narrative. It can be read aloud; presented on TV, computer, or slides; or dramatized.

Message Delivery and Response The action begins as simulators and players interact with one another:

Simulators communicate messages to players, and players resp ond as they would in a real emergency. Players make requests of simulators, and simulators react convincingly.

This ongoing exchange takes place according to the careful ly sequenced scenario of events that governs what takes place, when ea ch event occurs, and the messages used to inform the players. UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.13 How a Functional Exercise Works (Continued) Example: Message Delivery/Response A message comes in from the incident site commander (a simu lator) to the police chief (a player a real police chief). The message informs the chief of a traffic accident blocking emergency evacuat ion routes. The chief confers with aides, quickly plans a new traffic route, and telephones the incident site commander (simulator) with the instructions. The simulator carries out the instructions an d reports back.

Because the police chief might not react to the message as planned, simulators need to be prepared for a different response . They also must ensure that key events are kept active. For examp le, a player, not recognizing the importance of a key message, might delay action or fail to act. The simulator must then do something t o cause the player to retrieve the event. If the situation reach es a point where the exercise cannot proceed until a decision has been made, th e controller must force the issue. Messages can arrive on paper, by telephone, by radio, o r in person. Using telephones, when possible, increases the feeling of a real emergency, but whispered messages or written notes can also work well.

The success of the exercise depends on the extent to which t he participants are able to carry out their functions as if they were in a real emergency. Exercise participants should be encouraged to think of each message as an actual event. Encouraging Spontaneity The players should be able to decide among the full ra nge of responses normally available to them during an emergency. The ir ability to make decisions, communicate, or otherwise carry out their respon sibilities should not be constrained by the exercise situation. To allow the participants spontaneity, exercise controlle rs, and simulators must be well trained and prepared to handle the unex pected. While this provides a better exercise for participants, it does place a burden on controllers and simulators who must be ready to “go with the flow” to some degree when the situation calls for it. UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.14 Exercise Design How a Functional Exercise Works (Continued) Controlling the Action While simulators and players are transmitting messages and responding to them, the controller carefully monitors the interact ion and progress. Dealing with spontaneous decisions: The controller should be made aware of significant spontaneous decisions and make adjustm ents in the scenario where necessary. Example If a fire chief anticipated a later message by sending f ire trucks into an area, the controller might need to stop a simulator fr om inputting a later message asking for fire trucks. Adjusting the pace: The controller can control the pace of the exercise by adjusting the message flow slowing things down when the pace is too frantic or speeding it up when the exercise drags. The controller can also even out the pace among participants. Remember, o ne inactive organization can distract others and bring down the int ensity of the exercise. Avoid boredom by ensuring a smooth flow of m essages.

Some specific suggestions for adjusting the pace are given on the next page.

UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.15 How a Functional Exercise Works (Continued) Strategies for Adjusting the Pace of Messages Slow the pace by:

Rescheduling events to allow more reaction time. Have the simulators wait before sending messages. Discarding messages that are relatively unimportant or do not greatly impact other decisions. Throw away messages that don’t contribute to the objectives. Increase the pace and fill gaps by:

Speeding up the delivery pace (varying from the planned schedule).

Determining what is causing gaps and being ready to add or alter messages spontaneously when needed. Look at organizations with gaps to see if they have been unintentionally ignored . If so, add messages. (It may be, however, that the organization simply has little to do during a particular period.) Keeping a supply of optional messages on hand that can be added when needed.

Adding side events routine actions a department would have to continue throughout an emergency. (For example, insert a routine traffic accident to put stress on police and fire departme nts. Report an unrelated heart attack to challenge medical personne l.) Adding secondary emergencies events that develop out of the main flow of exercise events. (For example, insert utili ty outages, water main breaks, gas leaks, media calls, and similar ev ents to keep players involved between their own major events. ) Adding special planning requirements that would cause an inactive group to engage in a short-term preparedness a ctivity. (For example, have hospitals test emergency generators.) Adding misdirected messages messages given to the wrong agency. Such messages can be used to gauge the agency’s clar ity of role definition and to test whether they forward the message properly.

Relieve overloads on particular organizations by:

Reassigning. Verify that all messages are assigned to the right organizations. Then reassign any messages that could be u sed by another organization.

Thinning. Divide the overloaded messages into two piles: (1) Essential to the flow of the exercise and (2) Nice to hav e. Then get rid of some from the latter group.

Maintain an even message flow by maintaining a chart similar to the following. UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.16 Exercise Design How a Functional Exercise Works (Continued) Sample Message Flow Chart Check the times when messages are scheduled for del ivery to each organization. Participating Agency/Organization Fire EMS Public W orks EOC Facility CEO School Exercise Start 10:00 4 4 10:03 4 10:06 4 4 10:09 4 4 10:12 4 4 4 10 :15 4 4 4 etc. ( Note: A blank planning chart is provided as Job Aid 16 in Appendix A.) Skipping Time Functional exercises can depict events and situations that w ould actually occur over an extended time period (one or two weeks o r more). In order to include multiple phases of the emergency (preparatio n, response, recovery, mitigation) in a two-day exercise, it would be necessary to stop the exercise periodically and advance the time by a num ber of hours or days. These skip-time transitions should be kept to the minimu m necessary to cover the scope of the exercise. They can usually be plan ned to coincide with a natural break point.

Who handles the time skips? The controller is responsible for managing skip-time transitions and preparing transition updates to be presented to the participants before resuming the exerci se. Simulators are responsible for updating simulation displ ays to reflect the results of the previous events and participant actions. A ctions that would have been undertaken during the transition period wi ll be indicated as accomplished on the transition date. The following table illustrates a skip-time schedule for a functional exercise.

UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.17 How a Functional Exercise Works (Continued) Sample Skip-Time Schedule for a Two-Day Functional Exercise Actual Time Period Simulated Time/Activity Simulated Day 1 0900 1200 Alert (mobilization) 3 hrs First 3 hours (in real time) of Alert (mobilization) SKIP Transition Statement 1300 1600 Movement 3 hrs First 3 hours (in real time) following evacuation order ACTUAL TIME LAPSE OF 15 HOURS Day 2 0800 1100 Movement 3 hrs Situation Update 3 hours (in real time) of movement SKIP Transition Statement 1200 1530 Sustaining 2 hrs SKIP 1.5 hrs 2 hours (in real time) of early sustaining period Transition Statement 1½ hours (in real time) of later sustaining period UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.18 Exercise Design Facilities and Materials Location Exercise where you operate. To the extent possible, the functional exercise should take place in the same facility and in th e same operational configuration that would occur in a real e mergency usually the EOC or other operations center. A frequent objection to exercising at the operations cen ter is that there are not enough phones, or chairs, or restrooms. If that is the case, it is wise to find out in an exercise, not an emergency. If you can’t practice there, don’t expect to be able to conduct an emergency response there.

Room Arrangement Various room arrangements can work for a functional exe rcise, depending on the size of the exercise. These are the b asic requirements:

Space for players usually a table with plenty of work space Area(s) set aside for simulators Room for evaluators to observe A place from which the controller can operate Small exercises: In very small exercises, a single room can work. The diagram below shows a simple layout for a small function al exercise.

Sample Arrangement for a Small Functional Exercise Evaluators Simulators Simulators Controller UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.19 Facilities and Materials (Continued) Complex exercises: The following layout would be appropriate for an elaborate functional exercise. Two rooms are shown: Th e simulation room and the operations center, where the players are located. Sample Arrangement for a Complex Functional Exercis e Public Works & Engineering Simulator Police Simulator Fire Simulator Medical & Welfare Simulator Radiol.

Defense Simulator Shelter Simulator External Agencies Simulator Message Controller Simulation Switchboard Operator Simulation Plotting Map SimulationRoom Messengers Welfare Operations Section Medical & Health Operations Section Public Works & Engr. Op. Section Shelter Operations Section Fire Dept.

Operations Section Police Dept. Op. Section Military Liaison* Director PIO Chief Executives Disaster Analysis Grp Chief Resources* Damage Assessment Section Radiol. Def.Section Operational Units Map Shelter Status Board Area Map State Map Region Map Ci t y M apMajor Emergency Log Downtown Map B u l le ti n B o ar d Operational Units Plotter Emergency Log Plotters Operations Center *Optional Positions UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.20 Exercise Design Facilities and Materials (Continued) Simulation room: If more than one or two organizations or functions are being exercised, a simulation room is highly recommended . This room should comfortably house all of the simulators so that th ey can send, receive, and track messages and other communications with t he players. It should be equipped with telephones or radios if the y are to be used in the exercise. If message traffic is to be sent by hand, th e situation room must be near the players. Any layout should be adapted to the particular exercise and your local physical facilities. Whatever the layout, participant w ork spaces should be predesignated and working supplies made available. (In the previous diagram, notice the work space assignments of the simulato rs and players.) What About Communications Equipment? Communications equipment is useful when full simulation is the goal. However, often it is wise to use both electronic equipme nt and written messages. When working in compressed time, it is easy for problems to arise concerning: Development of telephone banks for the simulators.

Telephone overload for the players.

Equipment breakdown.

For these reasons, some managers leave extensive use of communications equipment for a drill. In any case, the use of electronic communication should be carefully and perhaps selectively planned. Equipment installation: When telephones will be the primary means of communication during an exercise, it may be possible to u se existing phones. Or, it may be necessary to install special lines a nd extensions to provide the necessary communication links. In some facilit ies, where a central switching system is used, an operator may handle a ll calls. UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.21 Facilities and Materials (Continued) Suggestions for Successful Communication Links Prepare a special exercise directory of telephone number s.

Include communications procedures in the directory. If you don’t have telephones, use a variety of other formats, such as: Written messages.

Simulated calls (sender whispers message in receiver’s ear) .

Hand signals (player who wants to call a simulator raise s a hand to bring the simulator over).

Simulated speaker phone or radio (simulator speaks lou dly to the players). If you use written messages, provide standardized message forms.

Displays and Materials Displays and materials maps, charts, message forms, lists, etc. are important in a functional exercise. (In the room arra ngement diagram for a complex exercise, notice the variety of maps available to the players.) These materials are used to provide details for the scena rio and keep track of activities. Generally speaking, it’s best to use what you use every day. An exercise is no time to get new maps and message forms. Test the o nes that you are currently using. Unit 9 will provide more detailed guidance about exe rcise enhancements, including communications equipment, displays and material s, and other matters. On page 6.24 is a checklist of facilities and materials sug gested for a functional exercise. Depending on the scope and complexi ty of the exercise, specific items may or may not apply. UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.22 Exercise Design Designing a Functional Exercise Design Process The full eight-step design process outlined in Unit 4 is used to develop a functional exercise. While a simplified version of that process can be used to develop a tabletop exercise, a functional exerci se even a small one requires careful attention to every step. Exercise Materials The success of a functional exercise rests on a carefully script ed scenario package that includes: A convincing narrative.

Major and minor events that grow out of the narrativ e and are carefully chosen to support the objectives.

Arrangement of the events in a realistic and convincing sequence from the beginning to the end of the exercise.

Expected actions that are tied closely to the objectives.

A great number of specific messages (perhaps 100 or more i n a larger exercise) that are so well conceived that players will respond with the expected actions.

Expect the Unexpected No matter how good you are at writing convincing messag es, sometimes players will respond in unexpected ways. Although you should try to limit the unexpected as much as possible, occasionally a spon taneous reaction is better than the response prescribed in the em ergency plan.

A master scenario of events list that includes all of the messages/events, delivery times, and expected actions.

When you have completed the scenario package, you will use the developed materials to create materials for the exercise participants, including the Exercise Plan, Control Plan, Evaluation Plan, and Player Handbook.

UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.23 Designing a Functional Exercise (Continued) Job Aids and Samples Job Aids 7, 8, 9, 10, 11, 12, 13, and 14 are provided in Appendix A. They are the job aids introduced in Unit 4. They are well suited to the design of functional exercises. In addition, a Functional Exercise Checklist is provided on the following pages. This checklist (which a lso appears as Job Aid 17 in Appendix A) summarizes the special considera tions for designing a functional exercise. In Unit 10, you will have an opportunity to develop a functional exercise using similar instruments.

UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.24 Exercise Design Designing a Functional Exercise (Continued) Functional Exercise Design Checklist: Special Cons iderations Facilities and Equipment Sufficient work space for simulators and players Simulation room (if needed) near player room Message center Control center Observer space Clear work surfaces Communication equipment (telephones, switchboard) Parking Adequate ventilation and lighting Displays and Materials Displays easily visible or accessible Maps (regional, state, local, area, downtown, operational units) Major events log Bulletin board Status boards Simulation plotting board Easels, chart paper Message forms Pencils/Paper Name cards Beginning: “No-notice” or scheduled (according to objectives) Briefing (short): Objectives Process Time period portrayed Ground rules and procedures Narrative: Verbal, print, TV, computer, slides, or dramatization Time-skips if needed Messages: Large number (depends on scope) Prescripted Optional prescripted for adjusting flow Message Delivery: W ritten Phone Other (verbal, speaker phone/radio, hand signals) Simulators prepared for spontaneous message develop ment Standardized forms for written messages Strategies for Adjusting Pace: Rescheduling Adding/Deleting messages Misdirecting messages Reassigning messages UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.25 Activity Activity: Identify Functional Exercise Responsibil ities For each of the following activities, indicate who has pr imary responsibility by placing a check mark in the appropriate col umn.

Controller Simulator Player Evaluator 1. Present the briefing. 2. Observe and record exercise progress. 3. Control the pace. 4. Decide how to implement emergency plan procedures. 5. Track progress. 6. Ad lib in response to unplanned player actions. 7. Decide how to handle unexpected situations in the exercise. 8. Respond to events. 9. Ensure that simulators and evaluators are trained. 10. Coordinate with other organizations on joint responses. 11. Deliver messages. 12. Present the narrative. 13. Inform the controller of deviations from the scenario. 14. Act the part of organizations participating in the exercise. 15. Act the part of organizations not participating in the exercise. 16. Ensure that activities run smoothly. 17. Supervise message input. 18. Portray private citizens and facilities. 19. Make decisions about departing from the planned event sequence. 20. Compare exercise conduct to objectives. 21. Update the situation board during skip-time transitions. UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.26 Exercise Design Activity: Identify Functional Exercise Responsibil ities (Continued) Suggested Answers: Controller Simulator Player Evaluator 1. Present the briefing. 4 2. Observe and record exercise progress. 4 3. Control the pace.

4 4. Decide how to implement emergency plan procedure s. 4 5. Track progress.

4 6. Ad lib in response to unplanned player actions.

4 7. Decide how to handle unexpected situations in th e exercise.

4 8. Respond to events.

4 9. Ensure that simulators and evaluators are traine d.

4 10. Coordinate with other organizations on joint re sponses. 4 11. Deliver messages.

4 12. Present the narrative.

4 13. Inform the controller of deviations from the sc enario. 4 14. Act the part of organizations participating in the exercise. 4 15. Act the part of organizations not participating in the exercise. 4 16. Ensure that activities run smoothly.

4 17. Supervise message input.

4 18. Portray private citizens and facilities. 4 19. Make decisions about departing from the planned event sequence.

4 20. Compare exercise conduct to objectives. 4 21. Update the situation board during skip-time tra nsitions. 4 UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.27 Summary and Transition Unit 6 was the second of three units providing in-depth information about specific types of exercises. This unit provided informatio n about the functional exercise, including key characteristics, participa nts, format, strategies for conducting the exercise, and key design consi derations.

Unit 7 will discuss the full-scale exercise.

Reference Library For More Information Many of the information resources cited in Units 1–4 al so contain information about functional exercises. UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.28 Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer that you have chosen. 1. The functional exercise: a. Simulates an emergency response in an actual field set ting.

b. Simulates an emergency situation in a relaxed group discussion.

c. Simulates an emergency as realistically as possible witho ut deploying people and equipment to the site.

d. Simulates an emergency involving all of the functio ns, organizations, and personnel that would respond to an actual emergency. 2. The goal of a functional exercise is to test or evalu ate the capability of one or more functions in the context of an emergency event. a. True b. False 3. An exercise that tested only notification procedures i n response to a terrorist bombing would be: a. A drill.

b. A tabletop exercise.

c. A functional exercise.

d. A full-scale exercise. 4. A functional exercise can test the same functions and re sponses as in a full-scale exercise without high costs or safety risks. a. True b. False 5. A functional exercise is similar to a tabletop exercise except that the functional exercise requires less scripting, planning, and attention to deta il.

a. True b. False 6. In a functional exercise, events are presented via pr oblem statements or messages and then discussed by the group. a. True b. False UNIT 6: T HE FUNCTIONAL EXERCISE Exercise Design Page 6.29 Knowledge Check (Continued) 7. A functional exercise is a good way to assess: a. Communication and information sharing among organi zations.

b. Response time of field personnel. c. Adequacy of response resources (personnel and equipment) .

d. Hazard analysis for developing the EOP. 8. _________ is the best location for a functional exer cise.

a. A field site similar to where an actual incident mig ht occur b. The Mayor’s office c. A 911 dispatch center d. The Emergency Operations Center 9. Key decision makers in the jurisdiction or organizati on being exercised would normally assume the role of: a. Players.

b. Simulators.

c. Controllers.

d. Evaluators. 10. Which of the following is NOT true of a simulator? a. He or she may deliver written messages.

b. He or she is often called upon to rate the performa nce of key players.

c. He or she may deliver messages verbally.

d. He or she sometimes needs to make up a response to a p layer.

11. It may be necessary to delete planned messages if the pace of the exercise starts to drag.

a. True b. False 12. When designing a functional exercise, the eight-step design process can usually be significantly shortened or simplified. a. True b. False UNIT 6: T HE FUNCTIONAL EXERCISE Page 6.30 Exercise Design Answers Knowledge Check (Continued) 1. c 2. a 3. a 4. a 5. b 6. b 7. a 8. d 9. a 10. b 11. b 12. b Unit 7: The Full- Scale Exercise UNIT 7 THE FULL -SCALE EXERCISE Exercise Design Page 7.1 Introduction This unit focuses on the full-scale exercise. We will look closely at the characteristics of the full-scale exercise how it differs from the other types of exercises, who participates, the role of the EOC , and key design considerations. At the end of the unit, you will deve lop an action plan for later use in designing a full-scale exercise for your org anization.

Unit 7 Objectives After completing this unit, you should be able to: Describe the purpose and characteristics of a full-scale exer cise.

Explain how designing a full-scale exercise differs from designing a functional exercise. Identify planning considerations for site selection and sce ne management for a full-scale exercise.

UNIT 7: T HE FULL -SCALE EXERCISE Page 7.2 Exercise Design What Is a Full-Scale Exercise?

A full-scale exercise is as close to the real thing as possibl e. It is a lengthy exercise that takes place on location, using as far as possible the equipment and personnel that would be called upon in a real event. In a sense, a full-scale exercise combines the interactivity of the functional exercise with a field element. It differs f rom a drill in that a drill focuses on a single operation and exercises only one organ ization.

Eventually, every emergency response organization must hold a full-scale exercise because it is necessary at some point to test capabil ities in an environment as near to the real one as possible. However, there is more to a full-scale exercise than just practice in the field. As we discussed in Unit 1, various regulatory age ncies have requirements for full-scale exercises which must be satisfied . In order to receive FEMA credit, for example, a full-scale exercise m ust fulfill three requirements: It must exercise most functions.

It must coordinate the efforts of several agencies.

In order to achieve full coordination, the EOC must be activated.

UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.3 Activity Activity: Know Your Regulatory Requirements Answer the following questions about your own organizat ion. If you are not sure of the answer, this question may require some re search. You are encouraged to find the answers now, before continui ng with the unit. Understanding your organization’s requirements will pro vide an important foundation for the concepts covered in this unit. 1. What agencies or groups impose exercise requirements or guidelines on your organization?

2. What do they require concerning full-scale exercises? (Consider scope, frequency, numbers of organizations involved, coor dination, communication, documentation, evaluation, or other issue s.) UNIT 7: T HE FULL -SCALE EXERCISE Page 7.4 Exercise Design What Is a Full-Scale Exercise? (Continued) Key Characteristics The key characteristics of full-scale exercises were discussed in Unit 2. Below is a brief summary of the main points.

Key Characteristics Interactive exercise, designed to challenge the entire e mergency management system in a highly realistic and stressful envi ronment.

Tests and evaluates most functions of the emergency manage ment plan or operational plan.

Takes place in an EOC or other operating center and at field sites.

Achieves realism through:

On-scene actions and decisions.

Simulated “victims.” Search and rescue requirements.

Communication devices.

Equipment deployment.

Actual resource and personnel allocation.

Involves controller(s), players, simulators (different fr om simulators in a functional exercise), and evaluators.

Players represent all levels of personnel, including resp onse personnel.

Messages may be visual (e.g., staged scenes, made-up victims, props) and scripted.

All decisions and actions by players occur in real time and generate real responses and consequences from other players.

Requires significant investment of time, effort, and re sources (1 to 1½ years to develop a complete exercise package). Atten tion to detail is crucial.

UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.5 What Is a Full-Scale Exercise? (Continued) The Purpose of Full-Scale Exercises There are numerous reasons for conducting a full-scale exe rcise. A full- scale exercise: Enables a jurisdiction or emergency management system to evaluate its ability to perform many functions at once. Is useful to test total coordination, not only among po licy and coordination officials, but among field forces. At the same time, it can test interorganizational coordination. Can pinpoint resource and personnel capabilities and rev eal shortfalls.

Greatly expands the scope and visibility of the exercise program.

If well-planned, can attract public attention and raise credibility. (However, to be successful, it must be the culmination of a comprehensive and progressive exercise program that has be en developed as the organizational capacity has grown.) What Does It Take to Run a Full-Scale Exercise? Some people wrongly believe that when a full-scale ex ercise is, started, it can run on its own steam. In fact, a full-scale exercise r equires a substantial commitment of time, money, personnel, and e xpertise and should not be undertaken without the necessary preparat ion. These are the most important requirements: Substantial experience with preparatory exercises of var ious kinds drills, tabletops, and functional exercises.

Total commitment of all emergency service organizations. Support from the chief elected and/or appointed offici als.

Adequate physical facilities, including space for the EOC and field command posts. Adequate communication facilities (e.g., radios and tele phones).

Plans in place to handle costs (both evident and hidden) , labor, time commitment, etc. Carefully thought out and planned site and logistics.

UNIT 7: T HE FULL -SCALE EXERCISE Page 7.6 Exercise Design Activity Activity: Compare Functional and Full-Scale Exerci ses In the following table, compare functional and full-sca le exercises by writing a brief description in each of the cells. Functional Full-Scale Degree of Realism Format/Structure Atmosphere Who Takes Part Who Leads Where Held Equipment Deployed Test Coordination Test Adequacy of Resources Test Decision-Making Process Relative Complexity/Cost Formal Evaluation UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.7 Activity: Compare Functional and Full-Scale Exerci ses (Continued) Suggested Answers: Functional Full-Scale Degree of Realism As realistic as possible without deploying resources As realistic as possible; resources deployed Format/Structure Interactive; simulators deliver “problem messages,” players respond in real time Interactive; simulators play roles at the scene, players respond Atmosphere Tense, stressful Highly tense, stressful Who Takes Part Controller players (policy, coordination, operations), simulators, evaluators Controller(s), players (all levels), simulators, evaluators Who Leads Controller Controller(s) Where Held EOC or other operations center EOC and field site(s ) Equipment Deployed No Yes Test Coordination Yes Yes Test Adequacy of Resources Yes Yes Test Decision-Making Process Yes Yes Relative Complexity/Cost Large scale; complex format; moderate cost Very large scale; highly complex; high cost Formal Evaluation Yes Yes UNIT 7: T HE FULL -SCALE EXERCISE Page 7.8 Exercise Design Full-Scale Exercise Roles Full-scale exercises involve one or more controllers, the participants, simulators, evaluators, and a safety officer. Controllers One or more controllers manage the exercise. In some ex ercises, where there are multiple sites or organizations, there may b e more than one controller. In this case, all of the controllers cooperat e under the direction of a chief controller. The controller (or chief controller) is responsible for e nsuring that the exercise starts on schedule. The controller also designates an exercise control point from which all communications should be mon itored.

Participants A full-scale exercise involves all levels of personnel, in cluding:

Policy makers— those who are responsible for making broad policy decisions. They might include the chief executive and hi s or her staff, the Public Information Officer, the emergency manager, key department heads, and other elected officials. Coordination personnel— people from various departments who coordinate decisions of policy makers and make plans for r esources.

Operations personnel— those who carry out the directives. Sometimes coordination and operations are the same. Field personnel— fire, police, EMS, search and rescue, volunteer groups, representatives of private enterprises who parti cipate in the response, and many others.

UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.9 Full-Scale Exercise Roles (Continued) Simulators Simulators in a full-scale exercise are different from t hose in a functional exercise. In a full-scale exercise, simulators are the vol unteers who pretend to be victims of the emergency event. For rea lism, they may wear makeup and they “act” injured, unconscious, hysterica l, or dead whatever the scene calls for. Evaluators Evaluators observe the action and keep a log of all sig nificant events. This is important because so many of the actions will not be prescripted, but rather spontaneous responses to other actions. Evalua tors may videotape exercise action. Safety Officer There are so many potential safety issues in a full-scale exercise that a safety officer should be designated. This person’s primar y responsibility is to analyze the entire exercise from a safety perspecti ve. We will return to safety measures a little later. How the Full-Scale Exercise Works Beginning The full-scale exercise begins in a fashion similar to the functional exercise; whether it is announced or “no notice” depends in part on the objectives. The exercise designer will decide how and wh en the exercise is to begin. The trigger may simply be a call from di spatch, a radio broadcast, or a telephone call from a private citizen. The beginning for each participant should be as realistic as possible (that is, personnel should receive notification through normal channels). Personnel from the emergency services that are taking pa rt in the field component must then proceed to the assigned location, whe re a “visual narrative” is displayed before them in the form of a mock emergency to which they will respond. Key decision makers those who would normally operate out of the EOC or command center during an emergency proceed to the EOC to fulfill their roles. Command posts are set up as required by th e event.

UNIT 7: T HE FULL -SCALE EXERCISE Page 7.10 Exercise Design How the Full-Scale Exercise Works (Continued) Action Actions in a full-scale exercise occur in the EOC, at one o r more field sites, and at the related command posts. Actions taking p lace at the event site and command posts serve as input to the simulat ion taking place at the EOC. Although medical personnel, hospitals, EMS, fire services, and other localized emergency operations do not usually require ce ntralized command from the EOC, they do require coordination wi th officials at the command posts. Sustaining Action Action is sustained by various means, including: Prescripted messages input by the controller(s).

Messages and actions from the field that require action a t the EOC.

Spontaneous responses to the various messages and actions.

Command Post Messages A field command post can be used as part of the message in put into the EOC. Either the command post can be written into the scenario and have a set of prescripted messages to be transmitted by ra dio, or the command post controller can monitor the sequence of event s and transmit spontaneous messages. UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.11 Exercise Locations Field sites: The main event site will depend on the exercise scenario and objectives. For example, if the central event inv olves a plane crash, the exercise might take place at an airport. A simulat ed terrorist attack could be staged in a public facility such as a convention ce nter or shopping mall. A hurricane or flood exercise might be dispersed over many locations. Most events will involve additional locations such as second ary event sites, hospitals, mortuaries, shelters, and other support l ocations, and command posts will be established near the emergency sites. In fact, one of the reasons a full-scale exercise is so complicated is that activity is taking place in various locations, and all of the sites m ust be coordinated. Emergency Operations Center: Activation of the EOC or other operations center is central to a full-scale exercise (just as it is in a functional exercise). The purpose of the EOC is to prov ide a policy and coordination facility for the Chief Executive Officer ( CEO) and staff to respond effectively to an emergency. In essence, the EOC is the voice of government during an emergency. Emergencies place strains on government the demand for service escalates, while ability to deliver diminishes. Gatheri ng information, making decisions, and directing necessary actions require clo se coordination between key officials. This coordination i s best obtained if officials and support staff are in a centralized location with direct lines of communications. The central location makes it possible to accomplish a numb er of tasks that would be impossible (or very difficult) separately . For example:

Information can be gathered, verified, and recorded i n one spot.

Officials can deploy resources in a timely and intelligen t manner.

Direction and control can be efficiently managed.

Officials can coordinate actions and decisions.

It’s easier to set meaningful priorities when the key a ctors collaborate. The size and makeup of the EOC differ according to the size of the jurisdiction or response system. The EOC may take up an entire floor of a building or a small room. UNIT 7: T HE FULL -SCALE EXERCISE Page 7.12 Exercise Design Designing the Full-Scale Exercise The design of a full-scale exercise can be quite difficult , often requiring the expertise of several response organizations. When d eveloping your first full-scale exercise, it is usually advisable to start small and build to more complex exercises. Many of the potential difficult ies relate to logistical problems, but others rest with the design. The entire eight-step design process is used to design full -scale exercises, although the process is applied somewhat differe ntly when designing tabletops and functional exercises. The differ ences arise from the fact that tabletop and functional exercises rely on words to obtain realism, while a full-scale exercise like a real emergency gains its reality from things. There is a shift from a written scenario to visual rea lity represented by a real site, real people (some of them simulating victims), and real equipment.

Let’s take a closer look at how each of the design steps i s applied to designing a full-scale exercise.

The First Four Steps As with any exercise, the first four design steps are: 1. Assess needs. 2. Define the scope. 3. Write a statement of purpose. 4. Define objectives. For a full-scale exercise, these steps require deeper anal ysis and greater attention to detail. So much rests on the outcomes of t hese steps that you must be sure that you have created a clear picture o f what is to be achieved through the exercise. If any of these areas is left at all vague, the problems will be greatly magnified later on.

UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.13 Designing the Full-Scale Exercise (Continued) Step 5: The Narrative The narrative is handled differently in a full-scale e xercise. Because a lengthy verbal description is not needed to set the scene , the narrative is shorter. Step 6: Major and Detailed Events Major and detailed events still exist in a full-scale ex ercise, but many of them exist as actual occurrences rather than as verbal descri ptions. For example, an earthquake scenario may have to rely on w ords to simulate some events. Other earthquake events can be simulated w ith fallen logs, bricks strewn around a building, dummies resting under b eams, people acting injured or frightened, and other props. Even when events are presented visually, they cannot be random and haphazard. Each event must be carefully planned and st aged to support objectives and generate the expected actions. Step 7: Expected Actions As with any exercise, expected actions must be specifically i dentified, based on the exercise objectives. A detailed list of exp ected actions is an important foundation for the exercise evaluation. Step 8: Messages There are two kinds of messages in a full-scale exercise: Visual and prescripted. Much of the action grows out of the initia l message and scene set-up. The scene that is set up contains a number of “visual” messages to which participants need to respond. It may al so be necessary to have some prescripted messages to move the actio n along.

For every event, try to anticipate all possible reactio ns. But it is important to be flexible enough to adapt to player actions and decisions that you hadn’t expected. Sometimes an unanticipated response wi ll be an improvement over the expected action. If your scenes (events) are well planned, the exercise wi ll go in the general direction that you planned, even if a few of the decisions are off course. UNIT 7: T HE FULL -SCALE EXERCISE Page 7.14 Exercise Design Designing the Full-Scale Exercise (Continued) Watch the Details!

Make sure that the scenes that you create are good enoug h to get the expected action. For example, if you have victims who don’t know anything about medicine, either coach them in advance a bout their symptoms or tag them with symptoms and vital signs and ap ply makeup to simulate injuries. At the end of the exercise, you don’t want players to say, “Well, I didn’t know what you were getting at.” Special Considerations A full-scale exercise represents a huge logistical challenge , and it’s easy to overlook details. One way to promote clear and cre ative thinking is to “walk the site” either physically or mentally. By doing so, you may b e able to identify potential problems and do more real istic planning. In mentally evaluating the scene, you should consider: Site selection.

Scene management.

Personnel and resources.

Response capability.

Safety and legal liability.

Emergency call-off.

The media.

The following are some criteria and guidelines related to each of these areas. UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.15 Special Considerations (Continued) Site Selection Selecting the right site for the exercise should be one of the first decisions that you make. Because the exercise requires the mobili zation of personnel and resources, space and realism are key. Site Selection Criteria Credibility Is the type of emergency one that has a real possib ility of occurring? (A credible emergency can elicit greater cooperation and partic ipation.) Realism Is the site as realistic as possible without interf ering with normal traffic or safety?

Examples: In simulating an overturned tanker truck on a main stretch of freeway, you can’t tie up rush hour traffic for hou rs. You’ll need to find a similar location to stage the crash. For a plane c rash, try a secondary airport instead of a national airport. Adequacy of Space Is it large enough to accommodate the number of vic tims, responders, and observers? Is there space for responders’ and observers’ vehic les? Scene Management Management of the scene refers to a number of issues, incl uding:

Logistics at the scene. Creation of a believable emergency scene.

Number of victims.

Management of props and materials.

Number of controllers.

The checklist on the following page includes questions you should consider related to scene management.

UNIT 7: T HE FULL -SCALE EXERCISE Page 7.16 Exercise Design Special Considerations (Continued) Scene Management Checklist Logistics 4 W here will players be set up?

4 If there is a mobile EOC, where will they park?

Believability 4 How will you simulate the emergency? (E.g., if you will simulate fire, what will you use for smoke? How will you simulate a spilled che mical, broken glass, flood damage, etc.?) 4 W ho will serve as victims? (For greater realism, s elect victims from different age groups, with different body types and physical char acteristics.) 4 How will you ensure that victims realistically port ray their roles?

Number of Victims 4 How many victims does the type of emergency call fo r?

4 W hat are the capability and capacity of hospitals a nd other resources to handle victims?

4 W hat does the history of past events indicate about types and numbers of injuries?

Props and Materials 4 W hat kinds of props and materials will be needed to simulate injuries, damage, and other emergency effects (e.g., victims’ makeup, dum mies, construction materials)?

Controllers 4 How many controllers will be needed to manage the e xercise sites? (In a multiple- site exercise, every site will require a controller .) UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.17 Special Considerations (Continued) Personnel and Resources The scenario will help determine how many people (par ticipants and volunteers) will be involved, how many and what kind s of equipment will be needed, and the potential costs. Consider the follo wing factors in planning for personnel and resources:

How many participants? (Sometimes it is necessary to scale down the exercise to a half day or less to increase participati on.) How many volunteers?

Scene set-up Victims Members of the public What kinds of equipment will be used?

How many pieces of each type of equipment?

How much fuel for vehicles and equipment?

What kinds of materials and supplies will be needed, an d how will they be obtained?

Expenses:

Overtime wages Vehicle and equipment fuel Materials and supplies Equipment Keep your scenario reasonable in terms of equipment. T rue, you need to determine whether you have the resources to meet no rmal emergency demands. But don’t get carried away. Hold your people to the use of actual equipment . Don’t let them simulate use of equipment that doesn’t exist. UNIT 7: T HE FULL -SCALE EXERCISE Page 7.18 Exercise Design Special Considerations (Continued) Response Capability In planning both personnel and resources, take into consi deration how the exercise might deplete the actual response capability of the organizations involved. It is unwise to compromise the ability to respond to real emergencies during an exercise. Here are some su ggestions:

Be sure that there are enough personnel and resources to continue their responsibilities if a real emergency occurs. (In som e cases, a call-off procedure will solve part of the problem.) Consider using second-shift personnel or mutual aid from other jurisdictions or organizations. Consider using volunteers as a smaller response shift. Safety Total operation safety is an absolute must in a full-scal e exercise. Awareness of safety issues must be carried throughout the p lanning and conduct of the exercise to ensure that safety problems are noted and eliminated. A safety officer should be designated whose primary responsibility is to analyze the entire exercise from a safety perspective. Suggested safety measures are on the next page. UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.19 Special Considerations (Continued) Suggested Safety Measures Include safety as one of the activities in exercise development.

Assign each exercise team member the responsibility of examining the exercise for safety within his or her discipline.

Identify all possible safety hazards and resolve ea ch one.

Address safety as part of the preexercise briefing. Include safety factors in simulator and evaluator i nformation packets.

Examine each field location before the exercise to guarantee that safety precautions have been taken.

Ensure that the safety officer has the authority to terminate an activity or even the entire exercise if a safety problem arises.

Provide for call-off procedures in the event of an actual emergency.

Emergency Call-Off A real emergency may occur during any exercise especially a lengthy one. Be sure to keep enough personnel in reserve to h andle routine problems. In some instances, it may be necessary to stop the exercise t o handle the real emergency. Every exercise should have a planned ca ll-off procedure that will result in the prompt return of personnel an d equipment to full duty status. The procedure should include a code word or phra se that the controller or safety officer can use to indicate that: The exercise has been terminated.

Personnel should report to their regular duty position s. All radio traffic will return to normal.

The procedures should be tested.

Legal Liability Legal questions of liability, including injuries during exercise conduct, must be researched by your local attorney. UNIT 7: T HE FULL -SCALE EXERCISE Page 7.20 Exercise Design Special Considerations (Continued) The Media A full-scale exercise of any magnitude will draw media attention whether you want it or not. If the exercise is well designed, favorable media reports are more likely. So, include the media in yo ur plans. They can be very helpful in gaining support for the program, and their presence will increase realism. Also make allowances for observers and public information people. Decide in advance where you will locate them, and give them an opportunity to observe. Job Aids and Samples The job aids introduced in Unit 4 (and provided in Ap pendix A) can be used to develop full-scale exercises. In addition, the ch ecklist provided on the next page covers special considerations for full-scal e exercises. (This checklist also appears as Job Aid 18 in Appendix A.) UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.21 Special Considerations (Continued) Full-Scale Exercise Planning Checklist: Special Co nsiderations Participants: Controller(s) sufficient to manage all event sites Simulators (mock victims) different age groups, body types, physical characte ristics Players (most functions, all levels policy, coordination, operation, field) Evaluators Safety Officer Site Selection: Adequate space for number of victims, responders, a nd observers Space for vehicles and equipment As realistic as possible without interfering with normal traffic or safety Credible scenario and location Scene Management: Logistics (who, what, where, how, when) Believable simulation of emergency Realistic victims Preparation of simulators to portray roles realisti cally Number of victims consistent with type of emergency , history of past events Types of injuries consistent with type of emergency, history of past events Victim load compatible with local capacity to handle Props and materials to simulate injuries, damage, o ther effects Personnel and Resources: Number of participants Number of volunteers for scene setup, victims, etc. Types and numbers of equipment Communications equipment Fuel for vehicles and equipment Materials and supplies Expenses identified (wages, overtime, fuel, materia ls and supplies) Response Capability Sufficient personnel kept in reserve to handle routine nonexercise events Safety Safety addressed through development Each design team member responsible for safety in o wn discipline Hazards identified and resolved Safety addressed in preexercise briefing, simulator and evaluator packets Each field location examined for safety issues Safety officer designated, given authority Legal Liability Legal questions of liability researched by local attorney Emergency Call-Off Call-off procedure in place, including code word/ph rase Call-off procedure tested Media Role of media addressed in planning, used as a reso urce to gain favorable exposure Media and observers considered in logistical planning UNIT 7: T HE FULL -SCALE EXERCISE Page 7.22 Exercise Design Activity Activity: Plan Ahead for the Full-Scale Exercise Think about your organization or jurisdiction in relati on to the special design considerations just discussed. Are there special probl ems you may need to work around when designing a full-scale exerci se? Do you have ideas about how to make use of particular situations or locations? (Remember the description in Unit 1 of the building co llapse exercise that was built around the planned destruction of a sports are na in Denver.) In the space below, record notes that will help you reme mber your ideas about planning for a full-scale exercise in the future. Site Selection: Scene Management: Personnel and Resources: UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.23 Activity: Plan Ahead for the Full-Scale Exercise ( Continued) Response Capability: Safety: Legal Liability:

Emergency Call-Off: The Media: UNIT 7: T HE FULL -SCALE EXERCISE Page 7.24 Exercise Design Summary and Transition Unit 7 was the third of three units providing in-dept h information about specific types of exercises. This unit provided informatio n about the full- scale exercise, including key characteristics and special plann ing considerations. The next unit will focus on exercise eval uation.

Reference Library For More Information Lessons Learned from Exercises and Incidents www.nrt.org/epa/nrt/home.nsf Media descriptions of past full-scale exercises: Response ’95: www.fema.gov/nwz95/95_106.htm Demolition of Denver’s McNichols Arena: www.fema.gov/r eg- viii/usar/macdemo.htm Ames/Moffett Airfield exercise: www.fema.gov/mobex/re sprep01.htm Volunteer Response 2000: www.bechteljacobs.com/news/2000- or/exerciseemer.htm Community awareness and emergency response exercise: www.ci.henderson.nv.us/news/news134.html Domestic counterterrorism exercise “TOPOFF”: www.usdoj.gov/opa/pr/2000/April/230ag.htm Lessons learned from “TOPOFF”: www.cdc.gov/ncidod/eid/vol6no6/hoffman.htm UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.25 Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer that you have chosen. 1. A full-scale exercise involves: a. All levels of personnel, including response personnel. b. Primarily the key policy and decision makers. c. Policy, coordination, and operations personnel.

d. The staff from one department or unit. 2. In a full-scale exercise, simulators: a. Are not required.

b. Have a somewhat different role than in a functiona l exercise.

c. Have the same role as in a functional exercise.

d. Play the roles of all field personnel. 3. In a full-scale exercise the narrative is largely rep laced by:

a. Scripted messages.

b. The exercise directive.

c. A staged scene representing the emergency.

d. The Player Handbook. 4. ______________ is a good use of a full-scale exercise. a. Testing emergency procedures and coordination of mult iple agencies or organizations b. Trying out a new and untested emergency plan c. Training personnel in negotiation d. Practicing group problem solving in a nonthreatenin g environment 5. A full-scale exercise: a. Does not use prescripted messages.

b. Uses only visual and action messages.

c. May include visual, action, and prescripted messages.

d. Uses the visual narrative in place of messages. 6. In a full-scale exercise, all decisions and actions by pl ayers occur in real time and generate real responses and consequences from other players. a. True b. False UNIT 7: T HE FULL -SCALE EXERCISE Page 7.26 Exercise Design Knowledge Check (Continued) 7. The best way to begin a full-scale exercise is to gath er all of the participants in a central location for a briefing that explains the objectives, r oles, responsibilities, and ground rules. a. True b. False 8. _______________ is a major difference between full -scale and functional exercises.

a. Formal evaluation b. Lead role of the controller c. Activation of the EOC d. Field-based action 9. Because field actions lead to decisions and further acti on in a full-scale exercise, it is unnecessary to develop a list of major and detailed eve nts.

a. True b. False 10. If you want to test response to a terrorist bomb at tack, a good approach would be to stage an unannounced full-scale exercise at a sports arena durin g a professional game attended by thousands of spectators. a. True b. False 11. In designing a full-scale exercise, it is usually advi sable to exclude the media and the public. a. True b. False UNIT 7: T HE FULL -SCALE EXERCISE Exercise Design Page 7.27 Answers Knowledge Check (Continued) 1. a 2. b 3. c 4. a 5. c 6. a 7. b 8. d 9. b 10. b 11. b Unit 8: Exercise Evaluation UNIT 8 EXERCISE EVALUATION Exercise Design Page 8.1 Introduction Evaluation is the process of observing and recording exercise activities, comparing the performance of the participants against th e objectives, and identifying strengths and weaknesses. Evaluation is a very complex topic, and this unit provi des only a very general overview. In this unit, we will briefly discu ss the importance of evaluation and its relationship to exercise development , evaluation team structure and duties, key aspects of evaluation methodolo gy, and evaluation tasks that happen after the exercise is finish ed.

Unit 8 Objectives After completing this unit, you should be able to: Describe the need for a systematic approach to exercise eva luation.

Identify and explain the tasks in the exercise evaluati on process. UNIT 8: E XERCISE EVALUATION Page 8.2 Exercise Design Integrating Evaluation Throughout Development As you have seen in the previous units, exercise developm ent is a continuous process that begins long before the exercise and continues after the exercise is over. It continues until needed ch anges have been made and suggestions have been incorporated into the ne xt exercise.

Evaluation is not something that is done when all of t he action is over. It begins when exercise design begins, when objectives are de veloped. You encountered the following chart in Unit 3. Notice again the kinds of evaluation activities that occur in the three exercise ph ases. In the remainder of this unit, we will take a brief look at each of the major evaluation activities listed in this chart.

Task Categories Preexercise Phase Exercise Phase Postexercise Pha se Design Review plan Assess capability Address costs and liabilities Gain support/Issue exercise directive Organize design team Draw up a schedule Design exercise (8 design steps, including developing objectives) Prepare facility Assemble props and other enhancements Brief participants Conduct exercise Evaluation Select evaluation team leader Develop evaluation methodology Select and organize evaluation team Train evaluators Observe assigned objectives Document actions Assess achievement of objectives Participate in postexercise meetings Prepare evaluation report Participate in followup activities UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.3 Why Evaluate the Exercise?

For an emergency management system to be effective, it is critical that the personnel, plans, procedures, facilities, and equipme nt be exercised and tested on a regular basis. Yet no amount of exerci sing will be constructive unless each exercise is followed by a structured evaluation that enables the emergency management organization to identify successes and shortfalls. Good evaluation can help the organization identify: Whether the exercise has achieved its objectives. Needed improvements in the EOP, procedures, or guideli nes.

Needed improvements in the emergency management system. Training and staffing deficiencies. Needed operations equipment. Need for continued exercising of the plan and the emer gency management functions. If these goals are to be met, the evaluation approach must be systematic methodical in procedure, thorough, and organized.

UNIT 8: E XERCISE EVALUATION Page 8.4 Exercise Design The Evaluation Team In the early stages of exercise design, a number of peop le will be brought together into a design team, led by a design team lea der or exercise director. One member of this team will become the eva luation team leader, or chief evaluator, and this person will in tu rn select the members of the evaluation team.

Team Structure The size and composition of the evaluation team will d epend on the type of exercise, its complexity, and the availability of pe ople to serve. A small exercise with limited objectives, few participating orga nizations, and only a few locations might need only a team leader and thr ee to six evaluators. A group this small would report directly to the team l eader.

A large full-scale exercise might require an evaluation director to supervise several team leaders at different sites who in turn supervise several evaluators. With a team of this size, the vari ous evaluators would be assigned to serve at selected sites and a means of coordi nation or communication set up among them. An organizational cha rt would be helpful to keep lines of authority straight. Role of the Team Leader What? The evaluation team leader is primarily responsible fo r the evaluation methodology, for selecting and training th e evaluation team, and for preparing the evaluation report. Who? Ideally, the team leader should have experience in evaluation, management, exercise design or participation, and train ing or education. Normally, he or she should be a member of the design t eam. In some cases, it might be necessary to bring in someone from outsi de the design team, although that is usually unnecessary if a voluntee r training and exercise officer is on the design team and is willing to serve. (Other design team members are likely to be too heavily invo lved in developing the exercise.) UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.5 The Evaluation Team (Continued) When? Selecting the evaluation team leader as early in the design process as possible has several advantages: It will ensure that the evaluation becomes an integral part of the exercise development effort. It will maintain the integrity of the evaluation fu nction and prevent it from overlapping with the control and simulation funct ions.

It will ensure that at least one person can devote time and mental effort to the large task of evaluation. Selection of Team Members The evaluation team leader is responsible for selecting and training the evaluation team. The ideal evaluator has many skills and personal attributes. Although it may not be possible to find a person who has all of these characteristics, the team leader will be looking fo r the following:

Skills Attributes Appropriate technical expertise in evaluation Communication skills, both verbal and written Organizational ability Ability to see the relationship between events and objectives Ability to adjust to rapidly changing situations “People skills,” sensitivity Objectivity Self-motivation W illingness to help Honesty and integrity (reports facts truthfully, keeps information confidential) Familiarity with the plan Often an evaluation team can be assembled by using a li ttle imagination and beating the bushes. Evaluators can be drawn from v arious sources, including: Neighboring jurisdictions.

Emergency services personnel who will not be playing in the exercise.

Professional evaluators.

State or Federal personnel.

College or university faculty.

Public service organizations.

UNIT 8: E XERCISE EVALUATION Page 8.6 Exercise Design The Evaluation Team (Continued) Training of Team Members The training of an evaluation team usually can be don e in an orientation meeting. The nature and length of the meeting will depend on the experience and skill of the members. Most evaluators no matter how experienced will need information on:

The exercise scenario.

Rules of play.

The objectives.

Evaluation requirements and procedures.

Evaluation forms.

Inexperienced evaluators may need some practice drills. Evaluators from outside the organization/jurisdiction will need inform ation about the organization/jurisdiction. Try to Be Unobtrusive It is well documented that the presence of an evaluator can affect the behavior of those being observed, possibly resulting in inaccurate data. Therefore, the evaluation team should plan ways to ob serve that are as unobtrusive as possible. Examples include: Being in position when the exercise begins so as not to a ttract attention.

Postponing taking notes for a few minutes until player s get involved in the play and stop noticing them.

UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.7 Evaluation Methodology Evaluation methodology is simply the procedures and stra tegy used to evaluate an exercise. The methodology includes: How the evaluation team will be structured.

Objectives to be measured.

Evaluation packet. Defining the Team Structure Aspects of the team structure will have a significant impa ct on how the evaluation proceeds. Therefore, decisions such as the foll owing are an important part of the methodology: Number of evaluators and their evaluation-related ex perience and knowledge.

Organization of evaluators at multiple sites (i.e., sub teams).

Lines of authority (e.g., evaluators, supervisors or tea m leaders, chief evaluator).

Communication and coordination among team members.

Evaluation Criteria One of the first steps in developing the methodology i s to determine the criteria that will be used to determine if the exercise is successful. These criteria are tied to the objectives and expected actions. At the outset of exercise design, the general objectives are defined. Then, in developing the scenario, the general objectiv es are broken into smaller units the expected actions. From the expected actions, specific points of review and other evaluation measures can be d eveloped.

UNIT 8: E XERCISE EVALUATION Page 8.8 Exercise Design Evaluation Methodology (Continued) As discussed in Unit 4, the objectives must be stated clearly and precisely, describing actions that can be observed and measu red. Using the SMART system will ensure that objectives are: Simple.

Measurable.

Achievable.

Realistic.

Task-oriented.

You may wish to review the material in Unit 4 now to refresh your memory about the important elements of good objectives. Evaluation Packet The evaluation packet, or Evaluation Plan, contains al l plans for the collection of data, including objectives and points of re view, checklists or other evaluation forms, and observation techniques. A sample Evaluation Plan, with instructions for customizing it fo r a specific exercise, is provided in the Exercise Design Tool Box, available at http://training.gov/emiweb/IS/is139lst.asp .

Data can be collected by various means, such as evaluation forms, running written narrative, audiotape, or videotape. Each method has advantages and disadvantages, which should be considered i n developing the methodology. Some suggestions are on the next page.

UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.9 Evaluation Methodology (Continued) Suggested Evaluation Strategies Plan the observation process. One approach is to use these four steps:

1. Recall the specific objectives of the exercise, the deta iled events, and the actions or decisions that they suggested.

2. Identify the players expected to take the actions or make decisions as those who should be observed.

3. Locate evaluators in a position to observe the playe rs.

4. Brief the evaluators on what actions or decisions are expected.

Provide points of review to guide evaluators. They make it possible to be very objective in collecting data. Example:

Objective: Demonstrate the adequacy of displays to support t he emergency operations during the exercise.

Points of Review: YES NO Status boards available in facility ______ ______ Status boards utilized ______ ______ Status boards kept updated ______ ______ Maps available ______ ______ Maps up to date ______ ______ Provide evaluation forms. They may include simple questionnaires, checklists, or rating sheets on which observa tions are recorded. They need not be complicated, but they must be objective, simple, and specific. UNIT 8: E XERCISE EVALUATION Page 8.10 Exercise Design Evaluation Methodology (Continued) The following table illustrates a checklist for carrying out the four observation steps. ( Note: This checklist also appears as Job Aid 19 in Appendix A.) Sample Observation Checklist Objective Action/Decision to Look For Players to Observe Where Expected Time Notify principals Activate “call-down” procedure Superintendent In EOC, school 10:15 Emergency contact of bus drivers “Call-down” procedure phone Transportation Supervisor EOC, School player 11:05 School closure announcements on TV/radio Message preparation and distribution Superintendent, Emergency Manager, Media EOC, Media player 11:10 Open cafeteria and gym for shelter Notify media, opening and supplying facilities Superintendent, Facilities EOC, School player 12:20 Key Event Monitoring Most exercise scenarios include a number of events specificall y designed to put stress on selected elements of the plan. These are termed key events . Evaluators should pay special attention to these even ts. When a key event message is input, the evaluators monit or the participants’ responses to the event. All responses should be noted on a Key Event Response Form. This form provides for multip le responses from several positions within the EOC as well as responses from outside the EOC. Problem Log The Problem Log allows participants, controllers, and sim ulators to document any observed action that may possibly create a p roblem. (While observing, it should be understood by everyone that what may seem to be a plan or procedure problem may actually be a participant or simulator error.) These potential problems can then be analyzed after th e exercise to determine which are serious enough to require corrective action and to determine their source(s) plan, preparedness, training, or simulation.

UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.11 Evaluation Methodology (Continued) Sample Forms Examples of the following forms are provided on the n ext few pages. ( Note: These forms also appear as Job Aid 20-25 in Appendix A .) Evaluator Checklist Narrative Summary Form Key Event Response Form Problem Log Exercise Debriefing Log Exercise Critique Form UNIT 8: E XERCISE EVALUATION Page 8.12 Exercise Design Evaluator Checklist Evaluator: __________________________ Date: _______________ Location: ___________________________ Objective No.: Function Being Evaluated:

Objective: Performance Criterion [#] Points of Review: Please answer the following: Y = Yes, N = No, NA = Not Applicable, NO = Not Observed Y N NA NO 1. 2. 3. 4. Comments: UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.13 Narrative Summary Objective Number: ______________________ Criterion Number: ________________ Evaluator: ______________________________ Location: ______________________ Issue: A specific statement of the problem, plan, or proce dure that was observed.

___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ Discussion:

A discussion of the issue and its specific impact o n operational capability.

___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ Corrective Action Recommendation: Recommended course(s) of action to improve performa nce or resolve the issue to improve operational capability.

___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ UNIT 8: E XERCISE EVALUATION Page 8.14 Exercise Design Narrative Summary (Continued) Office of Primary Responsibility: The department, agency, or organization responsible for implementation of corrective actions. Department, Agency, or Organization: ___________________________________________________ _________ Individual Responsible: ___________________________________________________ _________ Title: ________________ Date Assigned: ___ / ___ / ___ Suspense Date: ___ / ___ / ___ UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.15 Key Event Response Form Event No. Scheduled Date/Time Initially Input To Actual Date/Time Response Date/Time Position Responding Action Taken UNIT 8: E XERCISE EVALUATION Page 8.16 Exercise Design Problem Log Date: __________________________ Exercise Assignment: ____________________________ Tel. No: ______________________ Time Message Library No. (if known) Problem Analysis (Leave Blank) UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.17 Postexercise Meetings There are two types of postexercise meetings: The playe r debriefing and meetings of the evaluation team. Player Debriefing A short exercise debriefing should be conducted with the players immediately after the exercise. This debriefing gives them an opportunity to have their say about how things went, what they th ink should be changed, and commitments that they might make. This is how a debriefing generally goes: The controller conducts the debriefing, beginning with a review of broad objectives and commenting generally on both successes and shortfalls. The controller asks for a brief response (about 2 minutes each) from each player. As participants comment on their performance, the controller tries to maintain a balance between positive and negative comme nts and encourages everyone to contribute. Comments during the debriefing should be recorded for inclusion in the After Action Rep ort. An Exercise Debriefing Log may be used for this purpose. A n example is provided at the end of this section. UNIT 8: E XERCISE EVALUATION Page 8.18 Exercise Design Postexercise Meetings (Continued) The debriefing is for exercise participants. If evaluators wish to say a few words, they should concentrate on all the positive a spects of the exercise. It is a good idea to prepare a simple questionnaire (such as the Exercise Critique Form) for participants to fill out af ter the exercise. People who hesitate to enter into group discussions will often respond to a questionnaire. One possible format is a set of objective questions requiring only a check mark response, along wit h some open-ended questions about the performance in general (e.g., “What was good or bad about the notification procedure?”).

Keeping the Debriefing on Track The purpose of the debriefing is to examine player pe rformance. However, players will often want to critique the exer cise itself: It was too long, too short, had too many or too few messages. The goal is to keep the players on track, focusing on performance insofar as p ossible. Explain that they will have an opportunity later to provide input into exercise design. Briefly review the Exercise Debriefing Log and Exercise Critique Form on the following pages. Note: These forms also appear as Job Aids 24 and 25, respectively, in Appendix A. UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.19 Exercise Debriefing Log Exercise Debriefing Log Recorder _________________________ Date ____________ _____________ Problem Summary Recommended Action Responsible Agency /Person Exercise _______________________ UNIT 8: E XERCISE EVALUATION Page 8.20 Exercise Design Exercise Critique Form Please take a few minutes to fill out this form. Your opinions and suggestions will help us prepare better exercises in the future. 1. Please rate the overall exercise on the following sca le.

1 2 3 4 5 6 7 8 9 10 Very Poor Very Good 2. Compared to previous exercises, this one was:

1 2 3 4 5 6 7 8 9 10 Very Poor Very Good 3. Did the exercise effectively simulate the emergency e nvironment and emergency response activities? Yes _________ No __________ If no, briefly explain why: 4. Did the problems presented in the exercise adequatel y test readiness capability to implement the plan? Yes _________ No _____ _____ If no, briefly explain why: UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.21 Exercise Critique Form (Continued) 5. The following problems should be deleted or revised :

6. I suggest that you add the following problems for t he next exercise.

7. Please add any other comments or suggestions. UNIT 8: E XERCISE EVALUATION Page 8.22 Exercise Design Postexercise Meetings (Continued) Evaluation Team Meetings Evaluation team meetings are held to analyze the exe rcise and prepare the After Action Report. Evaluation team meetings ma y include:

A meeting of the evaluation team(s) shortly after the exercise to compare notes. A more formal meeting of the team a week or so later to analyze the findings and develop an accurate account of what worked and what did not. The team analyzes evaluation responses and an y other data, and discusses how well each of the objectives was met. Additional meetings as needed to analyze data and pre pare the report. The exercise design team may join the evaluat ion team at one or more meetings to offer feedback and suggestions. The report should be prepared within one to three weeks after th e exercise, while memories are still fresh. After-Action Report The findings of the evaluation team meetings are compi led in the after- action report, which documents the effectiveness of the ex ercise. It serves as the basis for planning future exercises, upgradin g the EOP or contingency plan, and taking corrective action. The after-action report is the responsibility of the ev aluation team leader or chief evaluator, working with the evaluation team .

Form The report may take a variety of forms. For example , a small exercise may warrant only a brief summary of the minutes of th e player debriefing, followed by a few recommendations. Sometimes a memo w ill do the job. For large exercises particularly functional or full-scale exercises the report should be specific and comprehensive.

Format There is no set organizational plan for an after-actio n report. However, the topics listed in the following outline are usually covered.

UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.23 After-Action Report (Continued) Sample Report Outline Introduction (Main purpose of the report, why it is being submitted , preview of main topics, evaluation methodology used, and perhaps a general summary of main problems and recommendations.) Statement of the Problem (Purpose of the exercise) Exercise Summary Goals and Objectives Preexercise Activities Participants and Agencies Description of Exercise Scenario Accomplishments and Shortfalls Evaluation Group Findings Summary of Postexercise Debriefing Recommendations Training Needs Changes in the Emergency Plan Other Corrective Actions A copy of the after-action report should go to the chie f elected official of the jurisdiction (or head of the organization involve d) and each participating entity. Other Reporting Requirements Other types of exercise reports are often required by a gencies mandating exercises. Often they are simple checklists which require l ess time to prepare than a full written report. Check with the appropriate regulatory agencies for specific reporting requirements. UNIT 8: E XERCISE EVALUATION Page 8.24 Exercise Design Implementing Change Recommendations for the future are the whole reason fo r conducting the exercise. The goals of an exercise are not achieved unti l the recommendations that come out of the evaluation are im plemented. The purpose of the evaluation is to improve the emergency management plan and the organization’s performance in carrying out tha t plan.

This is most likely to occur if objectives relate clearly t o emergency functions and the focus of the evaluation is on performa nce, not people. Specifically, the change effort centers around these issues: Are the procedures sound? Are resources sufficient to support the procedures? Are personnel adequately trained to follow the proced ures and use resources?

Remember: We test plans, but we train people.

UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.25 Activity Activity: Plan the Evaluation In Unit 4, you completed several design steps for a tablet op or functional exercise. In that unit, you:

Defined the scope.

Wrote a statement of purpose.

Developed three objectives.

Outlined a narrative.

Wrote two major events and two detailed events for each of them.

Composed a message.

Review the materials that you developed in those activitie s, then complete the following questions with regard to the sam e exercise.

1. What size of evaluation team and what team structur e would be appropriate for your exercise?

2. Who would you select for your evaluation team? Wh at resources are available to you for obtaining qualified evaluators? 3. What kind of training would you provide for the evaluators?

UNIT 8: E XERCISE EVALUATION Page 8.26 Exercise Design Activity: Plan the Evaluation (Continued) 4. Briefly describe what evaluation methodology would be appropriate for your exercise. Consider:

Observation strategies.

Documentation.

Forms.

Followup.

5. What type of report should be prepared, and how w ill it be generated?

6. How would you ensure that needed changes are implem ented after the exercise?

UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.27 Summary and Transition Unit 8 provided a general overview of the process for evaluating exercises. The next unit discusses materials and methods for enhancing an exercise simulation.

Reference Library For More Information FEMA exercise evaluation courses: Exercise Evaluation Course (G130): http://training.fema.gov/emiweb/nrcrs.htm Radiological Emergency Preparedness Exercise Evaluation ( E304): http://training.fema.gov/emiweb/rclist2002.htm UNIT 8: E XERCISE EVALUATION Page 8.28 Exercise Design Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer that you have chosen. 1. Good evaluation can help the organization identif y:

a. Training and staffing deficiencies. b. Qualified members of the evaluation team.

c. Exercise objectives.

d. The scope of the exercise. 2. Evaluation begins when the exercise gets underway usually when the narrative is presented.

a. True b. False 3. It is usually best to keep the design team and the e valuation team separate and to bring in an outside evaluation team leader.

a. True b. False 4. Every exercise should have an evaluation director, se veral teams of evaluators headed by team leaders, and a means of communicating among the te ams.

a. True b. False 5. The evaluation team leader is primarily responsible for the evaluation methodology, selecting evaluation team members, training the team, and preparing the evaluation report.

a. True b. False 6. During the exercise, evaluators should: a. Ensure that players are aware of when they are bei ng observed.

b. Offer suggestions that can improve the exercise results. c. Focus on the positive.

d. Avoid attracting players’ attention. UNIT 8: E XERCISE EVALUATION Exercise Design Page 8.29 Knowledge Check (Continued) 7. One thing evaluators should focus on during an exerci se is:

a. Explaining evaluation methodology to the players being observed.

b. Having players explain their reasons for actions take n.

c. Noting what actions are taken in response to key even ts.

d. Finding as many positive points as negative ones. 8. A constructive evaluation strategy would be to have players, simulators, and controllers document problems they observe during the exercise. a. True b. False 9. The main purpose of the postexercise debriefing is: a. To inform the participants of the evaluation result s.

b. To give the players a chance to comment on the exerci se performance. c. To obtain feedback on the exercise design.

d. To prepare the After Action Report. 10. The ________________ documents the effectiveness of the exercise and serves as the basis for taking corrective actions. a. Evaluation Plan b. Problem Log c. After Action Report d. Evaluator Checklist UNIT 8: E XERCISE EVALUATION Page 8.30 Exercise Design Answers Knowledge Check (Continued) 1. a 2. b 3. b 4. b 5. a 6. d 7. c 8. a 9. b 10. c Unit 9: Exercise Enhancements UNIT 9 EXERCISE ENHANCEMENTS Exercise Design Page 9.1 Introduction Sometimes creativity is the difference between an adequ ate exercise design and one that really gets the participants excited and involved. This unit discusses the various ways in which you can enhance an exercise with equipment, displays, people, props, and ot her strategies.

Unit 9 Objectives After completing this unit, you should be able to: Define the purpose of exercise enhancements. Identify resources available for exercise enhancement. Describe the benefits of specific exercise enhancement techni ques.

UNIT 9: E XERCISE ENHANCEMENTS Page 9.2 Exercise Design Why Use Enhancements?

The point of an exercise is to simulate an emergency as r ealistically as the type of exercise warrants. The more realistic the sce nario, the setting, the atmosphere, and the equipment and materi als made available to the participants, the more likely it is that the pl ayers will “get into” the action and get the most out of it. A variety of exercise enhancements can help achieve this r ealism. In a drill or full-scale exercise, the use of real equipment and actual locations is inherently realistic. Added touches such as simulated victims with convincing mock injuries can make a functional or full-scale exercise even more realistic. A tabletop or functional exercise must rely on material s and devices that you can bring into a room to make the exercise more re alistic.

It’s not necessary to spend a lot of money or energy to enhance realism. Resources can include ordinary items that are currently av ailable in the operations center or command post, or common items that ca n be easily obtained. Creativity Creativity is the key to good enhancements. There are many low-cost creative approaches you can try. Here are some examples: Videotape simulated “news broadcasts” depicting the disaste r and taped interviews with “victims.” Show these prerecorded clips as part of the exercise. Audiotape news broadcasts and play them on the radio. Use make-up and props to simulate injuries. Use computers to chart plumes and provide data on river flows.

If the telephone system is down as part of the exercise, leave the telephones in the EOC but don’t connect them. If power is supposed to be out, then actually run a back up generator. Turn off the lights and computers if they’re not hooke d up to the generator. (Of course, this can disrupt the real office work flow, but it will result in a more realistic exercise.) These are just a few examples to get your creative juice s flowing. We’ll discuss more ideas in the remainder of the unit, and you will undoubtedly have ideas of your own. That’s the whole idea!

UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.3 Communications Equipment Various types of electronic equipment can be used to commu nicate the narrative and advance the scenario. Communications equi pment can also be used to transmit some of the messages from simulato rs. In a highly simulated exercise, such as a complex functional exe rcise, try to transmit messages as you would in a real emergency by phone, radio, and TV.

Take Advantage of What’s Available There are always financial limitations, so plan to make use of the communications equipment that your community normally h as available during an emergency. Consider the following: Landlines and cellular telephone Radio phones and portable/handheld radios Hotline dedicated phones Military phone hookups Citizens’ Band (CB) Teletype systems and fax machines Amateur Radio Emergency Service (ARES) and Radio Amateur Civil Emergency Service (RACES) Monitors/scanners National Oceanic Atmospheric Administration (NOAA) Weather Radio National Warning System (NAWAS) Computerized Radio Packet UNIT 9: E XERCISE ENHANCEMENTS Page 9.4 Exercise Design Visuals No exercise is complete without a collection of visuals and displays: Maps, charts, status boards, black or white boards, chart pa per and easels, and the like. Videotapes and slides, although m ore difficult to obtain, can also enhance the realism of your exercise.

Maps Uses: Maps provide context and detail for a scenario. Because they are essential to the handling of an actual disaster, they ar e necessary for all types of exercises. Even in an orientation or a tablet op exercise, maps provide useful information and give players a clearer picture of the simulated event. For example, they may be used to po sition equipment or to determine the nearest facilities for resource depl oyment.

Formats: Maps may be reproduced on paper for individual use o r displayed on a wall. Acetate overlays make it possible to mark off areas or monitoring points and to reuse the map. Types: The types and number of maps required depends on the exercise type and the hazard being tested. Below is a partial list of maps that you may find useful. Types of Maps City street maps County street maps Subdivision maps Sewer maps (mains/facilities) W ater maps (mains/facilities) Electric maps (lines/facilities) Gas maps (lines/facilities) Flood plain maps Contour maps Police and fire district maps Center city (downtown) maps Facility maps (plant layout, rail yards, etc.) W eather maps Sources: Good maps can be obtained from a number of agencies. Before trying your hand as a cartographer, try local m unicipal agencies such as the City Planning Commission, Department of High ways, Engineering Department, Public Works, etc. The U.S. G eological Survey (USGS) is another good source of maps related to earthq uakes. FEMA’s National Flood Insurance Program has flood maps. NOAA has weather maps, satellite weather imagery, and weather photos. UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.5 Visuals (Continued) Charts Used in Exercises The accumulation and sharing of information is an impor tant operations center function. To ensure coordinated and timely emer gency response, visual displays allow everyone to comprehend quickly wha t actions have been taken and what resources and personnel are availab le. Display needs will vary with the nature and scope of th e exercise, but the charts listed below should be considered. Some displays ar e for use of the exercise staff only. Type of Chart Description Problem and Event Log Large events display board for posting major events Should be available for all in the EOC to review, a lso useful for EOC shift- change briefings All major problems that are reported are entered on the log as they are received May be divided into columns: Nature of problem, pr oblem number, assignment, response, remarks Damage Assessment Chart Divided into columns: Areas reporting damage, time of report, extent of damage Facility Charts and Status Boards Used to track facilities involved in the exercise s o participants are aware of available resources (companion maps are useful); ex amples of facility charts: Hospitals: Beds available, blood and other supply needs, personnel Congregate care facilities (e.g., those run by Red Cross or social service agencies): Space available; status of food, water, bedding, medical stocks Law enforcement resources: Numbers and locations o f sworn, reserve, and auxiliary personnel; status of mutual aid units Fire resources: Deployment and availability of fir e units, status of fire mutual aid forms Organization Charts Useful for staff as a means of anticipating what ag encies should be coordinating or reporting to other agencies Optional Master Scenario of Events List Mainly for controller’s use, to keep exercise on sc hedule; should not be seen by players Contains detailed sequence of events developed as p art of scenario Simulation Plotting Map Used by controller and simulators Depicts prescripted input exercise information Coded markers may be used to depict actions taken b y various organizations (police, fire, medical/health, public works, utilities, Red Cross/voluntary agencies) UNIT 9: E XERCISE ENHANCEMENTS Page 9.6 Exercise Design Visuals (Continued) Videotapes and Slides Videotapes can provide a very realistic presentation of an emergency that can be used to introduce the narrative or to give upda tes. News reports and interviews with politicians and the public can be pr erecorded to lend realism. Slides can be used for some of the same purposes. Computer Resources Many communities have their resources on a computerized i nventory list. Where available, this inventory should be used during conduct of exercises to track resources.

Miscellaneous Equipment and Supplies Sometimes exercise planners are so focused on the dramatic aspects of the exercise that they overlook common equipment and m aterials. Many of these items, available in most office stores at little cost, are indispensable. Equipment and Supplies to Consider Projector for overhead transparencies or slides Copy machine Portable radio Pagers and cellular phones Public address system Pens, pencils, and markers Chart paper Telephone books and directories Local and state contact lists EOC phone directory List of cellular phone numbers Name tags Resource lists Financial cost report form UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.7 People and Props In a full-scale exercise, the sense of reality occurs throug h the use of actual equipment in the actual setting. In this setting , fire trucks and the local airport would be considered types of enhancements. Designers of full-scale exercises also bring in real people or props to enhance the realism. For example, an exercise simulatin g a hotel fire could use: People playing the part of victims, made up to appear injured.

Mannequins to represent the dead (or victims trapped un der heavy beams).

Fake smoke.

Burnt boards and beams strategically placed at the event site.

A contained fire that the fire department would be r equired to extinguish.

Some jurisdictions use considerable ingenuity in these mat ters creating model cities to use in tabletop exercises or obtaining ma nnequins to substitute for humans in dangerous situations (e.g., trap ped under a beam). Such enhancements are limited only by budget, safety con siderations, and the imagination. So let your creativity work to enhance the exercise you are planning! UNIT 9: E XERCISE ENHANCEMENTS Page 9.8 Exercise Design Enhancement Resources One of the problems that designers often face is how to obtain materials, people, and equipment that lend realism without brea king a very small budget. Consider soliciting volunteers and donations fr om other agencies in the community. Many are civic-minded and are happy to help out by lending equipment or providing volunteers to serve as victims. Below are some places to begin. Potential Resources Hospitals Schools and colleges Business and industry Chemical Transportation Emergency Center (CHEMTREC) Boy and Girl Scouts Search and rescue volunteers Agencies Railroads The Salvation Army Public transportation Fire departments Police departments Chemical companies Amateur radio clubs Religious organizations National Guard/Military The American Red Cross Service organizations (Elks, Lions, Rotary, etc.) Costs and Liability In obtaining resources, it is important to consider costs (b oth initial and hidden) and the potential for liability. For examp le, consider:

Person hours expended in obtaining and returning equi pment or materials.

Potential for damage or replacement costs.

Arrangements for timely returns when items are borrowe d or volunteers are “on loan.” (Too often “victims” have b een left at the scene of the emergency site or the hospital because transpo rtation plans were not made.) Safety and liability concerns when using people and equ ipment. Be sure that provisions are in place for liability insurance and equipment replacement. UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.9 Enhancement Logistics In planning for exercises, someone needs to take responsib ility for managing the logistics related to enhancements. Be sure that the following questions are answered: How will enhancements be used?

Where will props be placed?

Who will be in charge of props?

How will people and props be picked up, transported, a nd returned?

What kind of return policy can be worked out for borr owed materials and equipment?

In what condition must equipment be returned?

Who will clean it?

Is normal wear a concern?

UNIT 9: E XERCISE ENHANCEMENTS Page 9.10 Exercise Design Activity Activity: Enhance a Scenario Read the following scenario. What exercise enhancements co uld be used to add to the realism of this exercise? Write your i deas on the next page. Functional Exercise: Potential Airplane Crash Participating organizations: Local dispatch, fire, police, search and rescue, EMS, eme rgency manager, mayor, airport crash/fire rescue, public works, hospital Location: Your EOC or operations center A jumbo jet that has experienced an inexplicable in-fl ight engine problem en route from Panama to New York will need to make an emergency landing a t a large airport along the route. Though plans have been made to land at a city 200 miles north that is suitable for landing an aircraft this size, the latest communication with the pilot indicates t hat the plane has lost engine power and is losing altitude too quickly to reach the airport. A lthough the runway at your airport is too short to accommodate a jumbo jet, the only hope to save any of the 285 passengers and crew is to attempt a landing here. Conditions at the airport are clear, but the surroundi ng area is very dry as a result of a continuing drought, and a hot, dry wind is blowing. The airport is in a suburban area. The likelihood of the pilot being able to control the huge plane and land within the runway limits is slim. The glide path passes over several high-density suburban housing developments. The airport control tower has alerted its own crash/fire rescue (CFR) units and has requested that local emergency service units provide fire, police, medical, and search and rescue assistance. Garbled radio communication from the airliner alerts t he airport control tower that the plane’s hydraulic system is not functioning. The pilot’s last messa ge indicates that he will be attempting a soft-impact landing, but the plane breaks apart, crash ing into the housing development and sending debris and bodies over an area of approximatel y one square mile. Smoke is visible in the area. A major part of the fuselage is at the beg inning of the runway. CFR units are proceeding to the main crash site. All available resour ces are en route. Traffic is at a standstill on the only roads leading to the site, however. UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.11 Activity: Enhance a Scenario (Continued) What enhancements could be used to increase the realism of this scenario? Write your ideas below. Possible Enhancements UNIT 9: E XERCISE ENHANCEMENTS Page 9.12 Exercise Design Activity: Enhance a Scenario (Continued) Suggested Answers The following enhancements would contribute to the rea lism of this scenario. However, there are many ways to enhance an e xercise, and you may have other ideas.

City street maps.

Subdivision maps for affected residential areas.

Airport map.

Fire district maps.

Map/diagram of crash area, staging areas, command posts, ingress/egress routes (Acetate overlay for updating movem ent of equipment and personnel are suggested.) Resource status board showing available equipment (types, numbers) and assigned locations. (Computerized tracking if available.) ICS organization chart.

Damage assessment chart.

Communications equipment to simulate communications among all involved agencies.

Prerecorded audiotape of pilot’s transmissions to tower.

Prerecorded background-noise audiotape of crash site (sire ns, fire, voices, etc.) for atmosphere (Could be accompanied by a prerecorded video footage of firefighting, smoke.) Master Scenario of Events List.

Simulation plotting map.

UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.13 Activity Activity: Plan Enhancements for Your Exercise Consider the exercise plans that you developed in Unit 4. Answer the following questions about possible enhancements for that exercise.

1. Where will participants be located? Describe what ki nds of facilities would be the most effective.

2. What kinds of communication equipment (if any) woul d be most appropriate for the exercise? What is available in yo ur EOC or other operations center? Where will you obtain what is not readily available?

3. Describe the visuals (maps, charts, status boards, computer resources, videotapes, slides, others) to be used, and wher e you can obtain them.

UNIT 9: E XERCISE ENHANCEMENTS Page 9.14 Exercise Design Activity: Plan Enhancements for Your Exercise (Con tinued) 4. How will you handle people and props? 5. What potential resources do you have for obtaining enhancements?

6. Who will be in charge of enhancement logistics? 7. What costs do you foresee associated with enhancements, a nd how will they be handled? UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.15 Summary and Transition Unit 9 provided information about the types of enhan cements that can be added to increase the realism of an exercise, including co mmunications equipment, maps, charts, display boards, computer resources, videotapes and slides, miscellaneous equipment and supplie s, people and props. In the next unit, you will have an oppor tunity to develop a functional exercise using the information and job aids p rovided throughout the course. Reference Library For More Information FEMA Disaster Mapping and Analysis Center: http://www.gismaps.fema.gov/index.shtml FEMA Flood Hazard Mapping: http://www.fema.gov/mit/tsd/ U.S. Geological Survey, National Mapping Information : http://mapping.usgs.gov/ NOAA National Climatic Data Center: http://lwf.ncdc.noaa.gov/oa/ncdc.html NOAA Photo Library: http://www.photolib.noaa.gov/ NOAA 3-D Weather Images: http://www.nnvl.noaa.gov/ UNIT 9: E XERCISE ENHANCEMENTS Page 9.16 Exercise Design Check Knowledge Check Carefully read each question and all of the possible answ ers before selecting the most appropriate response for each test item. Circle the let ter corresponding to the answer that you have chosen. 1. The purpose of exercise enhancements is to: a. Increase credibility with the public and the media.

b. Make the exercise more fun so more personnel will pa rticipate.

c. Increase the realism so participants will respond as they would in a real event.

d. Make it easier on the controller and the simulators. 2. In most tabletop and functional exercises, messages can’t be transmitted without sophisticated communications equipment.

a. True b. False 3. Because they are essential to the handling of an actu al emergency, maps are necessary for all types of exercises.

a. True b. False 4. _________ are an important enhancement because they allow everyone to comprehend quickly what actions have been taken and what resources a nd personnel are available.

a. Charts b. People and props c. Slides d. Audiotapes and videotapes 5. It is a good idea to seek out potential sources of ma terials, people, and equipment in the local community to enhance an exercise.

a. True b. False UNIT 9: E XERCISE ENHANCEMENTS Exercise Design Page 9.17 Answers Knowledge Check (Continued) 1. c 2. b 3. a 4. a 5. a Unit 10: Designing a Functional Exercise UNIT 10 DESIGNING A FUNCTIONAL E XERCISE Exercise Design Page 10.1 Introduction In this unit you will put together all of the skills a nd knowledge that you have learned from previous units and develop a functio nal exercise. In Unit 4, you learned the eight exercise design steps: 1. Assess needs. 2. Define scope. 3. Write a statement of purpose. 4. Define objectives. 5. Compose a narrative. 6. Write major and detailed events. 7. List expected actions. 8. Prepare messages. You have already practiced applying each of the design steps individually. Now you will put these steps together i n one coordinated process. As an option, you may also develop the four ma jor exercise documents (templates for which are found in the Exercise Design Toolkit at http://training.fema.gov/emiweb/IS/139lst.asp ):

Exercise Plan Control Plan Evaluation Plan Player Handbook UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.2 Exercise Design Unit 10 Objective After completing this unit, you should be able to: Design a small functional exercise using the eight-step de sign process. Select Your Exercise In Unit 2, you outlined a comprehensive exercise progra m for your organization. Review that outline now. If you wish to revise or update the outline, you may do so. Select a small functional exercise from your comprehensive plan. The exercise that you choose should be fairly small, for the sake of the time required to complete this course activity. If necessary, y ou may artificially limit a functional exercise in your plan. If you used a functional exercise as the basis of your abb reviated design activities in Unit 4, you may either use the same exerci se here (and develop it now in depth) or select a new one. When you have selected your exercise, you may proceed wi th the design steps, as outlined in the remainder of this unit. As yo u undertake each design step, feel free to go back to earlier units and review the guidelines and examples that are provided. UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.3 Step 1: Assess Needs The first step in developing any exercise is to assess the n eeds of your emergency management program. Indicate below the most pressing problem areas in your community or organization. Base your assessment on your review of the emergency plans and procedures an d what you have learned from previous exercises or emergencies. Exercise Needs Assessment 1. Hazards List the various hazards in your community or organizat ion. What risks are you most likely to face? You can use the following checklist as a starting point. Note: If your community has already conducted a hazard analysis, that is the best resource.

Airplane crash Sustained power failure Dam failure Terrorism Drought Tornado Epidemic (biological attack) Train derailment Earthquake Tsunami Fire/urban conflagration Volcanic eruption Flood Wildfire Hazardous material spill/release Winter storm Hurricane Workplace violence Landslide/mudslide Other ______________________ Mass fatality incident Other ______________________ Radiological release Other ______________________ UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.4 Exercise Design Step 1: Assess Needs (Continued) 2. Secondary Hazards What secondary effects from those hazards are likely to i mpact your organization?

Communication system breakdown Power outages Transportation blockages Business interruptions Mass evacuations/displaced population Overwhelmed medical/mortuary services Other ________________________ Other ________________________ Other ________________________ Other ________________________ Other ________________________ 3. Hazard Priority What are the highest priority hazards? Consider such fa ctors as:

Frequency of occurrence.

Relative likelihood of occurrence.

Magnitude and intensity.

Location (affecting critical areas or infrastructure).

Spatial extent.

Speed of onset and availability of warning.

Potential severity of consequences to people, critical faci lities, community functions, and property.

Potential cascading events (e.g., damage to chemical proce ssing plant, dam failure).

#1 Priority hazard: #2 Priority hazard: #3 Priority hazard: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.5 Step 1: Assess Needs (Continued) 4. Area What geographic area(s) or facility locations are most vu lnerable to the high priority hazards? 5. Plans and Procedures What plans and procedures emergency response plan, contingency plan, operational p lan, Standard Operating Procedures (SOPs) will guide your organization’s response to an emergency? 6. Functions What emergency management functions are most in need of rehearsal? (E.g., what functions have not been exercised recently? Where have difficulties occurred in the past?) You can use the following checklist as a starting point. Alert Notification (Emergency Response) Public Safety Warning (Public) Public Works/Engineering Communications Transportation Coordination and Control Resource Management Emergency Public Information Continuity of Government or Operations Damage Assessment Other ___________________________ Health and Medical Other ___________________________ Individual/Family Assistance Other ___________________________ UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.6 Exercise Design Step 1: Assess Needs (Continued) 7. Participants Who (agencies, departments, operational units, personnel ) needs to participate in an exercise? For example: Have any entities updated their plans and procedures? Have any changed policies or staff? Who is designated for emergency management responsibilit y in your plans and procedures? With whom does your organization need to coordinate i n an emergency? What do your regulatory requirements call for?

What personnel can you reasonably expect to devote to d eveloping an exercise?

8. Program Areas Mark the status of your emergency program in these and other areas to identify those most in need of exercising. New Updated Exercised Used in Emergency N/A Emergency Plan Plan Annex(es) Standard Operating Procedures Resource List Maps, Displays Reporting Requirements Notification Procedures Mutual Aid Pacts Policymaking Officials Coordinating Personnel Operations Staff Volunteer Organizations EOC/Command Center Communication Facility W arning Systems Utility Emergency Preparedness Industrial Emergency Preparedness Damage Assessment Techniques UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.7 Step 1: Assess Needs (Continued) 9. Past Exercises If your organization has participated in exercises befor e, what did you learn from them, and what do the results indicate about future exercise needs? For example, consider the following questions: Who participated in the exercise, and who did not?

To what extent were the exercise objectives achieved?

What lessons were learned?

What problems were revealed, and what is needed to re solve them?

What improvements were made following past exercises, an d have they been tested?

UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.8 Exercise Design Step 2: Define the Scope Working from your needs assessment and the Self-Assessment t hat you completed in Unit 3 about your organization’s readine ss for the exercise process, define the scope of the exercise (i.e., draw some meaningful limits). Make decisions according to (a) highest priority and (b) what can be realistically addressed in one exercise. Record your d ecisions below.

Scope Worksheet 1. Highest priority hazards (major and secondary): 2. Geographic areas/locations of greatest vulnerability to these hazards:

3. Agencies/departments/organizational units: List belo w the entities that have a significant role in emergency management/response. Then enter check marks in any columns that apply. Agency/Organization Limited experience with major emergencies New plans, staff, or organizational structure not yet exercised Problems revealed in prior exercises UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.9 Step 2: Define the Scope (Continued) 4. Types/levels of personnel that you want to have in the exercise:

Policy making (elected officials, chief operating office rs, department heads) Coordination (managers, EOC representatives, departme nt deputies) Operations (field personnel, headquarters staff level) Public representatives (media, PIOs, general public) Other: _____________________________________ 5. Types of operations/functions that you want participa nts to engage in (e.g., notification, evacuation): 6. Degree of stress, complexity, time pressure that the e xercise should have:

High Medium Low Stress Complexity Time pressure UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.10 Exercise Design Step 2: Define the Scope (Continued) Scope: Type of Emergency: Location: Functions: Organizations and Personnel: Exercise Type: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.11 Step 3: Write a Statement of Purpose Develop your statement of purpose using the following template.

Statement of Purpose The purpose of the proposed emergency management exerci se is to improve the following emergency operations:

a. __________________________ b. __________________________ c. __________________________ d. __________________________ by involving the following agencies/organizations/departments :

a. __________________________ b. __________________________ c. __________________________ d. __________________________ e. __________________________ f. __________________________ g. __________________________ h. __________________________ in a ________________________ exercise simulating a ________________________ [type of exercise] [type of emergency ev ent] at ________________________________________ on ___________________.

[location] [date] UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.12 Exercise Design Step 4: Define the Objectives In the space below, write objectives that support the pu rpose statement that you composed in the last step. Remember the SMART system: Simple, Measurable, Achievable, Realistic, Task-oriented . Use extra paper if needed. Objectives Obj. No. Objective Organization UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.13 Step 5: Compose a Narrative Outline the key points in your narrative using the fo llowing worksheet, then compose the script for the initial narrative in yo ur exercise.

Narrative Outline Event: How fast, strong, deep, dangerous: How you found out: Response made: Damage reported: Sequence of events: Current time: Advance warning: Location: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.14 Exercise Design Step 5: Compose a Narrative (Continued) Relevant weather conditions: Other factors that would influence emergency procedures: Predictions: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.15 Step 5: Compose a Narrative (Continued) Narrative Script UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.16 Exercise Design Step 6: Write Major and Detailed Events In the space provided on the following pages, list even ts that might occur in your emergency scenario. Break down each of the maj or events into two or three detailed events. Step 7: List Expected Actions For each detailed event, write some actions that you wo uld expect from participants involved in the exercise. Be sure to tie t hem to specific objectives. ( Note: You may wish to number the detailed events and expected actions for later reference.) UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.17 Step 7: List Expected Actions (Continued) Events and Actions Planning Sheet Obj.

No. Major Events Detailed Events Expected Actions Organizations UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.18 Exercise Design Step 8: Prepare Messages In the following message planning sheet, plan messages th at will generate all of the expected actions that you identifi ed. Then, compose the messages using the forms that follow. Message Planning Sheet Detailed Events Expected Actions OrganizationsMessage Outline UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.19 Step 8: Prepare Messages (Continued) Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.20 Exercise Design Step 8: Prepare Messages (Continued) Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.21 Step 8: Prepare Messages (Continued) Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.22 Exercise Design Step 8: Prepare Messages (Continued) Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: Emergency Exercise Message Emergency Exercise Messa ge To: Method: From: To: Method: From: Content: Content: UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.23 Constructing the Master Scenario of Events List In the space below, develop a Master Scenario of Events List (MSEL) for the exercise that you have just designed.

Sample Master Scenario Events List Time Message/Event Expected Actions UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.24 Exercise Design Constructing the Master Scenario of Events List (Co ntinued) Sample Master Scenario Events List Time Message/Event Expected Actions UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.25 Planning the Exercise Evaluation In the space below, briefly describe your plan for eval uating the exercise.

Exercise Evaluation Plan 1. Evaluation Team: How many members and what team structure? Who will you select, and where will you recruit them? What kind of training will be provided?

UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.26 Exercise Design Planning the Exercise Evaluation (Continued) 2. What methodology will be used to evaluate the exe rcise?

Observation strategies:

Documentation:

Forms:

Followup (What type of report? How will you ensure that needed changes are implemented?) UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.27 Planning the Exercise Enhancements Using the following form, indicate your plans for facil ities, displays, materials, equipment, and other exercise enhancements. Exercise Enhancements Communications Equipment: Hard-line telephone Radio phones Hot-line dedicated phones Portable/hand-held radios Cellular telephones Military phone hook-ups Citizens’ Band (CB) Teletype systems Fax machines Amateur Radio Emergency Service (ARES) Radio Amateur Civil Emergency Service (RACES) Monitors/scanners NOAA Weather Radio National Warning System (NAWAS) Computerized Radio Packet Maps: City street maps County street maps Subdivision maps Sewer maps (mains/facilities) Water maps (mains/facilities) Electric maps (lines/facilities) Gas maps (lines/facilities) Flood plain maps Contour maps Police and fire district maps Center city (downtown) maps Facility maps (plant layout, rail yards, etc.) Weather maps UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.28 Exercise Design Planning the Exercise Enhancements (Continued) Charts: Problem and Event Log Damage assessment chart Facility charts Status boards (specify): Organization charts Simulation plotting map Other:

Other: Computer resources Videotapes Audiotapes Slides Projector Copy machine Portable radio Pagers and cellular phones Public address system Pens, pencils, and markers Chart paper Telephone books and directories Local and state contact lists EOC phone directory List of cellular phone numbers Name tags Resource lists Financial cost report form UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Exercise Design Page 10.29 Activity Optional Activity: Exercise Design Documents Using the templates provided in the Exercise Design Too l Box (see the “Templates” directory), develop the following document s for the functional exercise you have just designed: Exercise Plan Control Plan Evaluation Plan Player Handbook UNIT 10: D ESIGNING A FUNCTIONAL EXERCISE Page 10.30 Exercise Design Summary and Transition Unit 10 provided an opportunity for you to apply th e entire exercise design process in developing a functional exercise. Unit 11 will provide a brief course review and the final exam. Unit 11: Course Summary UNIT 11 COURSE SUMMARY Exercise Design Page 11.1 Introduction This unit will briefly summarize the learning from th e Exercise Design course. After completing this unit you should be able t o summarize the key points of this course.

Reasons to Exercise Exercises are conducted in order to evaluate an organiz ation’s capability to execute one or more portions of its response plan or contingency plan. Exercises can be used to provide individual training and improve the emergency management system. Reasons to perform exercises include:

Testing and evaluating plans, policies, and procedures.

Revealing planning weaknesses and resource gaps.

Improve individual performance and organizational coo rdination and communications.

Train personnel and clarify roles and responsibilities.

Gain program recognition.

Satisfy regulatory requirements. Comprehensive Exercise Program A comprehensive exercise program is made up of progressive ly complex exercises, each one building on the previous one, until the exercises are as close to reality as possible. The program must be caref ully planned to achieve identified goals and should involve a wide ran ge of organizations in its planning and execution.

UNIT 11: C OURSE SUMMARY Page 11.2 Exercise Design Comprehensive Exercise Program (Continued) Types of Exercises There are five main types of activities in a comprehensi ve exercise program:

Orientation seminar: This is a low-stress, informal discussion in a group setting with little or no simulation. It is used to provide information and introduce people to policies, plans, an d procedures.

Drill: This is a coordinated, supervised exercise used to test a single specific operation or function. It involves deployment of equipment and personnel. Tabletop exercise: This is a facilitated group analysis of an emergency situation in an informal, stress-free environm ent. It is designed for examination of operational plans, proble m identification, and in-depth problem solving. Functional exercise: This is a fully simulated interactive exercise that tests the capability of an organization to respond to a simulated event. It takes place in the EOC and focuses on coordina tion of multiple functions or organizations. Strives for reali sm, short of actual deployment of equipment and personnel. Full-scale exercise: This is a simulated emergency event, as close to reality as possible. It involves all emergency respon se functions and requires full deployment of equipment and personn el.

Building an Exercise Program Building an exercise program is a multiorganization te am effort that includes:

Analysis of capabilities and costs.

Goal setting.

Development of a long-term plan.

Scheduling of tasks.

Public relations efforts.

UNIT 11: C OURSE SUMMARY Exercise Design Page 11.3 The Exercise Process The process of creating and staging an exercise includes a l engthy sequence of tasks that occur in three phases: Before, duri ng, and after the exercise. Some of the tasks fall under the heading of design and some are part of evaluation.

Major Task Accomplishments A simple way of viewing the exercise process is the sequence of five major task accomplishments:

1. Establishing the base 2. Exercise development 3. Exercise conduct 4. Exercise critique and evaluation 5. Exercise followup Establishing the Base Establishing the base is the process of laying the groundw ork for the exercise. Some important aspects of this preparation are :

Reviewing the current plan.

Assessing capability to conduct an exercise.

Addressing costs and liabilities.

Gaining support.

Assembling and organizing a design team. UNIT 11: C OURSE SUMMARY Page 11.4 Exercise Design The Exercise Process (Continued) Exercise Documents Four major documents are developed during the exercise design process: Exercise Plan Control Plan Evaluation Plan Player Handbook These documents are basically handbooks for particular aud iences. Much of the content of these documents comes from the eigh t exercise design steps. Exercise Design Steps Tabletop, functional, and full-scale exercises are based o n a design process that includes eight steps: 1. Assess needs. 2. Define scope. 3. Write a statement of purpose. 4. Define objectives. 5. Compose a narrative. 6. Write major and detailed events. 7. List expected actions. 8. Prepare messages. These are generally applicable steps regardless of the ty pe of exercise. Each type of exercise has some special considerations in how these steps are applied.

UNIT 11: C OURSE SUMMARY Exercise Design Page 11.5 Exercise Design Steps (Continued) Master Scenario Events List Outputs from the design process are pulled together in t he MSEL, a chart that the controller and simulators can use in keeping th e exercise on track and on schedule. The Tabletop Exercise Exercise Format The tabletop exercise is essentially a group brainstormin g session centered on a scenario narrative and problem statements or messages that are presented to members of the group. The form at is informal, and the exercise is self-evaluated by the participants. Facilitating a Tabletop The facilitator is responsible for: Setting the stage.

Distributing messages.

Stimulating discussion and pushing participants toward in- depth problem solving.

Involving everyone.

Controlling and sustaining the action.

UNIT 11: C OURSE SUMMARY Page 11.6 Exercise Design The Tabletop Exercise (Continued) Designing a Tabletop In applying the eight design steps, the first four steps (needs assessment, scope, purpose statement, objectives) are handled in the normal manner. The remaining steps can be simplified: The narrative can be relatively short.

Only a few major or detailed events are required, an d they are turned into problem statements.

Expected actions must be identified, but they may invol ve such things as discussion or reaching consensus.

Only a few (e.g., 10–15) carefully written messages or problem statements are needed.

The Functional Exercise Exercise Format The functional exercise usually takes place in the operat ing center and involves policy makers and decision makers. It uses an eve nt scenario to test multiple functions or organizations, emphasizing coo rdination and communication. Participants include the: Controller (the manager of the exercise).

Players (people responding to the scenario within their normal roles).

Simulators (people playing the parts of organizations and field units outside of the operations center, who deliver messages to players).

Evaluators (observers who record actions taken in response to messages).

Participants respond in real time, adding an element o f stress to the exercise. Communications equipment, displays, and other enhancements can be used to add to the realism.

UNIT 11: C OURSE SUMMARY Exercise Design Page 11.7 The Functional Exercise (Continued) Controlling a Functional Exercise The controller is responsible for: Monitoring interaction and progress.

Keeping the exercise on track.

Dealing with the unexpected.

Adjusting the pace as needed. (The flow of messages can b e adjusted by adding, deleting, misdirecting, or reassigni ng messages.) Designing a Functional Exercise The full eight-step process is used to develop functional exercises.

The Full-Scale Exercise Exercise Format The full-scale exercise combines the interactivity of the functional exercise with a field element and requires the coordina tion of the efforts of several organizations. It differs from a drill in that a drill focuses on a single operation and exercises only one organization. The full-scale exercise achieves realism through: On-scene actions and decisions.

Simulated victims.

Search and rescue requirements.

Communication devices.

Equipment deployment.

Actual resource and personnel allocation.

UNIT 11: C OURSE SUMMARY Page 11.8 Exercise Design The Full-Scale Exercise (Continued) Participant Roles All levels of personnel are involved, including policy makers, coordination and operations personnel, and field personnel. A contr oller manages the exercise; volunteers simulate victims; evaluators observe a nd keep a log of significant events; and a safety officer ensures that p otential safety issues are addressed. Designing a Full-Scale Exercise After the first four design steps, the following special considerations apply to the design process: The narrative is largely a staged visual scene, so the wr itten narrative can be minimal. The visual narrative must be planned in careful detail.

Major and minor events are often presented visually an d must be carefully planned.

Expected actions must, as always, be specifically identified .

Both visual and prescripted messages are used.

In a full-scale exercise, details are everything. Site Selection The site selected for the event must have adequate space and be as realistic as possible without interfering with normal tr affic or safety.

Scene Management Scene management involves planning and handling: Logistics at the scene.

Creation of a believable emergency scene.

Number of victims.

Management of props and materials.

Number of controllers.

UNIT 11: C OURSE SUMMARY Exercise Design Page 11.9 The Full-Scale Exercise (Continued) Other Special Considerations Other special considerations in a full-scale exercise include :

Managing personnel and resources (many volunteers, lots o f props).

Ensuring that the emergency management system maintains response capability for routine events.

Addressing safety issues.

Attending to issues of legal liability.

Having a plan for emergency call-off.

Working with the media. Exercise Evaluation For an exercise to be useful, it must be accompanied by a n evaluation less formal for the tabletop, structured for the functio n and full-scale. Good evaluations can help the organization identify:

Whether the exercise has achieved its objectives.

Needed improvements in plans, procedures, or guidelines or the emergency management system as a whole.

Training and staffing deficiencies.

Equipment needs.

Need for additional exercising.

The evaluation team leader usually drawn from the design team is responsible for evaluation methodology, selection and t raining of the evaluation team, and report preparation.

UNIT 11: C OURSE SUMMARY Page 11.10 Exercise Design Exercise Evaluation (Continued) Methodology The evaluation methodology includes: Evaluation team structure.

Objectives to be measured.

Evaluation packet (i.e., observation procedures and reco rding forms).

Postexercise Meetings Postexercise meetings include the player debriefing and meetings of the evaluation team to analyze the results and develop th e after action report.

After Action Report The After Action Report should describe the purpose of t he exercise and address goals, objectives, preexercise activities, participan ts, scenario, accomplishments and shortfalls, and recommendations. Exercise Enhancements Exercise enhancements are used to add to the realism of t he exercise. Depending on the type of exercise and available resour ces, enhancements may include:

Communications Equipment.

Visuals.

Other Equipment and Materials.

People and Props.

Resources.

UNIT 11: C OURSE SUMMARY Exercise Design Page 11.11 Next Steps You have now completed IS139 and should be ready to t ake the Final Exam. Complete the final exam in the back of the book by m arking the correct responses. To submit the final exam online, go to http://training.fema.gov/emiweb/IS and click on the courses link. Click on the title for th is course, and scroll down the course description page to locate the final exa m link. After you have selected the final exam link and the online answe r sheet is open, transfer your answers, and complete the personal identif ication data requested. To submit the final exam by mail using the standard an swer sheet, follow the instructions printed on the form.

Appendix A: Job Aids APPENDIX A JOB AIDS Exercise Design Page A.1 Job Aid 1: Exercise Needs Assessment Use this tool to analyze where you may wish to focus your or ganization’s exercise design efforts. In completing this needs assessment, you may wish to consult such resources as planning documents, demographic or corporate data, maps, and training records.

1. Hazards List the various hazards in your community or organizat ion. What risks are you most likely to face? You can use the following checklist as a starting point. Note: If your community has already conducted a hazard analysis, that is the best resource.

Airplane crash Sustained power failure Dam failure Terrorism Drought Tornado Epidemic (biological attack) Train derailment Earthquake Tsunami Fire/Firestorm Volcanic eruption Flood Wildfire Hazardous material spill/release Winter storm Hostage/Shooting Workplace Violence Hurricane Other ______________________ Landslide/Mudslide Other ______________________ Mass Fatality Incident Other ______________________ Radiological Release Other ______________________ APPENDIX A: J OB AIDS Page A.2 Exercise Design Job Aid 1: Exercise Needs Assessment (Continued) 2. Secondary Hazards What secondary effects from those hazards are likely to i mpact your organization?

Communication system breakdown Power outages Transportation blockages Business interruptions Mass evacuations/displaced population Overwhelmed medical/mortuary services Other ________________________ Other ________________________ Other ________________________ Other ________________________ Other ________________________ 3. Hazard Priority What are the highest priority hazards? Consider such fa ctors as:

Frequency of occurrence Relative likelihood of occurrence Magnitude and intensity Location (affecting critical areas or infrastructure) Spatial extent Speed of onset and availability of warning Potential severity of consequences to people, critical faci lities, community functions, and property Potential cascading events (e.g., damage to chemical proce ssing plant, dam failure) #1 Priority hazard: #2 Priority hazard: #3 Priority hazard:

APPENDIX A: J OB AIDS Exercise Design Page A.3 Job Aid 1: Exercise Needs Assessment (Continued) 4. Area What geographic area(s) or facility location(s) is(are) m ost vulnerable to the high priority hazards? 5. Plans and Procedures What plans and procedures emergency response plan, contingency plan, operational p lan, standard operating procedures (SOPs) will guide your organization’s response to an emergency? 6. Functions What emergency management functions are most in need of rehearsal? (e.g., What functions have not been exercised recently? Where have difficulties occurred in the past?) You can use the following checklist as a starting point. Alert Notification (Emergency Response) Public Safety Warning (Public) Public Works/Engineering Communications Transportation Coordination and Control Resource Management Emergency Public Information Continuity of Government or Operations Damage Assessment Other ___________________________ Health and Medical Other ___________________________ Individual/Family Assistance Other ___________________________ APPENDIX A: J OB AIDS Page A.4 Exercise Design Job Aid 1: Exercise Needs Assessment (Continued) 7. Participants Who (agencies, departments, operational units, personnel ) needs to participate in an exercise? For example: Have any entities updated their plans and procedures? Have any changed policies or staff? Who is designated for emergency management responsibilit y in your plans and procedures? With whom does your organization need to coordinate i n an emergency? What do your regulatory requirements call for?

What personnel can you reasonably expect to devote to d eveloping an exercise?

8. Program Areas Mark the status of your emergency program in these and other areas to identify those most in need of exercising. New Updated Exercised Used in Emergency N/A Emergency Plan Plan Annex(es) Standard Operating Procedures Resource List Maps, Displays Reporting Requirements Notification Procedures Mutual Aid Pacts Policy-Making Officials Coordinating Personnel Operations Staff Volunteer Organizations EOC/Command Center Communication Facility W arning Systems Utility Emergency Preparedness Industrial Emergency Preparedness Damage Assessment Techniques Other: APPENDIX A: J OB AIDS Exercise Design Page A.5 Job Aid 1: Exercise Needs Assessment (Continued) 9. Past Exercises If your organization has participated in exercises befor e, what did you learn from them, and what do the results indicate about future exercise needs? For example, consider the following questions: Who participated in the exercise, and who did not?

To what extent were the exercise objectives achieved?

What lessons were learned?

What problems were revealed, and what is needed to re solve them?

What improvements were made following past exercises, an d have they been tested?

APPENDIX A: J OB AIDS Page A.6 Exercise Design Job Aid 2: Comprehensive Exercise Program Planning Worksheet Timeframe: Present Problems: Long-Range Goal: Functional Objectives: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: APPENDIX A: J OB AIDS Exercise Design Page A.7 Job Aid 2: Comprehensive Exercise Program Planning Worksheet (Continued) Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: APPENDIX A: J OB AIDS Page A.8 Exercise Design Job Aid 2: Comprehensive Exercise Program Planning Worksheet (Continued) Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: Month: Exercise: For: Purpose: Rationale: APPENDIX A: J OB AIDS Exercise Design Page A.9 Job Aid 3: Self-Assessment: Resources and Costs 1. Plans How familiar are you with the emergency plans, policie s, and procedures of your organization or jurisdiction? Very familiar Only general familiarity Familiar with only a portion Need to thoroughly review plans, policies, and procedu res 2. Time a. How far in advance would your organization realist ically have to schedule to plan and design each of the following exercise activities effective ly?

Orientation ________________________________ Drill ________________________________ Tabletop exercise ________________________________ Functional exercise ________________________________ Full-scale exercise ________________________________ b. How much preparation time can reasonably be allocate d to developing an exercise?

Actual person days:

Elapsed time to exercise:

APPENDIX A: J OB AIDS Page A.10 Exercise Design Job Aid 3: Self-Assessment: Resources and Costs ( Continued) 3. Experience a. When was your organization’s last exercise? b. What is your previous experience with exercises? (Che ck all that apply.) Orientation: Presenter Participant Drill: Controller Participant Tabletop exercise: Facilitator Participant Functional exercise: Controller Simulator Player Evaluator Full-scale exercise: Controller Responder Evaluator Victim Took part in postexercise debrief.

Helped write an evaluation report.

c. What other exercise-related experience is available i n your organization?

4. Facilities What physical facilities do you use when conducting an eme rgency operation?

Will they be required for this exercise? Yes No Will they be available for this exercise? Yes No APPENDIX A: J OB AIDS Exercise Design Page A.11 Job Aid 3: Self-Assessment: Resources and Costs ( Continued) 5. Communications : What communication facilities and systems do you use in a r eal emergency? Will they be required for this exercise? Yes No Will they be available for this exercise? Yes No 6. Barriers : Are there any resource barriers that need to be overcom e to carry out this exercise? Yes No If so, what are the barriers and how can they be overco me?

APPENDIX A: J OB AIDS Page A.12 Exercise Design Job Aid 3: Self-Assessment: Resources and Costs ( Continued) 7. Costs a. What types of costs might be incurred for these exercise s in your organization? (Do not list exact figures just types of expenses, such as wages and salaries, transporta tion, etc.) For an orientation: For a drill: For a tabletop exercise: For a functional exercise: For a full-scale exercise: b. Are there ways that different organizations can red uce costs (e.g., by combining exercises, cost-sharing, resource-sharing) and still fulfill program requirements? Explain.

APPENDIX A: J OB AIDS Exercise Design Page A.13 Job Aid 4: Exercise Development Checklist Mission Scenario Needs Assessment Narrative Scope Major/Detailed Events Statement of Purpose Expected Actions Objectives Messages Personnel Logistics Design Team Safety Controller or Facilitator Scheduling Players Rooms/Location Simulators Equipment Evaluators Communications Management Phones Safety Radio Observers Computers Enhancements Information Maps Directives Charts Media Other:

Public Announcements Invitations Evaluation Community Support Methodology Management Support Locations Timeline Requirements Evaluation Forms Postexercise Debrief Training/Briefings After Action Documentation/ Train Simulators, Evaluators, Recommendations Controllers Evaluation Meeting Players’ Preexercise Briefing Evaluation Report Followup Ideas for Next Exercise APPENDIX A: J OB AIDS Page A.14 Exercise Design Job Aid 5: Activities Schedule Deadline for Completion Leader Activities Team Activities APPENDIX A: J OB AIDS Exercise Design Page A.15 Job Aid 6: Design Team Worksheet Name Agency Represented Contributions/Qualification s Leader Members APPENDIX A: J OB AIDS Page A.16 Exercise Design Job Aid 7: Scope Worksheet 1. Highest priority hazards (major and secondary): 2. Geographic areas/locations of greatest vulnerability to these hazards:

3. Agencies/departments/organizational units: List belo w the entities that have a significant role in emergency management/response. Then, enter che ck marks in any columns that apply. Agency/Organization Limited experience with major emergencies New plans, staff, or organizational structure not yet exercised Problems revealed in prior exercises 4. Types/levels of personnel that you want to have in the exercise:

Policy making (elected officials, chief operating office rs, department heads) Coordination (managers, EOC representatives, departme nt deputies) Operations (field personnel, headquarters staff level) Public representatives (media, PIOs, general public) Other: _____________________________________ APPENDIX A: J OB AIDS Exercise Design Page A.17 Job Aid 7: Scope Worksheet (Continued) 5. Types of operations/functions that you want participa nts to engage in (e.g., notification, evacuation): 6. Degree of stress, complexity, time pressure that the e xercise should have:

High Medium Low Stress Complexity Time pressure APPENDIX A: J OB AIDS Page A.18 Exercise Design Job Aid 7: Scope Worksheet (Continued) Exercise: Scope: Type of Emergency: Location: Functions: Organizations and Personnel: Exercise Type: APPENDIX A: J OB AIDS Exercise Design Page A.19 Job Aid 8: Statement of Purpose (Form 1) The purpose of the proposed emergency management exerci se is to improve the following emergency operations:

a. ________________________________________________ ________________________ b. ________________________________________________ ________________________ c. _________________________________________________ _______________________ d. ________________________________________________ ________________________ by involving the following agencies/organizations/departments :

a. ________________________________________________ ________________________ b. ________________________________________________ ________________________ c. _________________________________________________ _______________________ d. ________________________________________________ ________________________ e. ________________________________________________ ________________________ f. ________________________________________________ ________________________ g. ________________________________________________ ________________________ h. ________________________________________________ ________________________ in a ________________________ exercise simulating a ________________________ [type of exercise] [type of emergency ev ent] at ________________________________________ on ___________________.

[location] [date] APPENDIX A: J OB AIDS Page A.20 Exercise Design Job Aid 8: Statement of Purpose (Form 2) The purpose the proposed emergency management exercise i s to:

Coordinate the activities of city and county government , volunteer organizations, and private industry in their response to a major incident; To provide training to staff; To test and evaluate the ___________________________ ______________ Annexes; To enhance interagency coordination and cooperation by involving the following department or agency heads:

These entities will be tested on _____________________ in a simulated exercise involving a [date] _____________________________________ at __________ _______________________ [type of incident] [location] APPENDIX A: J OB AIDS Exercise Design Page A.21 Job Aid 9: Objectives List the exercise objectives below. Include the observable a ction, responsible party, conditions, and standards. Be sure each objective is SMART: Simple Measurable Achievable Realistic Task Oriented Obj. No. Objective Organization APPENDIX A: J OB AIDS Page A.22 Exercise Design Job Aid 9: Objectives (Continued) Obj. No. Objective Organization APPENDIX A: J OB AIDS Exercise Design Page A.23 Job Aid 9: Objectives (Continued) Obj. No. Objective Organization APPENDIX A: J OB AIDS Page A.24 Exercise Design Job Aid 10: Narrative Outline Event: How fast, strong, deep, dangerous: How you found out: Response made: Damage reported: Sequence of events: Current time: Advance warning: Location: Relevant weather conditions: Other factors that would influence emergency procedures: Predictions: APPENDIX A: J OB AIDS Exercise Design Page A.25 Job Aid 10: Narrative Script APPENDIX A: J OB AIDS Page A.26 Exercise Design Job Aid 11: Events and Actions Planning Sheet Events and Actions Planning Sheet Obj .

No. Major Events Detailed Events Expected Actions Organizations APPENDIX A: J OB AIDS Exercise Design Page A.27 Job Aid 12: Message Planning Sheet Message Planning Sheet Detailed Events Expected Actions OrganizationsMessage Outline APPENDIX A: J OB AIDS Page A.28 Exercise Design Job Aid 13: Emergency Exercise EMERGENCY EXERCISE TO: METHOD FROM:

NO:

TIME: CONTENT:

_________________________________________________ _________________ ACTION TAKEN: ___________________________________________________ __________ APPENDIX A: J OB AIDS Exercise Design Page A.29 Job Aid 14: Sample Master Scenario Events List Sample Master Scenario Events List Time Message/Event Expected Actions APPENDIX A: J OB AIDS Page A.30 Exercise Design Job Aid 15: Tabletop Exercise Checklist Design Needs assessment, scope, statement of purpose, and objectives developed. Narrative:

May be shorter Presented all at once or incrementally Events:

Limited number Presented as problem statements Expected actions:

May involve identification of appropriate responses, identification of gaps in procedures, reaching group consensus, developing ideas for change, etc. Messages:

Limited number (e.g., 10-15) Involve everyone Tied to objectives Facilitation Welcome participants Briefing:

Purpose and objectives Ground rules and procedures Narrative presentation (printed, verbal, TV, radio) Ice breaker questions directed at high-ranking officers Messages organized to involve all organizations Strategies to encourage the reticent Facilitate don’t dominate Model positive behaviors (eye contact, positive reinforce ment) Aim for in-depth problem solving Strategies for sustaining action Multiple event stages Varied pace Balanced pace Conflict resolution Low-key atmosphere APPENDIX A: J OB AIDS Exercise Design Page A.31 Job Aid 16: Functional Exercise Message Flow Plann ing Participating Agency/Organization (List organizations above the columns below. Check the tim es when messages are scheduled for delivery to each organization.) (Enter Msg.Times Below) Exercise Start APPENDIX A: J OB AIDS Page A.32 Exercise Design Job Aid 17: Functional Exercise Design Checklist: Special Considerations Facilities and Equipment Sufficient work space for simulators and players Simulation room (if needed) near player room Space for message center, control center, observers (as need ed) Clear work surfaces Communication equipment (telephones, switchboard) Parking Adequate ventilation and lighting Restrooms Displays and Materials Displays easily visible or accessible Maps (regional, state, local, area, downtown, operatio nal units) Major events log, bulletin board, status boards, simulat ion plotting board Easels, chart paper Message forms Pencils/Paper Name cards Beginning: “No-notice” or scheduled (according to objectives) Briefing (short): Objectives Process Time period portrayed Ground rules and procedures Narrative: Verbal, print, TV, computer, slides, or dramatization Time-skips if needed Messages: Large number (depends on scope) Pre-scripted Optional prescripted for adjusting flow Message Delivery: Written Phone Other (verbal, speaker phone/radio, hand signals) Simulators prepared for spontaneous message development Standardized forms for written messages Strategies for Adjusting Pace: Rescheduling Adding/Deleting messages Misdirecting messages Reassigning messages APPENDIX A: J OB AIDS Exercise Design Page A.33 Job Aid 18: Full-Scale Exercise Planning Checklist : Special Considerations Participants: Controller(s) sufficient to manage all event sites Simulators (mock victims) different age groups, body types, physical characteristics Players (most functions, all levels policy, coordination, operation, field) Evaluators Safety Officer Site Selection: Adequate space for number of victims, responders, and obse rvers Space for vehicles and equipment As realistic as possible without interfering with normal traffic or safety Credible scenario and location Scene Management: Logistics (who, what, where, how, when) Believable simulation of emergency Realistic victims Preparation of simulators to portray roles realistically Number of victims consistent with type of emergency, histo ry of past events Types of injuries consistent with type of emergency, histo ry of past events Victim load compatible with local capacity to handle Props and materials to simulate injuries, damage, other effects Personnel and Resources: Number of participants Number of volunteers for scene setup, victims, etc.

Types and numbers of equipment Communications equipment Fuel for vehicles and equipment Materials and supplies Expenses identified (wages, overtime, fuel, materials a nd supplies) Response Capability Sufficient personnel kept in reserve to handle routine non-exercise events Safety Safety addressed through development Each design team member responsible for safety in own di scipline Hazards identified and resolved Safety addressed in preexercise briefing, simulator and evaluator packets Each field location examined for safety issues Safety officer designated, given authority Legal Liability Legal questions of liability researched by local attorne y Emergency Call-Off Call-off procedure in place, including code word/phrase Call-off procedure tested Media Role of media addressed in planning, used as a resource t o gain favorable exposure Media and observers considered in logistical planning APPENDIX A: J OB AIDS Page A.34 Exercise Design Job Aid 19: Observation Checklist Objective Action/Decision to Look For Players to Observe Where Expected Time APPENDIX A: J OB AIDS Exercise Design Page A.35 Job Aid 20: Evaluator Checklist Evaluator: __________________________ Date: _______________ Location: ___________________________ Objective No.: Function Being Evaluated:

Objective: Performance Criterion [#] Points of Review: Please answer the following: Y = Yes, N = No, NA = Not Applicable, NO = Not Observed Y N NA NO 1. 2. 3. 4. Comments: APPENDIX A: J OB AIDS Page A.36 Exercise Design Job Aid 21: Narrative Summary Objective Number: ______________________ Criterion Number: ________________ Evaluator: ______________________________ Location: ______________________ Issue: A specific statement of the problem, plan, or proce dure that was observed.

___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ Discussion:

A discussion of the issue and its specific impact o n operational capability.

___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ Corrective Action Recommendation: Recommended course(s) of action to improve performa nce or resolve the issue to improve operational capability.

___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ ___________________________________________________ __________________________ APPENDIX A: J OB AIDS Exercise Design Page A.37 Job Aid 21: Narrative Summary (Continued) Office of Primary Responsibility: The department, agency, or organization responsible for implementation of corrective actions. Department, Agency, or Organization: ________________________________________________ Individual Responsible: ___________________________________________________ _________ Title: ________________ Date Assigned: ___ / ___ / ___ Suspense Date: ___ / ___ / ___ APPENDIX A: J OB AIDS Page A.38 Exercise Design Job Aid 22: Key Event Response Form Event No. Scheduled Date/Time Initially Input To Actual Date/Time Response Date/Time Position Responding Action Taken APPENDIX A: J OB AIDS Exercise Design Page A.39 Job Aid 23: Problem Log Date: __________________________ Exercise Assignment: ____________________________ Tel. No: ______________________ Time Message Library No. (if known) Problem Analysis (Leave Blank) APPENDIX A: J OB AIDS Page A.40 Exercise Design Job Aid 24: Exercise Debriefing Log Exercise Debriefing Log Recorder _________________________ Date ____________ _____________ Problem Summary Recommended Action Responsible Agency /Person Exercise _______________________ APPENDIX A: J OB AIDS Exercise Design Page A.41 Job Aid 25: Exercise Critique Form Please take a few minutes to fill out this form. Your opinions and suggestions will help us prepare better exercises in the future. 1. Please rate the overall exercise on the following sca le.

1 2 3 4 5 6 7 8 9 10 Very Poor Very Good 2. Compared to previous exercises, this one was:

1 2 3 4 5 6 7 8 9 10 Very Poor Very Good 3. Did the exercise effectively simulate the emergency e nvironment and emergency response activities? Yes _________ No __________ If no, briefly explain why: 4. Did the problems presented in the exercise adequatel y test readiness capability to implement the plan? ? Yes _________ No ___ _______ If no, briefly explain why: APPENDIX A: J OB AIDS Page A.42 Exercise Design Job Aid 25: Exercise Critique Form (Continued) 5. The following problems should be deleted or revised :

6. I suggest that you add the following problems for t he next exercise.

7. Please add any other comments or suggestions. Appendix B: Acronym List APPENDIX B ACRONYM LIST Exercise Design Page B.1 ARES Amateur Radio Emergency Services CB Citizen’s Band CEO Chief Elected Official CFR Crash/Fire Rescue CHEMTREC Chemical Transportation Emergency Center DOE Department of Energy EAS Emergency Alert System EMERS Emergency Management Exercise Reporting System EMI Emergency Management Institute EMS Emergency Medical Services EOC Emergency Operations Center EOP Emergency Operations Plan EPA Environmental Protection Agency FAA Federal Aviation Administration FEMA Federal Emergency Management Agency IEMC Integrated Emergency Management Course ICS Incident Command System JCAHO Joint Commission on Accreditation of Healthcare Orga nizations JIC Joint Information Center NAWAS National Warning System NOAA National Oceanic and Atmospheric Administration NRC Nuclear Regulatory Commission NWS National Weather Service OSHA Occupational Safety and Health Administration PDS Professional Development Series PIO Public Information Officer RACES Radio Amateur Civil Emergency Services SARA Superfund Amendment and Reauthorization Act of 1986 SOPs Standard Operating Procedures USGS United States Geological Survey US&R Urban Search and Rescue Appendix C: Exercise Tool Box APPENDIX C EXERCISE TOOL B OX Exercise Design Page C.1 For purposes of this Exercise Design course, the Exercise Tool Box contains several important items that you can use: Templates for: Exercise Plan Control Plan Evaluation Plan Player’s Handbook And Job Aids for all forms and charts used in the course ar e also contained in the Tool Box as “Exercise Design Job Aids.” You can access the Tool Box for the Exercise Design Indepen dent Study IS-139 at http://training.fema.gov/emiweb/IS/is139lst.asp . FINAL EXAM Exercise Design Page 1 1. Research has shown that:

a. Exercises are the best way to teach employees new skill s.

b. Extended lectures are an effective alternative to e xercises.

c. When possible, it is more effective to use real (rathe r than simulated) emergencies to test response procedures. d. People usually respond to an emergency in the way t hey have trained.

2. Which of the following is not a good reason to exercise?

a. To reveal planning weaknesses and gaps in resources. b. To clarify roles and responsibilities.

c. To reduce the need for organizational coordination and communication.

d. To satisfy regulatory requirements. 3. In a progressive exercise program, the exercises: a. Have a consistent format but are conducted with increa sing frequency.

b. Are organized to increase in complexity.

c. Are organized to decrease in complexity.

d. Are sponsored on a rotating basis by different organ izations.

4. Availability of resources: a. Is irrelevant to the exercise process.

b. Should not affect exercise design.

c. Should be considered in scheduling and planning an ex ercise.

d. Is seldom an issue for exercise design because exercises req uire few resources.

5. The major task accomplishments in the exercise process are :

a. Establishing the base, exercise development, exercise co nduct, critique and evaluation, and followup.

b. Conducting tabletop, functional, and full-scale exer cises.

c. Developing, conducting, and evaluating the exercise.

d. Conducting a needs assessment, writing objectives, and e valuating whether the objectives were achieved. FINAL EXAM Page 2 Exercise Design 6. The exercise design team leader should be: a. The Emergency Manager.

b. Someone who is familiar with the emergency plan.

c. A key operational member of a participating organi zation.

d. The chief official of the jurisdiction or organizati on.

7. Which statement is true of an orientation? a. It requires field sites and actual equipment.

b. It may be used to introduce or explain plans and po licies.

c. It involves a controller, simulators, and evaluators.

d. It is used to test a specific operation. 8. Which statement is true of a drill? a. It is best conducted in a conference room.

b. It involves a controller, simulators, and evaluators. c. It is used to test a specific operation.

d. It is aimed primarily at policy makers and decision m akers.

9. Which statement is true of a tabletop exercise? a. It involves a highly realistic simulation.

b. It involves a controller, simulators, and evaluators. c. It requires field sites and actual equipment deployme nt.

d. It is a facilitated analysis of an emergency situation .

10. Which statement is true of a functional exercise? a. It involves a controller, simulators, and evaluators. b. It is simple, informal, and stress-free.

c. It requires field sites and actual equipment deployme nt.

d. It may be used to introduce or explain plans and po licies.

11. Which statement is true of a full-scale exercise? a. It is aimed primarily at policy makers and decision m akers.

b. It requires field sites but actual equipment remains in the shed.

c. It is used to test a specific operation. d. It involves a highly realistic simulation. FINAL EXAM Exercise Design Page 3 12. The narrative:

a. Sets the stage for later action. b. Lists all of the events that will occur in the exercise .

c. Provides the master list of events and expected actions. d. Explains the objectives of the exercise. 13. The scope: a. Sets the mood for the exercise.

b. Establishes the limits of the exercise.

c. States the objectives of the exercise.

d. Lists the financial and human resources required to ru n the exercise.

14. An example of a good exercise objective is: a. A sufficient number of accurate messages will be transmi tted by the dispatch center to the communications center and primary response organizati ons in a timely and efficient manner.

b. At the time the evacuation notice is received, the E OC policy and coordination groups will examine the needs of schools and other special facili ties and organize notification according to SOPs.

c. Proper procedures will be followed to declare a disast er or ask for outside aid.

d. Interaction with other jurisdictions will be demonstr ated.

15. A ________________ is a useful tool to help the co ntroller keep the exercise on track and on schedule.

a. Message form b. Narrative c. Master scenario of events list d. Exercise directive 16. A convincing, unified scenario requires: a. 10 major events and approximately 100 detailed ev ents.

b. Messages written in a way will not allow unexpected responses to occur.

c. The players know the events in advance.

d. Careful scripting of events. FINAL EXAM Page 4 Exercise Design 17. In a tabletop exercise, an important part of the facilitator’s job is to:

a. Maintain an even pace and consistent approach. b. Adhere to a highly structured agenda.

c. Sustain action and keep everyone involved.

d. Make sure the entire set of problem statements is discu ssed.

18. Which of the following is not true of a simulator in a functional exercise: a. May deliver written messages.

b. Is often called upon to rate the performance of key players.

c. May deliver messages verbally.

d. Sometimes needs to make up a response to a player.

19. An exercise that tested only notification procedures in response to a terrorist bombing would be: a. A drill.

b. A tabletop exercise.

c. A functional exercise.

d. A full-scale exercise.

20. ______________ is a good use of a full-scale exercise :

a. Trying out a new and untested emergency plan b. Training personnel in negotiation c. Practicing group problem solving in a nonthreatening environment d. Testing emergency procedures and coordination of mult iple agencies or organizations 21. A full-scale exercise involves: a. All levels of personnel, including response personnel. b. Primarily the key policy and decision makers.

c. Policy, coordination, and operations personnel.

d. The staff from one department or unit.

22. A good evaluation can help the organization iden tify:

a. Training and staffing deficiencies.

b. Qualified members of the evaluation team.

c. Exercise objectives.

d. The scope of the exercise. FINAL EXAM Exercise Design Page 5 23. During the exercise, evaluators should: a. Ensure that players are aware when they are being observed.

b. Offer suggestions that can improve the exercise results. c. Focus on the positive.

d. Avoid attracting players’ attention. 24. One thing evaluators should focus on during an exer cise is:

a. Explaining evaluation methodology to the players being observed.

b. Having players explain their reasons for actions take n.

c. Noting what actions are taken in response to key even ts.

d. Finding as many positive points as negative ones.

25. The purpose of exercise enhancements is to: a. Increase credibility with the public and the media.

b. Make the exercise more fun so more personnel will pa rticipate.

c. Increase the realism so participants will respond as they would in a real event.

d. Make it easier on the controller and the simulators.