Last month's column dealt with the initial planning stages for the three main types of exercises. This month's column will continue with how to conduct an exercise and then properly evaluating it.
Before you can effectively exercise your plan, you need to have an exercise and evaluation team in place. First you need an evaluation team leader. Ideally, this person should have been part of the exercise team from the start. Facilitators and controllers are needed to keep the exercise moving and on course. A safety officer is required to prevent or stop any dangerous activity. The safety officer has the complete authority to immediately stop the exercise if he observes or becomes aware of any action or condition that could jeopardize the welfare of anyone involved.
The exercise schedule is planned using a Master Scenario Events List (MESL). The MESL lists the events and actions to take place during the exercise. This is a timeline of what is to happen and the "injects" if the exercise needs to be guided along. The MESL also tracks what is expected to happen during the exercise. The MESL assures that the exercise objectives are tested during the exercise and that the results are documented for later review.
The exercise evaluation team leader needs to be a member of the exercise design team to ensure the objectives are valid and can be verified during whatever type of exercise is conducted. The team leader has to determine the means in which the exercise will be reviewed, who will be the members of the evaluation team, the document to be developed and used and the planning of the post-exercise meetings.
As written about last month, exercises should be conducted in accordance with the Homeland Security Exercise and Evaluation (HSEEP) standards. The reason that exercises have to be developed, conducted and evaluated per HSEEP is to see that the exercise is consistent with the National goals of preventing and responding to a terrorist incident.
When developing an exercise on the local level, check with your county or state Office of Emergency Management for guidance and also for possible prior approval in order to receive credit for the exercise. Exercises also need to be coordinated with the county and State OEM to make sure the exercise is relevant and that it is not in conflict with any other exercises that may be conducted at the same time in a nearby location.
Participants in the exercise are referred to as players. Players are the actual members of emergency organizations, representing their agency and performing as they would in a real event. Actors and simulators are just that. They are used in full-scale exercises to take on the role of victims. The actors are often Boy and Girl Scouts, playing the role of victims as well as volunteer adults from non-participating departments.
During the course of table top, functional and full-scale exercises, controllers and facilitators keep the exercise on the MESL by making sure events go as planned. If the exercise bogs down or goes off the MESL, they help move the exercise along and help put the exercise back on the right track.
The exercise evaluation guide, EEG, is developed to be used in reviewing the results of the exercise. This document tracks the activities performed during the exercise to see that they are consistent with the goals and objectives that are expected to be validated.
Immediately after the completion of any exercise is a "hot wash" debriefing. The purpose of the hot wash is to discuss any potential life-safety conditions or lessons learned that should be acted upon immediately. The hot wash is led by a facilitator that has the key players of the exercise give their quick recap of what they observed during the exercise, and to document the strengths and weakness of what has just occurred. All of the comments and observations are documented so they may be included in the after action report (AAR). The hot wash also allows the players to state their observations while it is fresh in their minds. At the completion of the hot wash, the participants can also complete the evaluation forms. The purpose of the hot wash is not to criticize any agency or individual, but to learn from it. Remember, this is a no-fault exercise. Besides the overall hot wash, each participating functional group should have their own functional groups' specific debriefing, addressing issues unique to their own discipline. These functional specific debriefings are conducted by the controller who oversaw that group during the exercise.
Once all the feedback, evaluation forms, and EEGs are received, the AAR can be completed. The AAR addresses if the objectives of the exercise were met, not just how well everyone performed. The AAR focuses on how the systems or agencies performed, more so than on any individual.
The AAR should contain the evaluation of the different agencies' response to the exercise scenario and the adherence to existing policies and procedures. The AAR also needs to address any problems, whether they were the results of the exercise plan design or the performance of the participants. Good or bad, all actions need to be recorded in the AAR so the exercise can be properly judged.
The final component of the AAR is the recommendations for improvement. This section takes the lessons learned from the exercise and incorporates them into future planning and plan maintenance to learn from the exercise. Once the AAR is completed, it is distributed to all participating agency of the exercise and the governing officials of the jurisdiction where the exercise took place.
Once the improvements and the lessons learned from the exercise are put into use, you can start planning for your next exercise.
