To see how we’re doing in Massachusetts, read the annual reports of the Student Awareness of Fire Education (S.A.F.E.) Program. We summarize the results from the local fire departments to generate a picture of what our students are learning about fire and life safety through the S.A.F.E. Program. The results give us exactly the feedback we need: proven successes and opportunities for improvement and growth.
Evaluation is the last step in your program planning process and perhaps the most important. It helps you measure the impact of your program, reveals whether or not you have met your goals, and identifies a change in knowledge or behavior.
Evaluation is feedback. How are we doing? Is what we are doing working? Are the students learning what we think we are teaching? Reviewing your evaluation will let you know if your short-term, intermediate or long-term goals have been met and allow you to determine what changes are needed to improve your program or lesson plans.
Evaluation means collecting data, compiling it (simple statistics) and figuring out what it is telling you.
Rubrics are a great way to evaluate creative classroom projects such as home escape plans, essays, posters, poems, or videos. Students and teachers use rubrics in the classroom all the time. It makes expectations of success clear to everyone. Here are some examples:
- YouTube™ Burn Awareness Video Contest Rubric
- Home Escape Plan Rubric (courtesy of Topsfield Fire Capt. Brown)
- Helpful Rubrics website
- Blank Rubric
Pre- and Post-Tests
Pre-tests and post-tests are a common way of measuring changes in knowledge. These are usually in the form of multiple choice questions that are given before the program and then again at the end. The results of the pre-test provide a baseline of knowledge and points out the topics or issues that need to be covered in the lessons. Post-test results should show an increase in knowledge over the pre-test scores. An important aspect of pre and post-tests is how you word them. You might want to include action oriented questions rather than only knowledge based questions. Examples of the different types of questions are: “Has someone in your family tested the smoke alarms in the last month?” (action) or “how often should smoke alarms be tested?” (knowledge). These tests should not contain trick questions. The goal is to measure knowledge and behavior. If you use the same test for pre and post testing, you may rearrange the order of the questions.
Checklists are another way of scoring projects such as home escape plans. Again, share your checklist with the students so they know what is expected of them. What are the five things you want them to include in the home escape plan drawing or essay?
Home Escape Plan Checklist
Just like firefighters in training, you must track whether each individual student performs the skill correctly. Did each student perform “stop, drop and roll” correctly? Need coaching? Or not perform correctly (or at all)? Make a list of all the students and have someone keep track during the demonstration.
The Final Step: Collect and Review Your Data
The final step in evaluation is to organize your data and to review it. The DFS evaluation tools come with a spreadsheet for entering your data that automatically tallies the scores and has analytical tools that help you figure out which concepts the students understand well and which ones not so well. If you are using a rubric, a checklist, a pre and post test, or tracking demonstrations, you will need to compile your data to determine if the students are learning what you think you are teaching. How many of the students had all five items on your checklist? What percentage of the students correctly performed “stop, drop and roll”? With this information you will have evidence that fire education works. The data can also help you refine your lesson plans by identifying topics or areas where students did not provide successful responses.