Audit of the Department of Fire Services Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Department of Fire Services.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor (OSA) has conducted a performance audit of certain activities of the Department of Fire Services (DFS) for the period July 1, 2016 through December 31, 2018.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer, the conclusion we reached regarding each objective, and where each objective is discussed in the audit findings.

Objective	Conclusion
1. Does DFS properly administer its fixed assets, including the following?
a. new equipment purchases valued at $1,000 or greater	No; see Finding 1a
b. disposals	No; see Finding 1b
c. annual physical inventory	No; see Finding 1c
2. Does DFS review supporting documentation to confirm that earmarked funds allocated under its annual state budget appropriation are spent as required?	No; see Finding 2

 

We gained an understanding of the internal control environment related to our audit objectives by reviewing applicable DFS policies and procedures, as well as making inquiries and observations. In addition, we performed the following procedures to obtain sufficient, appropriate evidence to address our audit objectives.

Sampling Methodology

We used a nonstatistical sampling method for our tests of new equipment purchases, disposal of assets, and community earmarks. As a result, we cannot project the results of our testing to the entire population. In addition, we used a statistical sampling method for our test of DFS programs, with our population stratified (separated) into the following populations: Critical Incident Stress Management (CISM) Program, Student Awareness of Fire Education (SAFE) Program, and On-Site Academy earmarks. Because we stratified the population into three groups to test the programs individually, we decided not to project the results of our testing to the entire population.

Inventory Purchases

For new equipment purchases valued at or above $1,000, we obtained a list of all DFS cash expenditures from the Massachusetts Management Accounting and Reporting System (MMARS) for our audit period. We filtered the data to include only equipment-related expenditures of $1,000 or more. Using a nonstatistical sampling method and Audit Command Language (ACL) software, we selected a random sample of 40 out of 254 equipment purchases of $1,000 or more. We obtained evidence, including an inventory card/form and invoice/s, and physically verified the existence of each asset and tag. For each expenditure, we verified DFS’s compliance with its “Fixed Asset Policy and Procedures” for assets over $1,000—specifically, whether each purchase was required to be on the inventory, whether the inventory card/form was completed for each asset, whether each asset appeared on the inventory list, and whether each asset had an asset tag or an engraved asset number.

After conducting our initial testing of 40 new equipment purchases of $1,000 or more, we determined that 29 expenditures sampled were not new equipment purchases, but rather betterments¹ or expense corrections² to accounting records. We filtered out any betterments or expense corrections and created a new subpopulation of 54 from the original population of 254. Using a nonstatistical sampling method, we judgmentally sampled an additional 20 out of the subpopulation of 54, for which we collected the same evidence to conduct the test.

Inventory Disposals

For assets that had been disposed of, we obtained a list of all the assets that were in DFS’s possession during our audit period from the inventory tracking database IntelliTrack. Using a nonstatistical sampling method and ACL, we selected a random sample of 40 assets out of a population of 267 that had been disposed of. To conduct testing, we requested evidence from DFS and the Operational Services Division (OSD), including OSD Form 25s³ submitted by DFS to OSD; a list of surplus assets sent to the designated OSD employee (the State Surplus Property Officer); and emails between DFS and OSD regarding confirmation of asset disposal. We reviewed the evidence provided to confirm compliance with DFS’s “Fixed Asset Policy and Procedures”—specifically, whether each asset had an OSD Form 25 submitted by DFS, whether each asset was on the list of surplus assets sent to the State Surplus Property Officer, whether OSD approved disposal of each asset, and whether the date that the asset was listed as disposed of in IntelliTrack was on or after the date of OSD’s approval. 

Annual Physical Inventory

We reviewed the list of current assets to determine whether each asset had been verified⁴ during physical inventory in fiscal year 2018. We reviewed the list to determine whether any lost or stolen assets on the inventory report were reported to OSA as required by Chapter 647 of the Acts of 1989. Additionally, we obtained and reviewed all quarterly inventory reports from the audit period, and emails from division inventory liaisons to the director of capital asset management regarding changes to division inventory, to confirm compliance with DFS’s “Fixed Asset Policy and Procedures.”

Earmarked Funds

We tested earmarked funds allocated under DFS’s annual appropriation to ensure that appropriate supporting documentation, such as statements of work, invoices, receipts, or budget forms, existed for expenditures made with earmarked funds. For our tests, we created populations of earmark-related expenditures, including funds allocated for communities, the CISM and SAFE Programs, and On-Site Academy. We created one test for earmarked funds disbursed to communities and one test for earmarked funds allocated to DFS-specific programs, including the CISM and SAFE Programs and On-Site Academy.

Community Earmarks

We obtained a list of all DFS cash expenditures from MMARS during our audit period. For the community earmark test, we filtered the data to include only expenses related to disbursements to communities out of the earmarked funds. We selected a nonstatistical random sample, using ACL, of 20 from the population of 64. DFS provided us with a Commonwealth of Massachusetts Standard Contract, purchase requisition, and backup documentation, including invoices, receipts, spreadsheets documenting purchases, and letters from municipal officials attesting to purchases. We reviewed the documentation provided to confirm the following: that a Commonwealth of Massachusetts Standard Contract existed and was signed with the proper approval signatures, that a purchase requisition was completed, and that backup documentation was completed.

DFS Programs: CISM and SAFE Programs and On-Site Academy

Using the same list of all DFS cash expenditures from our audit period, we filtered the data to reflect only expenses related to disbursement of earmarked funds for the CISM Program, SAFE Program, and On-Site Academy. We selected a statistical random sample of 60 using ACL, from a total population of 1,069, with a confidence level of 95% and a tolerable error rate of 5%. We stratified the population to capture expenditures for the CISM Program, the SAFE Program, and On-Site Academy individually. Using ACL, we randomly sampled 2 expenditures from a population of 46 for the CISM Program, 56 expenditures from a population of 993 for the SAFE Program, and 2 expenditures from a population of 30 for On-Site Academy.

DFS provided us with a Commonwealth of Massachusetts Standard Contract, a purchase requisition, and backup documentation so that we could perform testing of earmarked funds for the CISM Program, the SAFE Program, and On-Site Academy. We reviewed the documentation to confirm that a Commonwealth of Massachusetts Standard Contract with the proper approval signatures existed, that a purchase requisition was completed, and that backup documentation was completed. Backup documentation for On-Site Academy included invoices and reimbursement requests. Backup documentation for the CISM Program included completed reimbursement request forms from CISM Program teams, receipts and invoices for CISM Program team expenditures, invoices for training held by Merrimack Valley Training Center (MVTC), the list of approved training for MVTC, speakers’ resumes and potential speaker topics, completed Engagement Confirmation Forms, completed Employment Status Forms, completed Response Evaluations, and speakers’ invoices. Backup documentation for the SAFE Program included completed grant applications; completed DFS checklists; award letters sent to district legislators, the Governor, and recipients of SAFE Program earmarked funds; extension forms; completed year-end reports; and invoice/reimbursement requests.

Data Reliability Assessment

In 2018, OSA conducted a data reliability assessment of MMARS for the period April 1, 2017 through March 31, 2018. This assessment focused on reviewing selected system controls, including access, security awareness, auditing and accountability, configuration management, identification, authentication, and personnel security.

In our current audit, we reviewed certain general information controls, including automated control testing, access controls, security training, and personnel screening, over IntelliTrack and MMARS to determine the reliability of the data therein. In addition, for the list of assets from IntelliTrack, we performed data integrity tests to identify any blank fields and duplicate records for our audit period. We tested for completeness and appropriateness of the IntelliTrack list and MMARS expense data. We determined that the data from IntelliTrack and MMARS were sufficiently reliable for our audit purposes.

1.     Betterments occur when assets are improved by a department. For example, DFS has a fleet of trucks, which sometimes need improvements or new equipment that increase their value and useful life.

2.     Expense corrections occur when an accounting department needs to put an expenditure in a different category from the one that was originally entered.

3.     OSD Form 25 is officially titled “Declaration of Surplus Property.” State agencies use it to inform OSD that they intend to dispose of an asset that is no longer of use.

4.     State agencies verify that assets are in their possession by physically locating them and confirming that information in their inventory lists is accurate.

Date published:	May 12, 2020