• This page, Audit of the Department of Veterans’ Services Objectives, Scope, and Methodology, is   offered by
  • Office of the State Auditor

Audit of the Department of Veterans’ Services Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Department of Veterans’ Services

Table of Contents

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Department of Veterans’ Services (DVS) for the period July 1, 2019 through June 30, 2021.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer, the conclusion we reached regarding each objective, and where each objective is discussed in the audit findings.

Objective

Conclusion

  1. Did DVS have a process in place to ensure that its Women Veterans’ Network (WVN) achieved its intended purpose of identifying women veterans and informing them about their potential eligibility for benefits?

No; see Finding 1

  1. Did DVS have an advisory committee on women veterans in accordance with Section 2 of Chapter 115 of the General Laws?

No; see Finding 2

 

To achieve our audit objectives, we gained an understanding of DVS’s internal control environment related to the objectives by reviewing applicable policies and procedures, as well as conducting inquiries with DVS staff members and management. To obtain sufficient, appropriate audit evidence to address the objectives, we performed the procedures described below.

To determine whether DVS had a process in place to ensure that WVN achieved its intended purpose, we performed the following procedures.

  • We interviewed the WVN director and the DVS chief of staff to discuss the process of WVN’s outreach to women veterans. We also asked these officials whether DVS management had developed any performance measures to evaluate WVN’s effectiveness and, if WVN did not achieve desired results, whether DVS management identified what needed to be done to improve WVN’s performance.
  • We requested from DVS a list (current as of the time of our fieldwork4) of WVN members. DVS gave us a list as of February 2022 that consisted of 1,545 records. Approximately 90% of the records were email addresses only. From this list, we identified 878 records that either contained a name (152 items), or did not contain a name but had information in the email address that allowed for the extraction of a name (726 items). We selected and tested a judgmental, nonstatistical sample of 60 members from these 878 records. We emailed a questionnaire to each of the 60 members and evaluated all 14 responses that we received to assess each member’s knowledge of WVN, their interaction with WVN, their awareness of the types of assistance WVN provides, and their perspective on WVN’s overall effectiveness.
  • Because there were no documented policies and procedures related to WVN, we used the job description for the WVN director position to identify activities that WVN is expected to complete. From this job description, we compiled a list of 16 outreach-related duties and requested documentation to substantiate that the director had performed them during our audit period.

To determine whether DVS had an advisory committee on women veterans in accordance with Section 2 of Chapter 115 of the General Laws, we performed the following procedures.

  • We interviewed DVS’s chief of staff, general counsel, and other key staff members responsible for monitoring and guiding the advisory committee’s activities to discuss the committee’s operating procedures and how the committee is managed.
  • We interviewed the Secretary of DVS to obtain knowledge and insight on committee history, committee appointments, and the support and guidance provided to the committee by DVS management during the audit period.
  • We requested a list of committee members from the audit period, as well as any details related to nominations, appointments, and membership terms. We reviewed a list of active advisory committee members as of June 30, 2021. DVS could not provide details related to committee members’ nominations, appointments, or membership terms.
  • We interviewed two people who were members of the committee as of the end of the audit period to determine each of these members’ level of involvement with the committee, how the committee operates, how often it meets, and what its overall activities are.

Where nonstatistical sampling was used, we could not project the results of our testing to the overall populations.

Data Reliability Assessment

The list of WVN members that DVS provided consisted of 1,545 email addresses. This list was the only source of information that DVS could provide regarding WVN members. We interviewed the WVN director, who is responsible for maintaining the list, and tested the list for duplicate records. In addition, we scanned the list for records that did not contain names and did not have information in the email address that allowed us to extract a name. We excluded all records without associated names. Because the list was missing key identifying information, such as members’ names, addresses, and telephone numbers, we could not determine whether some of the records were duplicates. Therefore, we could not determine the exact number of WVN members. However, because we only used this list to create the recipient list for our questionnaire, we determined that the data obtained for our audit period were sufficiently reliable.

4.    Because DVS does not maintain historical versions of WVN’s membership list, DVS could not provide a list of WVN members as of the end of our audit period. Instead, DVS provided a then-current list of WVN members.

Date published: September 27, 2022

Help Us Improve Mass.gov  with your feedback

Please do not include personal or contact information.
Feedback