Overview
During the audit period, attendance records for adult education (AE) courses in the Literacy, Adult and Community Education System (LACES) were not accurate and up to date. In our comparison of attendance records obtained from providers (i.e., source documents) to LACES attendance data, we found that records for 59 of 60 courses had been inaccurately entered in LACES, and source documents were not available for 1 course because the provider entered attendance directly in LACES. The attendance records for the 59 courses consisted of 112,125 accumulated total days, of which 24,263 (21.6%) were improperly entered in LACES. The improper entries included instances of students who were recorded as absent in source documents but present in LACES, students who were recorded as present in source documents but absent in LACES, and students who were recorded as present in LACES but whose attendance was not indicated in source documents.
Program funding is awarded based on estimated student attendance, so without accurate and up-to-date attendance data, the Department of Elementary and Secondary Education (DESE) cannot ensure fair and equitable distribution of funding to AE providers.
Authoritative Guidance
DESE’s “FY19 Massachusetts Policies for Effective Adult Education” state,
Data entry, including student attendance, assessment, [program development] participation, and other information as the need arises must be entered accurately. . . . Data entry must be consistent with program records in the grantee’s files.
Reasons for Issue
DESE did not have policies and procedures, including monitoring, to ensure that the AE course attendance information that providers entered in LACES was accurate and up to date.
Recommendation
DESE should implement policies and procedures to define a process for entering attendance records in LACES and implement controls to monitor the accuracy of AE course attendance data entered by providers.
Auditee’s Response
[DESE’s Adult and Community Learning Services Unit, or ACLS Unit] concurs with this finding. The audit year, FY19, was also the year [the] Massachusetts [Adult Basic Education Program] transitioned to a new [management information system, or MIS]. We believe that the timeline of the transition contributed to the data entry error. The legacy system, [the System for Managing Accountability and Results Through Technology, or SMARTT], was not available for attendance as of July 1, 2018. The new MIS, LACES, was launched three months later. This created a significant backlog as providers rushed to enter data and not fall further behind while administering the day-to-day services.
The learning curve was significantly steeper than ACLS and providers anticipated. Many providers sent only one staff person to the initial training. While the data fields in LACES are similar to those in SMARTT, the screens and workflow are very different.
The authorizing legislation, [the Workforce Innovation and Opportunity Act], requires grantees to provide much more detail. This requirement contributed to our decision to procure a new MIS. Rather than enter a single number of monthly hours for each student as had been the requirement for two decades, providers enter attendance for each class meeting. That could mean up to 20 times the number of entries for each student. This was a significant change and challenge for the adult education system and we anticipated errors.
ACLS has taken several concrete steps to improve data quality. First, all providers submit a data quality checklist as part of annual refunding. The FY20 version will be electronic and easier to monitor. Second, ACLS requires programs to submit desk review workbooks every other month and program specialists then analyze and review. This process ensures that providers and state staff examine the data closely and regularly. This process was included [in] the FY20 [Program Specialist Employee Performance Review System]. Finally, ACLS developed a systematic process to review statewide data to identify attendance irregularities.
Auditor’s Reply
Through its ACLS Unit, DESE is taking measures to improve the quality of the attendance data reported for AE courses. DESE should ensure that any controls it establishes over this activity are properly documented and followed. It should also consider implementing policies and procedures that define a formal process to ensure that AE providers enter attendance information in LACES in a complete and consistent manner.
| Date published: | June 2, 2020 |
|---|