Audit of the Appeals Court Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Appeals Court.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Appeals Court for the period September 1, 2018 through October 31, 2019. 

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Does the Appeals Court comply with Section 4 of Chapter 262 of the General Laws and Rules 8a–8f of the Massachusetts Rules of Electronic Filing to waive court fees?	Yes
Does eFileMA reduce the amount of time it takes for the Appeals Court Clerk’s Office to process cases?	Yes
Does the Appeals Court ensure that fees are assessed in accordance with Section 4 of Chapter 262 of the General Laws and are verified through reconciliation and collection, in accordance with the Comptroller of the Commonwealth’s guidance on cash recognition and reconciliation?	No; see Finding 1

 

We gained an understanding of the internal control environment and the design of controls related to our audit objectives by reviewing applicable Appeals Court policies and procedures, as well as making inquiries and observations. In addition, we performed the following procedures to obtain sufficient, appropriate evidence to address our audit objectives.

Waived Court Fees

For waived court fees, we extracted a data set of all case-related transactions from eFileMA for our audit period and filtered the data to include only waived cases. Using Audit Command Language (ACL), we selected a statistical random sample of 60 out of 9,867 waived cases. For each case, we obtained evidence, including forms and notices, and verified the Appeals Court’s compliance with the Massachusetts Rules of Electronic Filing for motions to waive. Specifically, we determined whether a motion to waive fees¹ was filed, whether an affidavit of indigency² / inmate affidavit³ and supplement to such affidavits⁴ was completed, and whether a notice of docket entry⁵ was filed and completed.

Case Processing Time

The Appeals Court processes motions and briefs filed for cases through a review of documents to confirm that they comply with the Massachusetts Rules of Appellate Procedure. For case processing time, we conducted analytical procedures and a test.

For the analysis, we obtained a Docket Entries Report, which showed all 12,233 motions and 7,411 briefs filed during our audit period, from Forecourt Paragon Case Management System at the Appeals Court. We confirmed with the Appeals Court the average time it took to process motions and briefs filed both through eFileMA and on paper. We calculated the total number of hours it took to process all paper briefs and motions and the number of hours it took to process all electronic briefs and motions. Then we determined the number of hours it would have taken if all paper briefs and motions had been filed electronically. Finally, we calculated the total number of hours that would have been saved if all paper briefs and motions had been filed electronically. We determined that the Appeals Court would have reduced the processing time by 313 hours if all paper briefs and motions tested during the audit had been filed electronically.

For the test, we obtained a data set of all cases entered in Forecourt at the Appeals Court during our audit period. We filtered the data to count the cases initially entered through eFileMA and those initially entered on paper. We selected a nonstatistical random sample of 60 cases out of a total population of 2,088 (533 eFileMA cases and 1,555 paper cases) using ACL. The sample consisted of 15 eFileMA cases and 45 paper cases. We obtained evidence (in the form of screenshots) of the docket sheet, which includes specific case activity information from Forecourt. For each case, we verified the total numbers of motions and briefs filed through eFileMA and on paper during the audit period and determined whether filing all motions and briefs through eFileMA would have reduced the amount of time it took for the Appeals Court to process cases. We determined that if all briefs and motions tested during our audit had used eFileMA, the processing time would have been reduced by 22%.

Fee Collection

For fee collection, we compiled a list of working days when the Appeals Court made deposits during the audit period. Using ACL, we selected a nonstatistical random sample of 45 out of 293 days when deposits occurred. We obtained reconciliation evidence, including bank receipts, copies of checks, reports from Forecourt, and credit card reports. For each deposit day, we verified the Appeals Court’s compliance with Section 4 of Chapter 262 of the General Laws and the Appeals Court’s fee schedule by confirming that assessed fees for filing and miscellaneous charges were accurate. In addition, we verified the accuracy of the Appeals Court’s reconciliation of daily revenue collections by reviewing log sheets and the reconciliation evidence for fees assessed.

Data Reliability Assessment

We reviewed certain general information controls and access controls over Forecourt and eFileMA, including security training and personnel screening for the Appeals Court. In addition, for cases from Forecourt, we performed data integrity tests to identify any blank fields and duplicate records for our audit period. For data obtained from eFileMA, we performed data integrity tests to identify any blank fields and dates outside our audit period. We tested the accuracy of Forecourt and eFileMA data by tracing electronic data to source documentation, such as documentation of the assembly of the record⁶ for each case. We could not test the completeness of information from Forecourt and eFileMA because no hardcopy files were available. We determined that the data from both systems were sufficiently reliable for our audit purposes.

1.     Motion to waive occurs when a filer completes all forms to prove that the filer is indigent and submits the documentation to the court for review and decision on status of indigency.

2.     An affidavit of indigency is a form used to determine whether a person is eligible to have court fees and costs waived.

3.    An inmate affidavit is a form used for the same purpose as an affidavit of indigency, used when the filer is in jail and cannot pay the fees assessed.

4.     The supplement is a report from a jail showing an inmate’s accounts with that institution, proving that the inmate does not have the means to pay a fee.

5.     A notice of docket entry is sent to a filer after the Appeals Court Clerk’s Office has reviewed and approved filed documents.

6.    The assembly of the record is the compilation, organization, and retention of case information by the clerk of the lower court until the final disposition of the appeal. This case information is then transmitted to the Appeals Court by a court order.

Date published:	November 2, 2020