Overview
In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Center for Health Information and Analysis (CHIA) for the period July 1, 2021 through June 30, 2023.
We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.
Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.
| Objective | Conclusion |
|---|---|
| No; see Findings 3a, 3b, and 4 |
| No; see Findings 1 and 2 |
To accomplish our audit objectives, we gained an understanding of the aspects of the internal control environment relevant to our objectives by reviewing applicable policies and procedures, CHIA’s internal control plan, and interviewing CHIA officials. In addition, to obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures described below.
To determine whether CHIA maintained information that was updated at least annually on the CompareCare website, we interviewed CHIA officials who were responsible for managing the CompareCare website. We then reviewed email notifications and associated metadata to determine whether the data used to provide cost estimates on CompareCare was updated at least annually.
We also reviewed the CompareCare website and interviewed management to determine whether the CompareCare website included the following, as required by Section 20(b) of Chapter 12C of the General Laws:
- comparative price and cost information for the most common referred and/or prescribed healthcare services;
- comparative health quality information for the most common referred and/or prescribed healthcare services;
- general information regarding healthcare services, such as comparative prices, costs, and quality;
- comparative quality information that is not service-specific, such as information related to patient safety or satisfaction with their experience;
- data concerning infections that patients may have developed while receiving healthcare for another condition;
- data concerning serious reportable events that resulted in adverse patient outcomes, such as (1) performing a procedure on the wrong part of the body, (2) the unauthorized discharge of a patient, or (3) injury or death related to a medication error;
- definitions of common health insurance and medical terms;
- a list of healthcare provider types, such as primary care physicians, nurse practitioners, and physician assistants and which services those provider types are allowed to perform in Massachusetts;
- factors that healthcare consumers may consider when they chose their healthcare coverage or provider, such as provider network coverage, insurance premiums, and service coverage;
- patient decision aids, which provided information regarding treatment options for various healthcare conditions;
- a list of healthcare provider services that were accessible for people with disabilities; and
- explanations of standard healthcare quality measures.
We obtained a site map of the CompareCare website and selected a random, nonstatistical sample of 40 webpages out of 365 webpages. Because this was a nonstatistical sample, we did not project any errors to the population of 365 webpages. To determine whether the webpages in our sample complied with the Web Content Accessibility Guidelines 2.1 standards for the Americans with Disabilities Act, we examined the following information:
- user accessibility, which relates to issues such as whether a webpage can be viewed in both portrait and landscape mode or whether a webpage is still legible when zoomed in;
- keyboard accessibility, which relates to issues such as whether all elements of a webpage can be navigated with a keyboard;
- navigation accessibility, which relates to issues such as whether a webpage contains a search tool and whether hyperlinks navigate to appropriate content;
- language, which relates to issues such as whether words are displayed in the appropriate language according to a user’s browser settings; and
- color accessibility, which relates to issues such as whether hyperlinks are displayed in a color that is easily distinguishable.
We determined that CHIA’s CompareCare website was not publicized, did not include certain required information and was not compliant with the Americans with Disabilities Act and the Web Content Accessibility Guidelines 2.1. See Findings 3a, 3b, and 4 for more information.
To determine whether CHIA monitored the financial conditions of acute care hospitals, we reviewed CHIA’s Massachusetts Acute Hospital and Health System Financial Performance reports, hospital profiles, and health system profiles for fiscal years 2021 and 20221 and determined whether each document included information required by Section 8(c) of Chapter 12C of the General Laws. We specifically determined whether these reports included the following:
- gross and net patient service revenue;
- sources of hospital revenue;
- private sector charges;
- trends in inpatient and outpatient case mix, payer mix, hospital volume, and length of stay;
- total payroll as a percentage of operating expenses;
- salary and benefits information for the top 10 highest compensated employees; and
- an identification and examination of hospitals in financial distress.
We determined that CHIA did not appropriately monitor the financial conditions of acute care hospitals and health systems. See Finding 1 for more information.
To determine whether CHIA took appropriate actions when acute care hospitals did not comply with reporting requirements, we obtained a list of all 61 acute care hospitals that are required to file financial reports with CHIA. We then reviewed emails and logbooks to determine which acute care hospitals filed their financial reports with CHIA by the required deadline. In cases when acute care hospitals did not file their required financial reports by CHIA’s deadline, we inquired with management to determine whether CHIA sent legal written notice to these acute care hospitals regarding their noncompliance with the financial reporting requirements. We also inquired with management regarding whether CHIA issued fines to acute care hospitals that were not in compliance with CHIA’s financial reporting requirements.
We determined that CHIA did not assess potential fines of $1,613,000 for acute care hospitals and health systems that did not file financial reports by required deadlines. See Finding 2 for more information.
Data Reliability Assessment
To determine the reliability of the Excel list of all 61 acute care hospitals that are required to submit financial data to CHIA, we performed validity and integrity tests on the data, including testing for blank fields, duplicate records, and duplicated identifiers.
To determine the reliability of the Excel list of 365 webpages, we performed validity and integrity tests on the data, including testing for blank fields and duplicate records. We selected a random sample of 20 URLs from the list of 365 webpages and traced the webpage name on the list to the URL on the CompareCare website.
Based on the results of the data reliability assessment procedures described above, we determined that the information we obtained was sufficiently reliable for the purposes of our audit.
| Date published: | January 3, 2025 |
|---|