Audit of the Massachusetts Life Sciences Center Objectives, Scope, and Methodology

An overview of the purpose and process of auditing the Massachusetts Life Sciences Center.

Overview

In accordance with Section 12 of Chapter 11 of the Massachusetts General Laws, the Office of the State Auditor has conducted a performance audit of certain activities of the Massachusetts Life Sciences Center (MLSC) for the period July 1, 2021 through December 31, 2023.

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Below is a list of our audit objectives, indicating each question we intended our audit to answer; the conclusion we reached regarding each objective; and, if applicable, where each objective is discussed in the audit findings.

Objective	Conclusion
Did MLSC administer the High School Apprenticeship Challenge (HSAC) program in compliance with Section 6(c)(5) of Chapter 23I of the General Laws?	No; see Finding 1
Did MLSC prepare and submit the annual report in compliance with Section 6(c)(9) of Chapter 23I of the General Laws?	No; see Finding 2
Did MLSC’s website adhere to Web Content Accessibility Guidelines (WCAG) 2.0 for user accessibility, keyboard accessibility, navigation accessibility, language, error identification, and color accessibility?	No; see Finding 3

To accomplish our audit objectives, we gained an understanding of MLSC’s internal control environment relevant to our objectives by interviewing MLSC staff members and management. In addition, to obtain sufficient, appropriate evidence to address our audit objectives, we performed the procedures described below.

HSAC Program

To determine whether MLSC properly administered the HSAC program as required by Section 6(c)(5) of Chapter 23I of the General Laws, we took the following actions:

• We interviewed knowledgeable staff members regarding the administration of the HSAC program.
• We reviewed the documentation MLSC provided that explains how it provides “additional considerations”²¹ to students within marginalized populations who attend schools where at least 80% of the population is eligible for free or reduced lunch. We also interviewed knowledge staff members about these “additional considerations.”

Additionally, to ensure that all interns were eligible for the program, we performed the procedures below on a random, nonstatistical sample²² of 40 internship applicants out of a population of 273 internship applicants.

• Applicant age: We reviewed the ages of interns, found within the list of interns provided by MLSC, to determine whether interns were, at the time the internship started, at least 16 years old.
• Applicant school: We reviewed the interns’ schools, found within the list of interns provided by MLSC, to determine whether interns had met one of the following descriptions:
  • The intern was enrolled in, or had recently graduated from,²³ a school that meets at least one of the following descriptions:
    • A vocational technical high school as determined by Chapter 74 of the General Laws or a comprehensive public high school with a life science Chapter 74 program.
    • A public high school (including charter schools) located in one of the following gateway cities²⁴ as determined by Section 3A of Chapter 23A of the General Laws: Attleboro, Barnstable, Brockton, Chelsea, Chicopee, Everett, Fall River, Fitchburg, Haverhill, Holyoke, Lawrence, Leominster, Lowell, Lynn, Malden, Methuen, New Bedford, Peabody, Pittsfield, Quincy, Revere, Salem, Springfield, Taunton, Westfield, or Worcester.
    • A public high school (including charter schools) with a student population who is at least 25% eligible for free or reduced lunch as defined by the Massachusetts Department of Elementary and Secondary Education.
• The intern was enrolled in the Metropolitan Council for Educational Opportunity program.

See Finding 1 for issues we identified with the HSAC program.

Annual Reports

To determine whether MLSC’s 2021, 2022, and 2023 annual reports included the criteria set forth in Section 6(c)(9) of Chapter 23I of the General Laws, we determined whether the following elements were present in the annual reports issued by MLSC during the audit period:

1. the submission date of the annual report (which was not later than December 1 of the respective year);
2. the total funds expended on high school internships;
3. the total funds expended on college internships;
4. the number of students participating in the internship programs from each high school, school district, college, and university in the Commonwealth;
5. the percent of high school internships awarded to students of color attending schools where at least 80% of the student population is eligible for free or reduced lunch;
6. the percent of college internships awarded to students of color enrolled full-time or part-time at a community college;
7. the racial and ethnic composition of the high school and college internship programs; and
8. an analysis of the impact of the college internship program on the ability of its participants to enter the full-time job market in life science industries after graduation.

See Finding 2 for issues we identified with the MLSC’s annual reports.

Web Accessibility

To determine whether MLSC’s website meets the requirements of WCAG 2.0 for user accessibility, keyboard accessibility, navigation accessibility, language, error identification, and color accessibility, we performed accessibility testing procedures on a random, statistical sample²⁵ of 60 MLSC webpages out of a population of 1,498 MLSC webpages, using a 95% confidence level,²⁶ a 0% expected error rate,²⁷ and a 5% tolerable error rate.²⁸ 

User Accessibility

• We determined whether content zoomed in to 200% was undamaged and without any issues.

Keyboard Accessibility

• We determined whether all elements²⁹ of each webpage could be navigated using only a keyboard.
• We determined whether any elements on each webpage prevented a user from moving to a different element when using only a keyboard to navigate the webpage.
• We determined whether the first focusable control³⁰ was a hyperlink that redirects to the main content of the website. The first focusable control is known as either a bypass block or a skip link.

Navigation Accessibility

• We determined whether each website contained a title that was relevant to website content.
• We determined whether there was a search function present to help users locate content.
• We determined whether related hyperlinks allowed navigation to the intended webpage.
• We determined whether headings within websites related to the content of the header’s section.

Language

• We determined whether video content found within the website had all important sounds and dialogue captioned.
• We determined whether the language of each webpage was tagged with the correct language attribute.³¹
• We determined whether words that appeared on each webpage matched the language to which the webpage was set.

Error Identification

• We determined whether mandatory form fields alerted users if the field was left blank.
• We determined whether there was a label for elements that required user input.
• We determined whether the label was programmed correctly.
• We determined whether examples were given to assist the user in correcting mistakes (for example, a warning when entering a letter in a field meant for numbers).

Color Accessibility

• We determined whether there was at least a 3:1 contrast in color and additional visual cues to distinguish hyperlinks, which WCAG recommends for users with colorblindness or other visual impairments.

See Finding 3 for issues we identified with MLSC’s website.

We used a combination of statistical and nonstatistical sampling methods for testing and did not project the results of our testing to any corresponding populations.

Data Reliability Assessment

MLSC Site Map

To determine the reliability of the site map that we received from MLSC management, we interviewed knowledgeable MLSC staff members and checked that certain variable formats (e.g., dates, unique identifiers, or abbreviations) were accurate. Additionally, we ensured that none of the following issues affected the site map: abbreviation of data fields, missing data (e.g., hidden rows or columns, blank cells, or absent records), and duplicate records. We also ensured that all values in the dataset corresponded with expected values.

Furthermore, we selected a random sample of 20 uniform resource locators (URLs)³² from MLSC’s site map and traced them to the corresponding webpages on MLSC’s website, checking that each URL and webpage title matched the information on the MLSC webpage. We also selected a random sample of 20 webpages from MLSC’s website and traced each URL and webpage title to the site map to ensure that we received a complete and accurate site map.

List of Interns Hired by Host Organizations

To determine the reliability of the list of interns hired by host organizations that we received from MLSC management, we reviewed the MLSC Application System’s system controls for access control, configuration management, contingency planning, segregation of duties, and security management. Additionally, we reviewed System and Organization Control reports³³ for the MLSC Application System. We also checked that certain variable formats (e.g., dates or abbreviations) were accurate. Additionally, we ensured that none of the following issues affected the list: abbreviation of data fields, missing data (e.g., hidden rows or columns, blank cells, or absent records), and duplicate records. We also ensured that all values in the dataset corresponded with expected values.

Furthermore, we obtained from MLSC physical documents known as Host Agreements and Reimbursement Summaries, which outline information for interns hired by host organizations during the audit period. We selected a random sample of 20 interns from the list of interns hired by host organizations and traced them back to the information compiled from physical documents, ensuring that the intern’s full name and hiring host organization matched. We also selected a random sample of 20 interns from the compiled physical documents and traced them back to the information pulled from the list of interns hired by host organizations to ensure that the intern’s full name and hiring host organization matched and that we received a complete and accurate list.

Based on the results of the data reliability assessment procedures described above, we determined that the site map and list of interns hired by host organizations were sufficiently reliable for the purposes of our audit.

21.   As mentioned earlier in this report, MLSC does not refer or recommend specific candidates for the HSAC internships. Instead, MLSC acts as an intermediary and created the MLSC Application System and has structured the HSAC eligibility criteria to increase the number of students within marginalized populations in the pool of applicants.

22.    Auditors use nonstatistical sampling to select items for audit testing when a population is very small, the population items are not similar enough, or there are specific items in the population that the auditors want to review.

23.   Recent graduates must have received their high school diploma no earlier than the past two calendar years and must not yet be attending college. High school graduates can intern the summer before their first year of college.

24.   According to Section 3A of Chapter 23A of the General Laws, a gateway city is “a municipality with a population greater than 35,000 and less than 250,000 with a median household income below the commonwealth’s average and a rate of educational attainment of a bachelor’s degree or above that is below the commonwealth’s average.” 

25.    Auditors use statistical sampling to select items for audit testing when a population is large (usually over 1,000) and contains similar items. Auditors generally use a statistics software program to choose a random sample when statistical sampling is used. The results of testing using statistical sampling, unlike those from judgmental sampling, can usually be used to make conclusions or projections about entire populations.

26.   Confidence level is a mathematically based measure of the auditor’s assurance that the sample results (statistic) are representative of the population (parameter), expressed as a percentage.

27.   Expected error rate is the number of errors that are expected in the population, expressed as a percentage. It is based on the auditor’s knowledge of factors such as prior year results, the understanding of controls gained in planning, or a probe sample.

28.   The tolerable error rate (which is expressed as a percentage) is the maximum error in the population that is acceptable while still using the sample to conclude that the results from the sample have achieved the objective.

29.       An element is a part of a webpage that contains data, text, or an image.

30.       The first focusable control is the first element a user will be brought to on a webpage when navigating with a keyboard.

31.       A language tag identifies the native language of the content on the webpage or PDF (e.g., a webpage in English should have an EN language tag). The language tag is listed in the webpage’s or PDF’s properties. This, among other things, is used to help screen readers use the correct pronunciation for words.

32.    A URL uniquely identifies an internet resource, such as a website.

33.    A System and Organization Control report, issued by an independent contractor, is a report on controls about a service organization’s systems relevant to security, availability, processing integrity, confidentiality, or privacy.

Date published:	June 12, 2025