Overview
CCA did not ensure that Dell Marketing LP met all of the customer service standards and service-level agreements (SLAs) that were established in CCA’s contract with Dell. Specifically, CCA did not properly monitor the quality of customer service provided by Dell and did not impose contractually agreed-upon penalties when Dell did not meet certain customer service standards. The table below summarizes the customer service standards in CCA’s contract with Dell.
SLAs Related to Customer Service
|
SLA |
Definition |
Penalties |
Incentives |
Evaluation Frequency |
Measurement Method |
|
SLA-2 |
Maintain a maximum monthly average time to answer of 60 seconds |
$5,000 if monthly average time to answer exceeds 60 seconds |
N/A |
Monthly |
Calls Report |
|
SLA-3 |
Maintain a monthly average percentage of calls answered in 30 seconds or less at 70% or higher |
$2,500 if monthly percentage of calls answered in 30 seconds or less is below 65% |
$2,500 if monthly percentage of calls answered in 30 seconds or less is above 80% |
Monthly |
Health Connector Audit |
|
SLA-4 |
Maintain a maximum monthly average abandoned call rate of 3% |
$2,500 if monthly average abandoned call rates exceed 3%; $5,000 if monthly average abandoned call rates exceed 5% |
$2,500 if monthly average abandoned call rate is less than 1.5% |
Monthly |
Calls Report |
|
SLA-5 |
Maintain a maximum time to answer of 2 minutes |
$2,500 if 10% or more of calls exceed 2 minutes; $5,000 if 30% or more of calls exceed 2 minutes |
N/A |
Monthly |
Calls Report |
|
SLA-8 |
Receive a top-two rating (on a five-point scale) in 85% of total customer surveys for customer satisfaction on CCA-approved customer surveys |
$2,500 if quarterly average overall customer satisfaction scores are less than 85%; $5,000 for each time quarterly average overall customer satisfaction scores are less than 75% |
$2,500 if quarterly average overall customer satisfaction scores are greater than 90% |
Quarterly |
Customer Satisfaction Report |
We reviewed samples of the monthly call reports and the quarterly customer satisfaction reports provided to CCA by Dell and noted that Dell did not meet the customer service standards for SLA-5 in February and May 2016 or for SLA-8 in the first two quarters of 2016.
Although CCA did properly assess the penalty for SLA-5 for May 2016, CCA did not assess all $15,000 of the penalties it was entitled to assess under the contract; it assessed only $2,500, as detailed below.
Findings from SLA Sample Testing
|
Data Tabulation Period |
SLA-5 |
SLA-8 |
Penalty |
|---|---|---|---|
|
August 2015 |
5% |
N/A |
$0 |
|
November 2016 |
0% |
N/A |
$0 |
|
January 2016 |
23%* |
N/A |
$0 |
|
February 2016 |
30% |
N/A |
$5,000 |
|
May 2016 |
14% |
N/A |
$2,500† |
|
First Quarter 2016 |
N/A |
73% |
$5,000 |
|
Second Quarter 2016 |
N/A |
76% |
$2,500 |
* The SLA was waived during open enrollment (November through January).
† Dell was properly penalized for not meeting the requirements of this SLA.
Authoritative Guidance
CCA’s contract with Dell (Agreement by and between Commonwealth Health Insurance Connector Authority and Dell Marketing L.P. for Customer Service Contract Center and Business Operations Services) requires CCA to designate a customer services agreement officer who will be responsible for reviewing and approving Dell’s payment vouchers and for verifying Dell’s compliance with the contract by reviewing the performance metrics. The contract specifies penalties CCA will assess to Dell for not meeting the performance metrics. (See the SLAs Related to Customer Service table for a detailed breakdown of the contractually required penalties.)
CCA management is responsible for making sure that the agency complies with all the terms and conditions of this contract and, in particular, that the responsibilities of the customer services agreement officer are performed effectively.
Reasons for Issue
CCA did not have controls in place to effectively monitor Dell’s compliance with the contractual terms and conditions related to the achievement of the specified customer service metrics. Specifically, although CCA did have policies in place requiring review of all vendor invoices by the customer services agreement officer before approval, there was no system in place to ensure that this review was performed before payment. Further, because of a lack of cross-training, Dell’s payment vouchers were not reviewed when the customer services agreement officer was unavailable.
Recommendations
- CCA should implement internal controls to allow it to monitor performance metrics to determine Dell’s compliance with the SLAs.
- CCA should provide adequate cross-training of key personnel who are responsible for reviewing invoices for accuracy.
Auditee’s Response
The total contract awarded to Dell amounted to $47,637,422 . . . for the period of July 1, 2014 through June 30, 2016. The agreement did contain incentives and penalties that were in place and enforced during this time period. On November 29, 2016 the CCA notified the State Auditor that it had identified an error in how Dell was assessed for a violation of Service Level Agreement (SLA) SLA-8, and issued a corrective memo identifying that an additional $5,000 should have been assessed at that time. The CCA took corrective actions and added these penalties to a December 2016 invoice, bringing the total assessed penalty for SLA-8 to $7,500.
In an effort to ensure vendor compliance and management, the CCA was actively seeking a Vendor Relations and Strategy Manager prior to and during the time of this audit to oversee and manage this contract. That person was hired on February 13, 2017. Among other responsibilities, the Vendor Relations and Strategy Manager ensures that SLAs are calculated correctly, that the vendor is held to the terms of the agreement, and that invoices are calculated by the vendor correctly. As a result of this control, CCA believes that the discrepancies identified by the audit will not recur.
Auditor’s Reply
In its response, CCA points out that it identified at least one instance in which it did not assess a penalty for Dell’s noncompliance with the SLAs that we included in our analysis. CCA states that it made the appropriate adjustment in this instance, and we believe it should continue to resolve the issue of any outstanding unassessed fines. However, as noted above, our concern is that CCA did not have controls in place to effectively monitor Dell’s compliance with the contractual terms and conditions related to the achievement of the specified customer service metrics. Without such controls, this problem could keep occurring and go undetected. Based on its response, CCA is taking measures to address our concerns in this area.
| Date published: | January 16, 2018 |
|---|