Slide 1: Evaluation Update: One Care Demonstration under the Financial Alignment Initiative
Jennifer Baron
Medicare-Medicaid Coordination Office
Centers for Medicare & Medicaid Services
July 9, 2019
One Care Implementation Council meeting
Slide 2: Evaluating the Impact of the Demonstrations under the Financial Alignment Initiative
•	Section 3021 of the Affordable Care Act requires an evaluation of CMS demonstrations using Innovation Center authority that includes:
o	‘‘(i) the quality of care furnished under the model, including the measurement of patient-level outcomes and patient centeredness criteria determined appropriate by the Secretary; and 
o	(ii) the changes in spending under the applicable titles by reason of the model.” 
•	In 2011, the CMS Innovation Center selected RTI International as the evaluation contractor for the Financial Alignment Initiative (FAI)

Slide 3: Evaluation Includes Qualitative & Quantitative Components
Qualitative analyses: 
•	Use data from interviews, focus groups, and surveys
•	Answer questions such as: 
o	Do beneficiaries perceive improvements in access to care, care delivery, personal health outcomes, and quality of life?
o	How was the implementation experience?
Quantitative analyses: 
•	Use an “intent to treat’ methodology analyzing the experience of the demonstration eligible population, not just the enrolled population
•	Use a demonstration-specific comparison group comprised of similar beneficiaries from select states
•	Answer questions such as: 
o	What is the impact of the demonstration on utilization of inpatient services? Medicare Parts A & B costs? 
Slide 4: Data Sources
•	Implementation status: 
o	Stakeholder interviews
o	Data provided by MassHealth
o	Demonstration policies, contracts, and other materials
•	Beneficiary experience: 
o	Enrollee focus groups
o	Grievance and appeals data provided by the plans, MassHealth, CMS, and My Ombudsman
o	Beneficiary surveys, including: 
•	The Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey 
•	The Quality of Life survey administered in 2015 and 2016 by MassHealth with the Implementation Council and UMass Medical School
o	Medicare service utilization: 
•	Encounter data submitted by plans, Medicare claims
o	Medicare Parts A & B cost results: 
•	Capitation payments to participating plans 
•	Medicare claims
Slide 5: FAI Evaluation: Demonstration Results to Date
•	So far, CMS has posted 11 evaluation reports covering 7 demos:
o	4 demos have regression-based utilization results using a comparison group
o	6 demos have regression-based Medicare Parts A & B cost savings results using a comparison group
Cumulative results for the 4 reports with regression-based utilization results (Statistically significant at p-value ≤ 0.05):
Statistically significant reductions (desired effect):
•	Inpatient: 3
•	SNF: 3
•	Long-stay NF: 3
Results not statistically significant (suggesting no effect):
•	Inpatient: 0
•	SNF: 1
•	Long-stay NF: 0
Statistically significant increases (undesired effect):
•	Inpatient: 3
•	SNF: 0
•	Long-stay NF: 1
Slide 6: One Care Highlights: Beneficiary Experience 
Focus Group Results

•	In 2016, RTI conducted 3 focus groups (FGs) in Holyoke and Springfield with 12 participants. 2 FGs were conducted in Spanish and 1 in English. The English group focused on the experiences of black enrollees
o	Many expressed satisfaction with the demo, especially with the availability of new benefits/ provided through care coordination, such as:
•	Dental, optical, smoking cessation, nutrition, weight loss coaching, in-home behavioral health services and the elimination of pharmacy co-pays
o	Most expressed satisfaction with their primary care providers, though feedback was mixed on access to services, the degree of provider choice and whether various providers worked as a team
•	In 2017, RTI conducted 8 FGs in Springfield and Boston with 44 participants. 2 FGs were in Spanish and 6 in English. 2 FGs each focused on the experiences of: enrollees with behavioral health needs, enrollees using long-term services and supports, black enrollees, and Hispanic enrollees
o	Participants generally expressed satisfaction with One Care, often referencing the receipt of new or expanded services
o	Most participants were able to identify a person responsible for coordinating their care; many participants also reported care coordinator turnover
Slide 7: One Care Highlights: Beneficiary Experience 
CAHPS Results
•	Participating plans are required to conduct the CAHPS beneficiary survey annually and include supplemental questions from RTI and from MassHealth and the Implementation Council 
o	To report CAHPS in a given year, the plans must have a minimum of 600 enrollees as of July 1st of the preceding calendar year
o	In 2016, 68 - 87% of respondents for each plan rated their health plan at the highest levels, a 9 or 10 
o	In 2017, 70 - 78% of respondents for each plan rated their health plan at the highest levels, a 9 or 10
o	Overall performance on key composite measures generally improved from 2015 (the first year of CAHPS data for One Care) to 2017
Slide 8: CAHPS: Percent rating health plan a 9 or 10
Note: A bar graph depicts the difference between CCA, TUFTS, Medicare Advantage avg, and MMP avg during the years of 2015, 2016, and 2017.
•	In 2015, CCA: 70%, TUFTS: 62%, Medicare Advantage avg: 62%, MMP avg: 51%.
•	In 2016, CCA: 77%, TUFTS: 58%, Medicare Advantage avg: 61%, MMP avg: 59%.
•	In 2017, CCA: 75%, TUFTS: 64%, Medicare Advantage avg: 64%, MMP avg: 63%.
Slide 9: CAHPS: Percent rating drug plan a 9 or 10
Note: A bar graph depicts the difference between CCA, TUFTS, Medicare Advantage avg, and MMP avg during the years of 2015, 2016, and 2017.
•	In 2015, CCA: 76%, TUFTS: 62%, Medicare Advantage avg: 62%, MMP avg: 56%.
•	In 2016, CCA: 78%, TUFTS: 67%, Medicare Advantage avg: 61%, MMP avg: 61%.
•	In 2017, CCA: 78%, TUFTS: 70%, Medicare Advantage avg: 63%, MMP avg: 64%.
Slide 10: CAHPS: Percent reporting health plan “usually” or always” gave them information they needed
Note: A bar graph depicts the difference between CCA, TUFTS, Medicare Advantage avg, and MMP avg during the years of 2015, 2016, and 2017.
•	In 2015, CCA: 83%, TUFTS: 86%, Medicare Advantage avg: 80%, MMP avg: 73%.
•	In 2016, CCA: 82%, TUFTS: 76%, Medicare Advantage avg: 81%, MMP avg: 79%.
•	In 2017, CCA: 90%, TUFTS: 89%, Medicare Advantage avg: 87%, MMP avg: 86%.
Slide 11: One Care Highlights: Service Utilization
Comparison Group
•	Regression analyses compare demonstration and comparison group experiences, controlling for beneficiary and area-level characteristics
•	RTI selected comparable metropolitan statistical areas (MSAs) from states in/outside Massachusetts, to closely match the demonstration MSAs
o	Comparison group states include: 
•	Alabama, Kentucky, Maryland, Massachusetts, Mississippi, North Carolina, Pennsylvania, Virginia, West Virginia, and Wisconsin
Slide 12: One Care Highlights: Service Utilization
Results
Cumulative One Care results through 2016 show: 
•	Decrease in the probability of long-stay nursing facility use 
•	No change in ER visits or SNF admissions
•	Increases in inpatient admissions, ACSC admissions, and all-cause readmissions through 2016
•	Improvements from 2015 to 2016
Note: Graph labeling the difference between the years of 2013 to 2015 and 2013 to 2016
Probability of long-stay nursing facility use 
•	2013 – 2015: -10.6%
•	2013 – 2016: -12.2%
Inpatient admissions
•	2013 – 2015: +3.9%
•	2013 – 2016: +3.8%
Inpatient admissions for ambulatory care sensitive conditions (ACSC)
•	2013 – 2015: +15.8%
•	2013 – 2016: +10.8%
30-day all-cause readmissions
•	2013 – 2015: +12.7%
•	2013 – 2016: +7.3%
ER visits
•	No change for either
Skilled Nursing Facility (SNF) admissions
•	2013 – 2015: +4.9%
•	2013 – 2016: No change
Slide 13: One Care Highlights: Medicare A/B Cost 
Results
•	Regression analyses compare demonstration and comparison group experiences, controlling for beneficiary and area-level characteristics
•	Medicare Parts A & B cost results to date indicate neither gross Medicare savings nor losses due to One Care through 2016
•	Analyses do not yet include Medicaid data. They will be incorporated into future calculations as Medicaid data become available
Note: Graph labeling the difference in each time period with PMPM cost impact and the 95% confidence interval (none of the results are statistically significant)
Demo Year 1 (Oct 2013 – Dec 2014) 
•	PMPM cost impact − $10.42
•	95% confidence interval - (−41.36, 20.52)
Demo Year 2 (Jan 2015 – Dec 2015)
•	PMPM cost impact − $9.82
•	95% confidence interval - (−20.53, 40.18) 
Demo Year 3 (Jan 2016 – Dec 2016)
•	PMPM cost impact − $20.09
•	95% confidence interval - (−26.44, 66.62)
Cumulative results (Oct 2016 – Dec 2016)
•	PMPM cost impact − $5.26
•	95% confidence interval - (−23.65, 34.17)
Slide 14: Evaluation Reports
•	The fourth evaluation report for One Care expected in 2020
•	The One Care evaluation reports, and reports for other demonstrations under the FAI, are posted on the MMCO website.
Slide 15: Additional Data on One Care and FAI
•	CMS also posts online performance data for One Care and other capitated model demonstrations under FAI, including: 
o	CAHPS results 
o	Enrollment, enrollee age, and assessment completion rates
o	Care coordination