Metrics That Matter For States Under H.R. 1

State agencies are planning vast changes to their Medicaid and Supplemental Nutrition Assistance Program (SNAP) eligibility systems in accordance with H.R. 1, the megabill legislation enacted in July. 

Many agencies struggle to identify effective ways to track the impact of system changes over time and the experience of people impacted. Without adequate data, decision-makers may not be able to make informed choices,  which can hurt beneficiaries, state staff, and the program’s performance.

To address this challenge, states should implement a legible, flexible metrics system. States that collect and analyze relevant metrics in the midst of systems change will be able to track impact, and react more quickly as a result. We understand the importance of tracking these measurements at Code for America, and we prioritize them in every process we build. Here, we lay out how to build, capture, and effectively use metrics that are most relevant to H.R. 1 work requirements.

Bring diverse perspectives to the table

The first step to designing a metrics system is to convene the team that will decide on the relevant metrics and how to implement them. This team should incorporate a diverse set of voices responsible for the following:

• Policy: ensuring that the metrics reflect compliance with policy
• Client experience: tracking that new metrics capture client interactions with the system
• Operations: training staff to understand any new fields in the eligibility system
• Data: using the metrics available to produce reports and dashboards
• Engineering: determining how to store the necessary data elements

The “translation” aspect of this work—where policy and experience language meets technology—can be difficult. Bringing in civic technologists for these conversations can significantly increase how quickly a group agrees about which metrics to use and the ideal strategy.

Create processes to monitor and respond to metrics

The value of measurement is lost without an action plan. When metrics change, what should be done? Who is responsible for taking action? When creating a metrics system, create associated plans for validating, acting on, and revising those metrics.

Work with the teams above to determine:

• Who is responsible for deciding which metrics matter? When measuring impact, compliance, and experience, who defines the most important metrics?
• Which team is responsible for monitoring these metrics? When that team notices a change, who is notified?
• Which team is responsible for validating the metrics? If metrics are found to be inadequate, what is the process for revising them?

Ensure existing metrics are accessible to staff

Many Medicaid agencies implemented new required metrics during the Public Health Emergency (PHE). While SNAP agencies did not see the same requirements, both programs have existing metrics that can be utilized to determine the impact of changes. Before building new metrics, ensure that existing ones are accessible to those that need them; this allows impact to be measured throughout implementation.

Example: Ex parte rate

Let’s say a state is implementing a new data source for H.R. 1. Following the implementation, their ex parte rate drops 10 percentage points. The state finds that the new data overrides existing data that was used for ex parte in some cases.

Without monitoring the ex parte rate, the state couldn’t do this “before and after” comparison. Code for America’s Safety Net Scorecard lists metrics that states are likely already tracking, and that can be used to monitor the impact of a new implementation.

Brainstorm new metrics

Begin brainstorming a list of new metrics that provide visibility into H.R. 1 changes. The list below is a starting point. Consider the specific needs of your state, then adapt and expand the list accordingly.

• Number of people currently exempt from work requirements, broken down by exemption type
• Number of people currently compliant with work requirements, broken down by compliance type
• Number of people not exempt or compliant (accruing countable months—SNAP only)
• Number of people referred for SNAP E&T (SNAP only)
• Number of people with terminated benefits due to not demonstrating that they meet work requirements (this will look different in SNAP and Medicaid, but will happen in both)
• Number of people being automatically determined exempt and via which method
• Number of people being automatically determined compliant and by which compliance activity

Capture data elements in the system

The metrics above are calculated using data elements—the actual individual values being captured in the system. Data elements can be thought of as the building blocks used to create the metrics themselves. These elements are stored in the database and used by the data team to create metrics and reports.

Example: Number of people exempt from work requirements

Let’s say you want to know how many people are exempt from work requirements. While this number could be stored directly, it’s not flexible. What happens when you want to see how many people fit into a specific exemption category, for example: medical frailty?

Instead, the metric should be stored as data elements:

• For each person in the system, store their exemption status (TRUE or FALSE)
• For each person in the system, store their exemption reasons (one-to-many enumerable value)

Then, it’s easy to report the number of exempt people: it’s the number of people with (exemption status = TRUE).

It’s also easy to break down: the number of people exempt for medical frailty is (exemption status = TRUE) and (exemption reasons contains MEDICAL_FRAILTY).

Without data elements, agencies will have to exert outsized effort whenever they need to modify existing metrics or calculate a new metric. Storing data elements—the building blocks of the metrics—results in a more flexible system.

The data and engineering teams should work together to determine which elements make up the metrics requested. We’ve created an appendix of elements from which to start.

Create clear and actionable documentation

When a stakeholder—either internal or external—requests new metrics, one of the barriers to producing them is a lack of clarity regarding available data elements. Engineers and data team members must track down elements in disparate tables or databases, sometimes with confusing or vague names, which can cause delays.

A data documentation culture mandates that engineers and data team members create documentation while designing the data elements. Each data element should be documented with information including:

• What does this element represent?
• Where did this element come from? 
• What data type is this element?
• When was this element last updated?

Documentation will significantly reduce the amount of time needed to produce new metrics from existing elements or to change current elements.

Moving forward

Metrics for H.R. 1 implementation must be designed and implemented in parallel with systems changes. Without adequate metrics, states cannot measure the impact of changes and ensure they meet expectations. With them, however, agencies can optimize performance and ensure the best possible outcomes for clients, state staff, and programs as a whole. This feedback loop is critical to ensuring a compliant and high-performing program moving forward.

As states begin planning their changes to systems and the associated measurements, Code for America can provide targeted support, from technical assistance office hours with our technology and policy experts, to dedicated engagements for streamlining compliance verification and exemption determinations. For more information, see our support offerings or get in touch with us.