How do you know if you are building a healthier place?

How do you know if you are building a healthier place?

This important question is a challenging one because it takes years for ambitious initiatives to achieve healthier communities. When we set out to evaluate the Best Babies Zone (BBZ) Initiative^[1], we realized that we needed an approach that would measure the long-term impact while also capturing the nuance of incremental community change. With this in mind, BBZ leaders at UC Berkeley partnered with Harder+Company Community Research on a developmental evaluation framework to measure the growth and expansion of the BBZ Initiative. Developmental evaluation allows the BBZ Initiative to learn in real-time to ensure that they are choosing the best way forward. This framework has helped measure and document accomplishments, challenges, and lessons learned in the six Best Babies Zones. We attribute our success to date to three aspects of our approach: valuing incremental change, flexibility in approach, and mixed-methods data collection.

Valuing incremental change 

In today’s data-driven environment, funders often ask grantees to demonstrate outcomes within the first year of funding. However, community transformation is an incremental process that takes years or generations to achieve. Evaluation must incorporate short-term indicators to uncover the otherwise invisible changes that support the broader outcome.

To assess progress in local Best Babies Zones, Harder+Company worked with leaders of the BBZ Initiative to define a pathway of health outcomes (illustrated on the right), weighing a sense of urgency in addressing inequities in birth outcomes against the time it takes to demonstrate change in a multi-generational health equity initiative. By considering indicators along a pathway to success, BBZ demonstrates progress towards long-term change and identifies challenges as they occur.

For example, early in the initiative we conducted a partnership survey with the first three zones. Survey questions measured:

• changes in the level of collaboration among partners,
• shared understanding of the mission and vision across the network, and
• partners’ perceptions of strengths and opportunities for the BBZ lead agency.

Survey findings explained how collaboration among community partners supports the transformational change the BBZ Initiative hopes to foster. It also identified areas of focus for local leadership including lack of common language among cross-sector networks and misaligned expectations about the nature of collaboration. Measuring partnerships at a point-in-time and changes over time has helped to track progress towards improved coordination in each zone. A wonderful place to get started with measurement is to use the Build Healthy Places Network’s MeasureUp site, which provides examples, tools, and resources to help you make your case, without having to become an economist.

Using a flexible approach

The BBZ approach is not one-size-fits-all. Rather, it is tailored to community conditions, opportunities, and aspirations. Because residents drive the action in each BBZ, the evaluation must also be tailored to fit local needs and interests, an increasing pressure as the Initiative has grown from three to six – and soon to be nine – unique communities.

Early in the evaluation, we learned that the same questions and methods could not be used in each pilot zone uniformly. Programming, interests, and priorities differed dramatically from community to community. To support the diversity and distinctiveness of each zone, Harder+Company developed a toolkit that offers a menu of eleven methods including community surveys, focus groups, and other community based participatory methods. Each method is accompanied by a step-by-step guide for implementation. With support from Harder+Company, zone leadership and partners choose the methods best suited to assess the programmatic efforts underway in their community.

Valuing a mixed methods approach

Quantitative population data can be instrumental in understanding local conditions and providing concrete evidence that neighborhood conditions are improving. However, population-based data is not always available or is limited by geographic boundaries. To supplement population-based data, the Harder+Company evaluation team worked with community partners to collect primary quantitative data using a community survey.

Yet, even with primary quantitative community data, not everything zones are interested in knowing is always captured. The use of quantitative data alone can leave practitioners wondering why something happened the way it did or how a program contributed to a particular outcome. Those questions call for qualitative information. The use of a mixed-method approach can lead to a deeper understanding of the change taking place, and provides information on harder to measure metrics including the experiences of racism and existing community protective factors.

With this understanding, the evaluation team conducted focus groups with residents who participated in BBZ Castlemont’s economic development and entrepreneurship programming. Data showed how BBZ Castlemont is making progress along the anticipated outcomes pathway including increased job training, increased financial literacy, and increased microfinance opportunities. Beyond confirming achievement of short-term and mid-term outcomes, participants provided a more nuanced explanation of how BBZ programming influenced positive trends in these indicators. 

Our approach to place-based evaluation

The BBZ Initiative is not alone in striving to measure change in complex environments. Changing the social, economic, and structural conditions of neighborhoods is a long-term effort, and requires a long-term investment in evaluation. The tension of showing outputs in place-based work must be balanced with a vision for showing that outcomes will shift over time, using incremental evaluation and assessment as a tool to support continued learning and growth. Figuring out what success looks like – and how we would know if we were there – continues to be central to the BBZ evaluation and to building local capacity to make change in the zones.

To learn more about Harder+Company Community Research, visit www.harderco.com. To learn more about the Best Babies Zone Initiative, visit www.bestbabieszone.org.

Footnotes

^[1] The Best Babies Zone Initiative, based at the UC Berkeley School of Public Health, supports communities across the United States to implement a resident-driven, place-based, multi-sector approach to community transformation – with the ultimate goal of reducing racial inequities in birth outcomes.