Post From Expert Insights
How do you know if you are building a healthier place?
This important question is a challenging one because it takes years for ambitious initiatives to achieve healthier communities. When we set out to evaluate the Best Babies Zone (BBZ) Initiative, we realized that we needed an approach that would measure the long-term impact while also capturing the nuance of incremental community change. With this in mind, BBZ leaders at UC Berkeley partnered with Harder+Company Community Research on a developmental evaluation framework to measure the growth and expansion of the BBZ Initiative. Developmental evaluation allows the BBZ Initiative to learn in real-time to ensure that they are choosing the best way forward. This framework has helped measure and document accomplishments, challenges, and lessons learned in the six Best Babies Zones. We attribute our success to date to three aspects of our approach: valuing incremental change, flexibility in approach, and mixed-methods data collection.
Valuing incremental change
In today’s data-driven environment, funders often ask grantees to demonstrate outcomes within the first year of funding. However, community transformation is an incremental process that takes years or generations to achieve. Evaluation must incorporate short-term indicators to uncover the otherwise invisible changes that support the broader outcome.
The BBZ approach is not one-size-fits-all. Rather, it is tailored to community conditions, opportunities, and aspirations. Because residents drive the action in each BBZ, the evaluation must also be tailored to fit local needs and interests, an increasing pressure as the Initiative has grown from three to six – and soon to be nine – unique communities.
Early in the evaluation, we learned that the same questions and methods could not be used in each pilot zone uniformly. Programming, interests, and priorities differed dramatically from community to community. To support the diversity and distinctiveness of each zone, Harder+Company developed a toolkit that offers a menu of eleven methods including community surveys, focus groups, and other community based participatory methods. Each method is accompanied by a step-by-step guide for implementation. With support from Harder+Company, zone leadership and partners choose the methods best suited to assess the programmatic efforts underway in their community.
Valuing a mixed methods approach
Quantitative population data can be instrumental in understanding local conditions and providing concrete evidence that neighborhood conditions are improving. However, population-based data is not always available or is limited by geographic boundaries. To supplement population-based data, the Harder+Company evaluation team worked with community partners to collect primary quantitative data using a community survey.
Yet, even with primary quantitative community data, not everything zones are interested in knowing is always captured. The use of quantitative data alone can leave practitioners wondering why something happened the way it did or how a program contributed to a particular outcome. Those questions call for qualitative information. The use of a mixed-method approach can lead to a deeper understanding of the change taking place, and provides information on harder to measure metrics including the experiences of racism and existing community protective factors.
With this understanding, the evaluation team conducted focus groups with residents who participated in BBZ Castlemont’s economic development and entrepreneurship programming. Data showed how BBZ Castlemont is making progress along the anticipated outcomes pathway including increased job training, increased financial literacy, and increased microfinance opportunities. Beyond confirming achievement of short-term and mid-term outcomes, participants provided a more nuanced explanation of how BBZ programming influenced positive trends in these indicators.
Our approach to place-based evaluation
The BBZ Initiative is not alone in striving to measure change in complex environments. Changing the social, economic, and structural conditions of neighborhoods is a long-term effort, and requires a long-term investment in evaluation. The tension of showing outputs in place-based work must be balanced with a vision for showing that outcomes will shift over time, using incremental evaluation and assessment as a tool to support continued learning and growth. Figuring out what success looks like – and how we would know if we were there – continues to be central to the BBZ evaluation and to building local capacity to make change in the zones.
 The Best Babies Zone Initiative, based at the UC Berkeley School of Public Health, supports communities across the United States to implement a resident-driven, place-based, multi-sector approach to community transformation – with the ultimate goal of reducing racial inequities in birth outcomes.