Developing solutions to build a brighter future for children, families and communities

The Annie E. Casey Foundation is a private foundation created in 1948 by UPS founder Jim Casey and his siblings, who named it to honor their mother. In 2017, the Foundation provided $135 million in grants, direct technical assistance and consulting, working with partners and grantees in all 50 states, as well as Washington, D.C., the U.S. Virgin Islands and Puerto Rico.

In an interview with Ayo Atterberry, a senior associate with the Foundation’s Evidence-Based Practice Group and Leap Ambassador, community of experts in the field of high performance, we examine how a foundation can use data and evidence to advance organizations working on behalf of disadvantaged children and families.

Promising programs for young people often struggle to get data, funding and time to measure their success and take their solutions to greater scale. The Foundation’s Expanding Evidence portfolio, which Atterberry leads, aims to help emerging programs for families of color build evidence of their effectiveness.

The Casey Foundation provides grant funding to improve outcomes for children and families through innovative practice, policy, and service delivery. Could you highlight the reasons the Foundation focuses on evidence building and how that relates to your specific area of work?

Our overall goal is to help build a brighter future for children and families in our communities and our perspective is that, since limited resources are available for such an ambitious goal, we should focus on initiatives that are demonstrating that they are proven effective in diverse settings. We are interested in supporting what works.

This means following a dual path: on one side, we have to support those efforts that have proven to be effective in order to make them become stronger, more resilient and expand their reach. On the other, we want to work on innovation, by providing early evidence about new solutions that can help tackle the issues we care about.

My area of work also supports Casey’s Racial and Ethnic Equity and Inclusion portfolio to make sure all children are able to reach their full potential in life regardless of their race, ethnicity or community of residence. If you look at the latest data in our Race for Results: 2017 Kids Count Policy Report, there are 74 million children under the age of 18 in the United States, and 49 percent of them are children of color. The obstacles facing many children of color, particularly those in immigrant families, are difficult to surmount. African-American, American Indian and Latino children face some of the biggest obstacles on the pathway to opportunity.

How does this evidence-based orientation guide your funding choices?

We think of evidence as a continuum or set of stairs that ranges from innovation to established programs, and we reach out to organizations doing work at each of these evidence levels. For new programs, we look for a strong program design and focus on helping organizations document that design, as well as helping them think about capacity to get to the next level. This means we will normally invest in very early-stage development. Key questions we might ask include: is the organization strong in terms of leadership? Does it need technical empowerment? Should we help it develop a logic model or theory of change to better define the way it will achieve outcomes? Or does the organization have a clear model and the need is to implement a performance management system? We also support organizations in a more advanced stage of development, with a running program that has already gone through some assessment suggesting there may be positive outcomes. In that case, our interest would be in funding a randomized controlled trial (RCT) to rigorously assess actual effectiveness.

Basically, our flexible funding aims at understanding what the specific organization needs at that point in time and to help it embark on the evidence-based path – whether this means developing a logic model, adopting a performance management system and an internal protocol to collect data, or rigorously assessing results through an RCT. We want to put our grantees in the evidence continuum so that they can start a reasoned path to results-oriented growth.

We want to put our grantees in the evidence-continuum so that they can start a reasoned path to results-oriented growth

Do you have a dedicated pool of resources to support evidence-based practice and to what extent are you actively engaged with your grantees once funding is granted?

Within my current portfolio of 17 grantees, we are funding three RCT studies and on average we tend to have the same percentage of grantees in early and advanced stages of development. Within Casey, we have a Research and Evaluation Unit that is dedicated to evaluation and to providing support to the other departments of the Foundation on outcomes assessment within particular grants.

We work with grantees in different ways, depending on where they are. With early-stage organizations, we might work very closely with someone building a logic model because we need to be sure that they are taking a focused approach on the outcomes we would like to reach. Also, until they are ready to assess the difference they make in outcomes, we might help them look for what other successful organizations are doing for similar participants and complement that with what research and data show to inform the direction they are taking to produce change.

If the program grantee is working in an advanced phase of development along our evidence continuum and participating in an RCT, we would be less present, and rely on independent evaluators to provide that level of evaluation. My role is mainly to identify needs and monitor the work by providing direct support and identifying experts, consultants, researchers and other specialists that we need to engage to assist our grantees. 

Early-stage but promising programs seeking to show that they improve outcomes for children and families often face a daunting hurdle: how can they develop systematic processes and data that will help them measure their success?

What is the Expanding Evidence Portfolio?

Early-stage but promising programs seeking to show that they improve outcomes for children and families often face a daunting hurdle: how can they develop systematic processes and data that will help them measure their success? Casey’s Expanding Evidence portfolio aims to strengthen the capacity of emerging programs for families of color to hone their services, their identity and the processes that make them successful at what they do so they can systematize and measure the efficacy of their program.

The purpose is to support organizations that are developing programs for people of color to become more evidence based by providing technical assistance to help them build their evaluation capacity, develop logic models and ultimately ready them to meet the requirements of University of Colorado’s Blueprints for Healthy Youth Development, the industry standard-bearer for evidence-based intervention and prevention programs. Although the programs vary in the services they provide, they are all at the point in their development where they can be brought to the next level of professionalization.

Which are the main suggestions you would point out to develop an effective assessment framework?

In my previous experience, I worked as director of outcomes, assessment and learning at  Venture Philanthropy Partners (VPP), a philanthropic investment organization led by Carol Thompson Cole and founded by Mario Morino that helps great leaders build strong, high-performing nonprofit institutions. They had a great model which envisioned three to five years of support to grantees. The experience taught me that investing over a longer term is most helpful when it comes to building evidence.

Besides providing resources and funding evaluation studies, philanthropy should support ongoing incremental processes (such as the development of a performance management system of the grantee) to make sure that gains are sustained long after their investments end. Our goal should be to integrate a culture of evidence and assessment within the organization rather than supporting a once-in-a-while assessment effort – which can be helpful in the moment, but does not help shape the culture, spirit and vision of the grantee in the long term.

If funders want to see change, they should aim to instill in their grantees a culture of evidence and evaluation, rather than supporting isolated assessment initiatives.

For further information