Focused Framework: A New Approach to Policy Research

A Focused Framework for Community Development
Build, Measure, Learn through Event Mapping
July 2015
Virginia Carlson, DataRubrics

I. Overview

The Focused Framework (Framework) is a measurement tool that supports a paradigm shift in social sector work. Rather than developing products or programs that may meet a need, social sector actors are beginning to understand that it is critical for philanthropy and its partners to continuously explore the causes of the problems they wish to address in order to achieve scalable and enduring change on behalf of low income people and under-resourced places. Such exploration may result in initiatives that are not necessarily new formal programs, such aligning existing funding or partnerships, activating existing assets, or changing administrative procedures. There is a growing sense that these latter kinds of initiatives are more likely to have a long-run systemic impact that will endure after the formal funding has ended.

The Framework is meant to assist an initiative in focusing and developing its approach by stipulating a series of “thinking and measuring steps” that elucidates connections between problems and solutions and forces actors to lay out and test assumptions, hypotheses, and logical connections between interventions and expected outcomes. Connection-testing is done with build/measure/learn activities via event mapping.

Embedded tenets:
• Actors work best when they are open to questioning their own assumptions by engaging in a build/measure/learn feedback culture in order to explore those assumptions
• Much can be learned from lean learnings—quick trials, research, and experiments in order to design strategies
• Solution-finding must involve the people whose stories we are trying to change
• The “measures” in build/measure/learn activities can become key metrics for understanding how healthy communities function
II. The Learning Framework

Recognizing that the practice of systems intervention is neither linear nor predetermined, the Learning Framework is nevertheless laid out as a phased process. The activity of approaching the collective tables’ thinking in a step-by-step manner forces everyone to confront fuzzy thinking, helps slow down the impulse to implement large-scale (as opposed to scalable) projects or programs, and gives sites a tool by which they can monitor their own progress. In brief, the steps are
• Define Problem
→ articulate the dimensions of the problem to be addressed
• State Goal
→ set a realistic goal
• Identify Key Assumptions and Choose Strategy Areas
→ articulate understanding of the root causes of the problem: “This problem exists because”
→ choose a few strategy areas, based upon assumptions about why the problem exists and where the actor has leverage or experience
• Articulate Hypotheses and Choose Specific Efforts
→ make connections between efforts and possible outcomes: “If we do X, then Y will come about”
• Trial-run Efforts
→ using Build Measure Learn (BML) activities with event mapping, trial-run the chosen “X’s”
• Reflect
→ reflect on learnings from the BML activity with stakeholders
• Act
→ take next steps based on learnings, which may include returning to a prior step, scaling up the effort, or re-running the activity

Example
In order to address the problem of under-resourced immigrant communities, we’ve set a goal to increase family income by 2% within a year. Several assumptions have presented themselves through field research, brainstorming, etc.: immigrant communities are under-resourced because of language barriers, because of a lack of skills, because families are reluctant or unable to access supplemental income programs such as food stamps and energy assistance. We decide to focus our strategy on income support programs. We hypothesize that if only immigrant families knew about the programs, they’d sign up for them.

In order to test this, the specific effort (linked to the hypothesis) we undertake can be summarized as “if we send home information in children’s school backpacks then their parents will be enticed to access supplemental income programs.” (We might have chosen another effort within this same hypothesis, such as information via telenovelas).

Yet, instead of jumping to a large program that creates materials and targets all public school children we decide to run a BML activity. We target just one school where we know that 99% of the families qualify for free or reduced-price lunch (so are likely eligible for supplemental programs), and one supplement program (say, energy assistance) for the test activity.

We begin by creating a simple event map matrix that marks out the goal the steps, and the measures for each step. As we run the test, we also collect data that tells us what happened. (See below).

In this example we learn that our expectations were not met in terms of timing for the development of the materials; that we aren’t sure how many of the families received and read the materials (only about 25% of families sent back the “yes we saw this material” card), and the energy assistance program only reported two new families from this school took up energy assistance.

Reflecting upon the data from this experiment, we might decide several things: 1) sending home materials through school is not a viable option, since we have no way of knowing whether or not families saw the material; but at the same time 2) of the 23 families who DID report having seen the materials, two did take up energy assistance and so we did make our goal in terms of percentages (we expected approximately 10% of 118 families to take-up, we did get about 10% of the 23 families we know saw the materials). Perhaps we can conclude that once having gotten the information, families did take action, but we are not sure whether the delivery mechanism is correct.

We also get feedback from stakeholders, including the voice of the client, teachers themselves, the energy assistance program, etc. From them we learn that not all teachers knew what to do with the cards that were returned, and a bundle ended up in the dust bin; a number of parents believed that by returning the card, they’d get more information on the energy program from the school; a number of parents don’t bother reading any of the flyers/information that comes home in backpacks; the materials were written in formal Spanish but most of the families speak Latin American Spanish, etc.

Taking action from these learnings, we are likely to decide that our effort—“send information home in children’s backpacks” is likely not a good approach. We don’t throw away the entire hypothesis, which was that if we gave families information about the supplemental income programs, they would sign up for them. But we’d perhaps test one or another efforts—peer learning through CBO’s or infomercials on daytime television. If neither of these worked, then we’d might likely conclude that the hypothesis is wrong—even when families know about the programs, they still do not sign up for them.

We also don’t throw out the assumption—that families are reluctant or unable to access supplemental income programs such as food stamps and energy assistance. We’d turn to our other hypotheses suggested by the assumption (e.g., families need safe space to apply) and test out efforts within that. When we find something that works, we then consider whether it really is scalable, about what would have to be re-aligned, to create population-level change.

In this way, the process of discovery and learning repeats itself. As we learn, other assumptions and strategies emerge and are fed back into the “stepped learning process” we have described here.

A note about a strategy for measuring change and whether we are moving to longer-term goals. In this example, by involving a local administrator for energy-assistance we have been able to attain our person-level, short-term measure of “whether or not enrolled in a supplemental income program.” What we will want to track in the medium- and long- term then are more aggregate measures related to income assistance and children living in poverty. It is likely that state-run assistance programs release aggregate data on families receiving income assistance, and American Community Survey data will give us the information we need to look at the relationship between assistance take-up and poverty levels.

 

Leave a Reply

Your email address will not be published. Required fields are marked *