Originally posted on July 30,2015
Data collection and data analysis through collective impact is challenging to the private, public and nonprofit sectors because of challenges unique to each of their working environments.
My team just closed out a whirlwind tour of our The Integration Initiative sites, in which we provided Results Based Accountability training to our collective impact partners. The Results Based Accountability (or RBA) framework is a useful tool to help social sector collaborations better align their work through a process of identifying a shared result, then developing outcomes and shorter-term measures that link programs to this overall shared result. We are now using the model with all of our Integration Initiative partners as a fundamental part of their collective impact work.
Because our site partners brought representatives from the private, public and nonprofit sectors to these RBA trainings, my team and I had a unique look at how each of the different sectors approach conversations about outcomes and data tracking in collective impact. Each sector has their own challenges that they bring to conversations about social service data infrastructure. Understanding these challenges and overcoming them can help us build a New Urban Practice that creates dramatically better results for low income people, faster.
Nonprofit Sector: Collecting Data
Most nonprofits have a basic problem with data infrastructure: collecting the data in the first place. Most nonprofits do not have robust enough infrastructure to actually track data in a way that can be helpful to a collective impact initiative.
This lack of capacity exists in the nonprofit sector for many reasons, and if you work at a social service nonprofit you know them better than I do. Many funders do not fund what they see as “overhead” expenses, which is often what nonprofits need to fund data infrastructure. Funders also require nonprofits to collect certain types of data, which can take away staff time for collecting data that may be more relevant to their work. (Ford’s recent shift to provide more flexible funding to its grantees is a great example of how philanthropy can better support data infrastructure in service of creating a New Urban Practice.) It is not just about the funders, though: Many nonprofit staff members feel that collecting and analyzing data takes away from time to serve clients, which is the true purpose of their work.
Private Sector: Valuing Social Change Beyond Dollars
The private sector’s major challenge with building data infrastructure to track social outcomes centers on their struggle with valuing social change beyond that of dollars earned or created. That isn’t to say businesses only care about profit—most, if not all, businesses understand there is more to creating value than just profit. But typically, businesses are more comfortable measuring things like revenue created from minority-owned small businesses, or increased earnings from low-income people in a particular under-resourced neighborhood.
Businesses can learn from nonprofits and government, who understand that change happens outside of traditional economic forces. Nonprofits and government tend to be more comfortable in agreeing to track things like increased confidence in ability to secure a job or if a person feels safe in his or her community.
Private sector engagement has been a challenge for many collective impact initaitives, and we think that creating a more inclusive data infrastructure system can help private sector actors become more involved in collective impact initiatives. For example, The Integration Initiative site in New Orleans has created an outcomes framework that includes employment rates—which has a clear link to economic activity. But the collective impact initiative also wants to change enforcement policies, which may require businesses to get outside their comfort zone to understand how they can contribute.
Public Sector: Measuring Impact, Not Outputs
The public sector’s big challenge is moving beyond collecting data on outputs to managing data tracking systems that can show impact on people’s lives.
Governments tend to be more comfortable working with data that show how well a program is doing what it is supposed to be doing, such as providing job referrals to unemployed residents. And, many agencies already collect data on types services provided and clients served, which are important data to have in collective impact work. But to know whether or not a collective impact initiative is making broad, systems level changes, it needs to build data infrastructure that can track impact, such as whether or not a high school graduate can earn a living.
Agreeing to track data on impact requires agencies to think on a bigger scale, and potentially partner with other agencies or organizations. The bureaucracies within government agencies can make these larger-scale partnerships difficult, and therefore create resistance from internal staff. There is also more external scrutiny of the public sector which makes it culturally difficult to announce goals that may not be achieved in the short term.
If we can get government to think on the scale of impact, it could open up a lot of opportunities, not just for collective impact initiatives, but for the entire social sector. Recent work on Pay for Success has shown what happens when government begins focusing on impact rather than outcomes.
These insights are, of course, generalizations and have many exceptions. We want to hear from your experiences—particularly if you have been using the RBA framework in collective impact. Have you observed similar challenges? Or others we might have missed? How have you overcome them? Or how have you seen other organizations overcome them? Are there any promising examples of organizations that don’t have these challenges? We’d love to hear in the comments examples of organizations that don’t have these challenges, or have been able to overcome them.