NextBillion - An Initiative of the William Davidson Institute at the University of Michigan

As impact investing evolves, it is exciting to witness the growing demand for impact measurement by investors and the ecosystem at large. But while the momentum and seriousness behind measuring impact is laudable, as a practitioner at a fund, it can get confusing. There is a risk of drowning in the deluge of available information, frameworks and methods. So how can investors choose the right impact measurement strategy? What are the factors to consider while adopting a method? How much is too much, and what are the boundaries for what should be measured and what shouldn’t be measured?

At Upaya Social Ventures, we are constantly asking ourselves these questions. We work at the forefront of fighting poverty, by funding and supporting scalable businesses that create jobs for the extremely poor. Since our founding in 2011, our investments have created 25,000 quality jobs for the poor in India – and we are determined to accelerate our reach over the next three years, to create an additional 25,000 quality jobs in India’s poorest regions. To achieve this ambitious goal, it’s more crucial than ever for us to create a robust strategy, responsive to the changing times, that helps us measure the breadth and depth of our impact.

To that end, we are currently consolidating our learnings, studying relevant models and best practices in the industry, and working on our impact measurement and management strategy, to help us positively influence job quality for the extremely poor in India. As we work through this process, we’re sharing what we have learned in our 11 years of supporting impactful entrepreneurs, and we’re outlining the tenets that will continue to be our guiding light for the next decade. We’ll discuss some of these learnings and best practices below.

Impact measurement: do it for those who matter

When Upaya started conducting primary surveys for impact measurement in 2011, our goals were simple and twofold: to make sure that we were reaching our intended group of the most marginalised people in a given community, and to assess their progress, if any, out of poverty. But as we quickly learned, vulnerability is multi-dimensional, and we had to evaluate a number of indicators for each household (e.g., housing quality, food consumption, educational access, healthcare, hygiene, etc.) to get the fullest picture of their poverty levels and needs. We then decided to return every two years and collect information against the same indicators again, in order to gauge how much progress was made.

From the beginning, we have been guided by jobholder feedback, and we have modified our indicators based on what is important and relevant to them. Had we provided only the reporting required by our funders, we’d be looking at a very limited, and not entirely relevant, set of indicators. Hearing directly from the jobholders of our investees will continue to be the bedrock of Upaya’s journey forward. And it’s great to see the broader industry moving in the same direction, by basing their impact measurement activities not on the upward reporting demands of funders, but rather on an effort to understand the impact of investments on the lives of the communities being served.

Measure quality in addition to quantity

Even a few years back, the general trend in impact reporting was to report large numbers – in the thousands or millions – based on “lives touched” by investments, products and services. For many investors, this data was enough to fulfill their obligation to report on impact. But while reporting on quantity is a good start, it’s also important not to miss the quality of impact.

In Upaya’s case, this means measuring not just the number of jobs we’ve created through our investments in early-stage social enterprises, but also the quality of these jobs. We do this to ensure that these jobs are lifting people out of extreme poverty and easing their vulnerabilities. By assessing how a particular job meets certain critical needs (like increasing the stability of workers’ cash flows), we can understand how these jobs have contributed to poverty alleviation. We can also generate other insights that can boost our impact, by addressing questions like:

• Do we need other interventions besides jobs?
• What sectors and what job types are more effective than others?
• How long does one need to stay in a job to make economic progress?
• Can we make generalizations about job design that can inform other poverty alleviation efforts?
• Where can we find opportunities to collaborate with other programmes, institutions and initiatives?

These are critical questions that we will continue to ask across our investments. Other organizations might aim to improve the quality of their impact in other ways: For instance, an environmental intervention might track long-term health outcomes to validate the extent and quality of the impact they’ve created. Or a digitization initiative in agriculture might go beyond reporting only on the number of farmers they’ve engaged, going deeper to understand how their digitization efforts are helping beneficiaries’ families and communities, and whether they are ensuring that smallholder farms are not left behind. Regardless of the sector or impact goals, moving beyond the question of “how many?” to ask “how are they being affected?” can uncover more meaningful evidence of impact.

Keep entrepreneurs and investors on the same page

We weigh an entrepreneur’s intent to have an impact quite heavily during due diligence. But we have learned that it is important to clarify the entrepreneur’s impact goals up front, and not assume that they view impact with the same lens that we, the investor, do. Our analysis becomes more valuable and actionable when we relate it back to the entrepreneur’s own goals.

Additionally, it takes time and intentional effort to build trust with entrepreneurs, to the point where they let us interview their jobholders independently. And it takes even more effort to establish a feedback loop through which both sides can convey, discuss and deliberate on any concerning impact measurement results. But taking this additional time and effort is well worth it, especially when it results in modifications to enterprises or initiatives that enable them to cater more to the poor and better help them progress out of poverty on their own terms.

Impact measurement needs dedicated resources

Like Frank Sonnenberg, an award-winning author on moral character and values, said, “A good intention is like an idea that you keep to yourself. If you don’t do something with it, it’s like it never existed.” To avoid seeing their impact goals meet this fate, investors must dedicate resources to their impact measurement needs that are aligned with the fund’s overall ambition.

We at Upaya allocate 10% of our operating budget towards impact measurement, rather than being dependent on separate external funding. This is a significant proportion for an early-stage impact fund, and our industry peers are often surprised by the size of our allocated budget towards impact measurement. Most investors do not spend on collecting primary information on the impact created by their portfolio companies, due to the costs involved: Instead, impact information is generally self-reported by investees. Having a dedicated budget has not only enabled us to walk the talk on impact measurement, it has helped us to implement an intentional and independent inquiry into the state of impact created through our portfolio. When impact measurement is funded through restricted external funds, there is always a risk of mission drift, discontinued funding and misaligned expectations with different funders.

Choosing a successful impact measurement strategy is a hard decision in itself. In designing an effective strategy, it’s important to understand which decisions can be better informed with more impact data, and to ensure that there are sufficient available resources to acquire this data. But it’s also essential to keep the strategy aligned with the fund’s overall impact goal. A successful strategy for measuring and managing impact will not only advance the fund’s mission and goals, it will also equip the fund’s team with the information it needs to make decisions. At Upaya, creating dignified and quality jobs for the extremely poor is our North Star – the overarching goal that helps us navigate the myriad impact measurement models and tools available, and make confident choices about which strategy to pursue. What’s your North Star?

Shruti Goel is the Director of Impact Measurement and Management at Upaya Social Ventures; and Sachi Shenoy is Founder of Calidris and Co-Founder and Board Member of Upaya Social Ventures.

Photo courtesy of Kathrine Birch.