Rigorously evaluating cash transfer programs in the United States: Considerations, challenges, and future research questions

Cash transfer interventions—cash payments that can be one-time or recurring, conditional or unconditional, and direct or indirect—are gaining more attention worldwide. In September 2023, researchers conducting randomized evaluations of cash transfer programs in the United States gathered at Duke University to discuss their ongoing and completed research projects. They shared common challenges regarding measuring outcomes, implementing and designing cash transfer programs, and communicating about results to the public. In this post, we highlight key takeaways from these conversations.

Due to their demonstrated effectiveness in low- and middle-income countries, cash transfer programs have received growing attention from researchers and policymakers in the United States. Several dozen pilots or studies have emerged nationwide to assess the impacts of these programs. Some fall under a broad endeavor to understand the effects of guaranteed income (e.g., The Cook County Promise Guaranteed Income Pilot), while others deploy cash as an intervention within existing safety net systems or as an approach to understand the impacts of poverty reduction (e.g., Baby's First Years). While each study employs a different design (e.g., one-time versus recurring payment) and target population, numerous lessons can be learned by looking across this body of work domestically and abroad. 

Understanding ongoing and completed studies in relation to each other and to the broader landscape is key to building a robust body of rigorous, policy-relevant evidence. With the support of Duke University Population Research Institute (DUPRI), I (Lisa) convened a small group of multidisciplinary scholars conducting ongoing randomized evaluations of cash transfers in the United States to share challenges and emerging lessons and findings. Both the cash transfer programs and the evaluations varied widely. Some administered one-time payments while others gave recipients anywhere from $333 to $1000 per month for nine to fifty-two months. 

Throughout the one-day workshop, scholars discussed various themes and common challenges from their work and similar studies from both the United States and abroad. 

Selecting and measuring appropriate outcomes

Unconditional cash and its inherent flexibility can theoretically change many aspects of people’s lives. However, researchers cannot feasibly measure every possible outcome and therefore need to make difficult tradeoffs about what to measure and how. The scholars at the convening spoke about the difficulty of deciding what to measure; needing to balance competing priorities between minimizing burdens on participants, scientific rigor and integrity, and the interests of policymakers and practitioners. For example, many lawmakers are interested in whether cash transfers reduce employment or increase purchases of “temptation” goods, such as alcohol and cigarettes. However, other stakeholders have raised concerns about the potential moral implications of monitoring how people spend money and perpetuating stereotypes—especially racial stereotypes—of individuals with low incomes. 

In addition to the challenges of deciding outcomes of interest, researchers discussed limitations of and opportunities for how to measure their chosen outcomes. For example, what are the merits and drawbacks of objective versus subjective measures? One study found that while self-reported (subjective) mental health did not improve, mental health-related emergency room visits as measured through administrative data (objective) decreased for those who received cash. 

Researchers discussed how community members, including participants themselves, can help inform outcomes. For example, qualitative data can be a helpful tool for determining both what to measure and how to measure it. Most of the research teams represented at the convening incorporated focus groups, interviews, or open-response survey questions into their research designs to integrate the voices of recipients as well as community leaders. One asked study participants to rank how they planned to spend their money and then used this information to guide researchers’ decisions about which outcomes to measure. Qualitative data and community input serve as tools for identifying and operationalizing outcomes. They can shed light on the mechanisms behind certain findings and what types of information are most useful for policymakers and practitioners. 

Designing cash transfer programs

The way that these programs are designed may have an impact on the way recipients spend the money. Examples of design choices include lump sums versus small installments; debit cards versus bank account deposits; whether cash is being disbursed by a private or public source; and whether programs suggest the cash be used for a specific purpose, such as food or housing.

While many researchers were interested in testing these various design choices and their impacts, each of the researchers in attendance had various levels of control and involvement in shaping the design of the cash support. Some studied programs designed and administered by a government (e.g. Compton Pledge) and therefore had little input into the program’s mechanics. Others, such as the Basic Income Project or the COVID-19 Cash Transfer Studies, were designed and implemented directly by the researchers. For every evaluation, program designs were determined by balancing competing priorities from communities, partners, and research teams.

Sharing results and the current policy landscape

Finally, researchers shared concerns over how to discuss findings in an evolving political and economic landscape in the United States. Politicians and policymakers have been increasingly invested in cash transfer pilots, as exemplified by organizations such as Mayors for Guaranteed Income. However, there is also skepticism. Because initial evidence from some of the studies suggests null impacts for one-time cash payments, evidence should be situated into the broader policy debate with particular care and attention—especially in preventing the proliferation of reductive narratives. 

Attendees discussed the importance of being clear about what researchers learned from each study—what they know and what they do not know. For example, many of these studies took place during Covid-19, when people were receiving other stimulus support. While the benefit of randomized evaluations is that they ensure similarities across treatment and control groups, the effect of the cash may fluctuate under naturally occurring changes in the economy. Other outcomes, such as formal employment, may also be affected by the pandemic due to school and childcare closures or workplace conditions. Because communicating about these results requires nuance, researchers also discussed and emphasized the importance of building partnerships and trust with implementing partners and government to do this well. While controlling the political narrative may be unrealistic, researchers can play an important role by being clear and precise in contextualizing the evidence. 

The future of research on cash transfers 

Throughout the day, researchers discussed questions for future investigation. For example: 

• What is the impact of providing cash to entire communities (as opposed to only a subset of individuals in a community)?
• What is the impact of cash transfers compared to other interventions of the same cost? (A similar evaluation was conducted in Rwanda). 
• How can the impact of cash transfers be understood in the context of social safety net programs, like SNAP? 
• How can holistic assessments of impacts from cash transfers be considered given differences in context, populations, and design features? 

The opportunity to learn from each other in a small, informal setting and from both domestic and international scholarship proved to be both productive and cathartic: scholars expressed interest in ongoing conversations to continue in this community of shared learning.

As this body of research continues to grow, we hope researchers in the J-PAL North America community and beyond will continue to work together, discuss challenges, and share best practices in the interest of building collective insights. Many of these research teams expect to have publications in the spring and summer of 2024. To stay up to date on study results and ongoing discussion, make sure to subscribe to the J-PAL North America newsletter. 

In an accompanying post, Paul Niehaus shares his perspective on studies of cash transfers in low- and middle-income countries and on how the United States can learn from other contexts.