At Development Works Changemakers (DWC) we have a passion for social change, and this is seen in our number of case studies and successful evaluations. In April 2019 we started working with the Western Cape Department of Health (WCG: Health) and Desmond Tutu HIV Foundation (DTHF) to provide design and implementation evaluation in the health and social development sector. The project continued until August 2019 and was funded by Global Fund to Fight AIDS, Tuberculosis and Malaria (GF).
The Cash plus Care programme is a component of the Western Cape Department of Health’s (WCG Health) Young Women and Girls Programme (YW&G), funded by the Global Fund (GF). This programme was implemented in two neighbouring sub-districts in Cape Town, Mitchells Plain and Klipfontein.
The aims of the YW&G programme were to: decrease new HIV infections in girls and young women (aged 15-24); decrease teenage pregnancies; keep girls in school until completion of grade 12, and increase economic opportunities for young people through empowerment. The objectives of the intervention were to: enhance the sexual and reproductive health of participants through preventative and promotive health and psychosocial interventions whilst enhancing their meaningful transition to adulthood; and to reduce HIV and STI incidence, unintended pregnancy and gender-based violence amongst Cape Town women in late adolescence.
The programme, also called “Women of Worth” (WoW), provided 12 weekly empowerment sessions and access to youth-friendly healthcare, seeking to address a range of biomedical, socio-behavioural, structural and economic vulnerabilities amongst participating young women. Cash incentives of R300 per session were provided to the young women for participation in empowerment sessions. Implementation of the programme was sub-contracted to the Desmond Tutu HIV Foundation (DTHF) by the Western Cape Department of Health in November 2016, and implementation of the programme began in April 2017.
DWC was contracted by the WCG: Health to conduct an evaluation to assess the Cash plus Care programme’s design and implementation. Specifically, the evaluation focussed on the appropriateness of the programme design, incentives, and recruitment processes; beneficiary and stakeholder satisfaction; and the quality of implementation. The evaluation also identified successes and challenges/barriers and made recommendations to the Global Fund and WCG: Health to inform the design and implementation of future programmes.
Project deliverables were:
- A theory of change workshop with all key stakeholders from WCG: Health and the DTHF
- A theory of change diagram showing all key contextual factors, assumptions, inputs, outputs and outcomes
- A draft evaluation report
- A final evaluation report
- A final report in 1:5:25 format
This evaluation was formative and clarificatory aimed primarily at learning and improvement, but including some elements of early impact assessment (particularly unintended consequences), and summative aspects that informed decision-making on the future of the programme and similar programmes.
The evaluation adopted a mixed-methods evaluation design, which used mostly qualitative data. Existing quantitative data was drawn on where appropriate from the various programme and other documents reviewed.
Both primary and secondary qualitative data were gathered and analysed to answer the evaluation questions. This evaluation, which was essentially a rapid assessment, relied on the design strength produced by a mixed-methods approach which allows for triangulation of method, observers and measures. Given that the programme experienced several changes in its initial design, the developmental approach followed emphasised learning and improvement.
Secondary data was obtained from project documents, while primary data was obtained from the following sources:
- A Theory of Change workshop with a large range of stakeholders;
- Key Informant Interviews with WCG Health and DTHF staff (16 interviews);
- One day site visits to five of the 11 operating empowerment session venues;
- During the site visit, interviews with the site facilitator, at least one FGD with current participants at the site, and one-on-one interviews with any other participants, where targeted. Empowerment sessions were also observed and key aspects were noted on an observation tool;
- Telephonic interviews with previous graduates who were “cash-no”;
- Telephonic interviews with programme dropouts, both “cash-yes” and “cash-no”.
In all, interviews and FGDs involving 73 beneficiaries were conducted.
The value of this design and implementation evaluation was firstly that it provided the WCG: Health and the DTHF with an independent examination of the complexities of implementing the Cash Plus Care programme in the Western Cape. A number of challenges had been experienced by the programme implementers since its inception. Many of these related to various delays with contracting the sub-recipient (DTHF) and the subsequent rush to catch up by the sub-recipient on various aspects of the programme. Another key challenge was that this programme was both an intervention (with the above-stated aims) and an academic random control trial, designed to inform the GF of the efficacy of using cash incentives to change behaviour around risky behaviour.
Many of the design and implementation challenges had to do with trying to align the needs of a random control trial with the realities of implementing an empowerment programme with young women living in a complex socio-economic setting. The fact that half of the targeted participants were randomly allocated to the “cash-yes” group, while half were allocated to the “cash-no” control group caused numerous problems.
For a start, those in the control group quickly learnt that they were not receiving cash and many then dropped out. Recruitment of young women for the study was also not effective at the beginning of the programme, which meant it struggled to reach its targeted numbers in time. Only once cash incentives were made available to all participants and a proper community mobilisation team was put in place did the numbers pick up and the programme was able to reach its goal. This evaluation brought to light many of these issues and made recommendations for how to mitigate them in future such programmes.
The delayed commencement of the programme and subsequent rush also resulted in the biometric system which was being used to manage the participation and incentive system not being properly ready. Many glitches were experienced which had to be corrected and mitigated along the way. This evaluation helped to document these problems and the ways in which they were solved.
The evaluation also brought to light the experiences of participants and the value they felt the programme had brought to their lives. It showed some emerging elements of empowerment and behaviour change, as well as new forms of social cohesion forming between attendees. The many recommendations made, based on these findings, were of great value to the programme implementers and funders, who received the evaluation very positively.
Development Works Changemakers Evaluation
Over the next couple of months, we’ll be showcasing more of our case studies and highlighting the various methods of our approach.
To stay up to date with industry news and happenings, you can sign up to our newsletter and follow us on social media.