As part of our routine reflection and learning, we’ve convened to consider our inroads, failures and persistent gaps in the 18 months since our Midterm Evaluation Report.
What have we learned around adaptive programming; contraceptive continuation; and evaluation methodology? Our thoughts and insights, follow.
In 2016, we launched Adolescents 360 (A360).
With young people at the fore, we delved into designing around the challenges that the adolescent sexual and reproductive health and rights (ASRHR) community had long grappled with. What would it take to make contraception relevant, valuable and accessible for girls aged 15-19 across the developing world?
Through A360’s four programs across three countries:
- We’ve explored, and demonstrated the value of a youth-powered approach that prioritizes and builds the evidence for meaningful youth engagement.
- We’ve shifted the contraceptive conversation to one that leads with girls’ aspirations, and we’ve embraced failures through adaptive programming.
- We’ve seen the fruits of government collaboration lead to tangible outcomes, from expanding the reach of Matasa Matan Arewa in Northern Nigeria to scaling up Smart Start through integration into Ethiopia’s national Health Extension Program.
- And we’ve surpassed our adopter goal – supporting 328,560+ girls aged 15-19 to voluntarily adopt a contraceptive method since implementation began in 2018 through January 2020.
We’ve come a long way since the design phase and our 2018 Midterm Evaluation Report (released in mid-2019). But many of the hard questions remain.
We explore what we’ve learned, where we’ve failed and what challenges lie ahead, below. Want to go deeper? See our full report here.
Q: What single adaptation generated the most impact this year?
In Tanzania, we knew we still needed to address friction-points in girls’ pathway to care. We had young designers as members of our team, but we felt we needed young people on the ground to identify what needed to change, and how.
“We on-boarded a ‘SWAT’ team comprised of young people from the communities in which we work who serve as our eyes during events. The team leverages their youth perspective to share, in real-time, the challenges girls face and, together with programmers, identify potential solutions.
For example, earlier iterations of Kuwa Mjanja didn’t include activities to engage girls after meeting with a provider. We knew that girls needed more support to understand side effects to continue contraceptive use while in need. In collaboration with the SWAT team, we designed a digital game for girls to play to test their knowledge and build their confidence in understanding any side effects that they may experience.”
– Edwin Mtei, A360 Tanzania Project Director, PSI Tanzania
Q: What adaptation failed? What did we learn from the experience?
We understand that failures don’t mean that we’ve failed; they’re merely indicators for what needs to change for the benefit of girls’ and the health systems’ needs.
Through Adaptive Implementation, we fail fast and get up and move forward, just as quickly.
“In Southern Nigeria, we piloted 9ja Girls by offering beautifully branded, girls-only service delivery rooms in public health facilities. It was delightful; but it wasn’t cost-effective nor sustainable, especially as we progressed toward government integration.”
It was a fail.
“So, we tested 9ja Girls with limited branding, keeping only the 9ja Girls mantra and logo. We learned that our target audience—unmarried girls aged 15-19—care less about what the room looks like and more that the providers are youth-friendly.”
– Fifi Oluwatoyin Ogbondeminu, A360 Nigeria Service Delivery Specialist, SFH/Nigeria
In Tanzania, we wanted to build our outreach teams’ capacity to mobilize girls in a youth-friendly way.
“We first engaged young designers to train the outreach team on how to make their mobilization youth-friendly. They stayed for a week, but it wasn’t enough. In response, our SWAT teams step in after the young designers leave to provide continuous support to outreach teams.”
– Edwin Mtei, A360 Tanzania Project Director, PSI Tanzania
Q: What have we learned about mobilizing adolescent girls?
In Northern Nigeria, we’ve learned that engaging influencers is key, the gender of mobilizers are important and the location, timing and approach in which we mobilize matters.
“There is high demand for ASRHR information, support and services, even across northern Nigerian settings.
That is: only when the entry point and packaging is right. From community leaders to husbands, mothers-in-law and co-wives, mobilizers reach those that girls identify as most critical to their ability to engage with us and garner their support to enable married girls to gain the skills and know-how to contribute to their families’ health and stability. Given many married girls live in polygamous households, an inclusive approach is critical.
We, therefore, adapted programming to support co-wives that girls live with to have access to SRH services, too. In fact, many of them want the full mentorship program we’ve designed for girls. That presents an opportunity for the future!”
– Fatima Muhammad, A360 Nigeria Project Director, SFH/Nigeria
Q: What is the top insight gained this year on discontinuation?
Across all A360 countries, contraceptive continuation is still an area ripe for improvement.
“High-quality services and constant contact improves continuation; with girls’ consent, providers follow up with a girl three days after her visit, and 48 hours prior to her next visit to ensure girls have the support they need to continue their method of choice.”
– Fatima Muhammad, A360 Nigeria Project Director, SFH/Nigeria
But as the data shows, even when we deliver highest-quality clinical services and girls are thoroughly supported to understand and manage side effects, social influences can still trigger girls to discontinue.
“In Ethiopia, many married rural girls who discontinued shared that their husbands encouraged discontinuation when side effects appeared despite having been supportive of contraceptive uptake at the start.”
– Metsehate Ayenekulu, A360 Ethiopia Project Director, PSI Ethiopia
Our takeaway? We will keep exploring how we can address barriers to contraceptive continuation. That’s our focus as we move forward.
Q: What have we learned about evaluating innovation?
As a design-based project, A360 was committed to innovation from the start. But we didn’t fully manage to get our accountability and performance management approaches to reflect it.
“We now know that monitoring and evaluation systems should not be finalized until the design period has ended, so that teams can build monitoring indicators that reflect the full breadth and value of the interventions that teams have designed. And we understand that process indicators are far more relevant and valuable as a performance ‘metric’ during the design phase.
Plus, evaluation methods work best when they match an intervention’s technical design. That, too, is best done after the design phase, when the interventions can be fully understood and so that evaluations fully speak to them.”
– Claire Cole, Sr. Sexual and Reproductive Health Advisor, PSI
“Human Centered Design (HCD) creates a culture of prototyping, which primed us to get comfortable with failing fast… and moving on, openly. An adaptive approach to implementation was a natural, and complementary fit.
Adaptive programming fuels continuous improvement. This has been evidenced in the progress the project has made since the MTE in mid-2018. Inevitably there are still challenges we wrestle with, like better understanding and supporting contraceptive continuation and integrating into health systems. But we’re confident adaptive implementation, a culture of failing fast and meaningful youth engagement will enable us to design and deliver sustainable and scalable solutions to these intractable challenges.”
– Matthew Wilson, A360 Program Director, PSI
