Comparing two rape prevention programs

Here is a summary of all 157 stories that mentioned rape in Kenya & Uganda:

Two rape prevention programs stand out from this overall group: Mrembo program and Sita Kimya (which I discussed previously).

The word clouds of these stories reveal two very different perspectives. First, Mrembo:

mrembo stories.doc

And second, Sita Kimya (a USAID funded effort to change behavior in men around their treatment of women in the Nairobi slum of Kibera):

sita kimya 23

Differences:

  • Whose stories? The stories about Mrembo program are told almost exclusively by young girls under 16; Nearly all the stories about Sita Kimya are told by men, ages 16-30.
  • Where? Kamukunji for Mrembo, Kibera for Sita Kimya
  • [GIRL] Mrembo girls talk about sexual help from a first person perspective. Notice how the word girl/girls doesn’t appear in their tag cloud, whereas it dominates both the other groups.
  • [HIV/AIDS] Mrembo stories emphasize the relationship between risky sexual behavior (like hanging out with boys on soccer fields, according to some stories) and AIDS. HIV/AIDS is hardly mentioned in Sita Kimya, relative to frequency of being mentioned in the overall rape stories set.
  • GOOD NEWS: Both the Sita Kimya and Mrembo stories are drawn from their target audiences: Young sexually-active men and younger pre-sexually-active girls, respectively.
  • Mrembo girls choose “Inspiring” or “Important” to describe their stories, and rarely “Horrible” or “Not memorable.” Sita Kimya guys choose all of these labels about equally.
  • Mrembo girls are almost all “actors” or “affected by” the events in their stories. Nearly all the guys are “observers” in these stories, but not “actors” or “affected by.”
  • Both groups considered their story more related to a specific organization, over family, friends, or a community.
  • Mrembo girls mostly talked about a recent event from last 6 months, whereas Sita Kimya guys talked about more distant events ( no stories from the last 3 months)
  • Both groups share stories that reflect success and failure, but Mrembo girls are a bit more likely to think of their story as somewhere in between:
  • Lastly, the English translations of both Kiswahili words do not appear in stories: “Stop” or “Silent” for stopping rape: Sita Kimya means “we will not be silent!” or “Beautiful” for Mrembo means “beautiful” [girls].

These groups of stories were analyzed using SenseMaker(R) from Cognitive-Edge.Com

[Addendum: Download: Links to 87 Stories about mrembo on GlobalGiving]

Looking at wordle subtraction:

This version is a composite of 87 stories about mrembo, about twice the number used before:

And here are an astounding 258 stories about Sita Kimya, funded by USAID:

I figured out how to subtract one wordle from the other, to really sharpen the contrast and the overlap between these two story collections.

Words from Mrembo stories, with Sita Kimya words subtracted out:

Words from Sita Kimya stories, with Mrembo words subtracted out:

Words that they both share:

Words that appear in either group, but not both:

[Also known as the symmetric difference of two sets] This one is the most difficult to interpret:

Note that I removed the words Sita Kimya, Mrembo, people, cases from the above set, since they were obviously dominating.

Conclusions

  • Mrembo program emphasizes teaching girls about HIV/AIDS, early marriage, and avoiding bad situations so that one avoids rape. Stories are about what the girls in the program learned.
  • Sita Kimya stories emphasize rape cases involving girls from a man’s point of view. Schools, community, and organization appear to be a common part of the stories. Seemingly esoteric topics like books, water, and life also get mentioned.
  • Most astounding: When Sita Kimya men talk about rape, the idea of early marriage and HIV/AIDS is absent.

Update using a wordtree comparison method

Here is a map of all the words from stories that mention Sita Kimya (in blue) and Mrembo (in red). The purple nodes are words that are shared between both stories about equally. It underscores just how different these two sets of stories are, in terms of what the storytellers emphasize — and might be easier to quickly read than the wordle method:

 

Mrembo

  • HIV and AIDS
  • Parents, pregnancy, marriage, avoid, and body

Sita Kimya

  • Police, hospital
  • know, men

Both

  • Rape, sex, girl
Advertisements

9 thoughts on “Comparing two rape prevention programs

  1. This is really interesting. Could you maybe share the objectives of both projects so that you are comparing like for like. Sita kimya for example was not addressing early marriage and sexual violence manifested itself in other more prominent ways like intimate partner violence, child defilement etc. Also do you have any information on the duration of the two projects. Sita Kimya for example was implemented in under a year. Lastly, how did you gather your data Sita kimya for example segmented beneficiaries into cohorts- males 15-24 (Morio) Females 15-24 ( Sella’s) Males 25+ (Oyoo) and Females 25+. The project actually reached more females than males.

    I look forward to your response

  2. Re: my comment above its important to look at the objectives to determine the success of the intervention. Did Mrembo achieve an increase in the uptake of Post rape care services? was it one of their objective at all? well sita kimya did.. and it was one of its objectives

  3. Michael, while the program objectives are important to program managers, these stories are from people who know nothing about the intentions of program managers. They speak of what they know – which means that the topics mentions are a better reflection of the program’s actual flavor than any document which set out to define the objectives. We did not collect data on either program specifically, but received stories organically from people who chose that program to talk about.

    The fact that both Mrembo and Sita Kimya are talked about in hundreds of stories is evidence that both projects are reaching groups of people in their respective locations.

    As shown in the post, storytellers describing Sita Kimya were almost exclusively male. and both groups are present (15-24 and 24+ aged males.

    You can now do this analysis yourself using the online tool: http://www.globalgivingcommunity.com/diff.html

  4. Information about the Mrembo project is here:
    http://www.globalgiving.org/projects/improving-lives-of-200-girls-in-majengo-slums/

    And they worked with girls ages 12 to 16 in Majengo to prepare them to AVOID rape, so trying to increase the use of post-rape services would be an irrelevant goal, in fact it’s a form of failure – as girls would still be getting raped in that case. They also talked about a broad spectrum of female adolescent issues in their weekly school sessions.

    They did not perform an RCT or formal evaluation as the whole budget for the project was around $1000-2000. So if we were really comparing like-for-like as you suggest, we would also have to adjust for the cost of each intervention. This approach to evaluation is about getting reasonable conclusions at a reasonable cost (free). If you want to control for budget size, duration, sub-populations, and programmatic objectives in your evaluation you will need to put a lot more money into the program, more than the total cost of mrembo – which is why so many smaller interventions remain completely ignored and unanalyzed by larger NGOs like USAID (who funded Sita Kimya). Learning should not be expensive, even if what you can conclude is not rigorously defensible – but aggregate patterns of many pseudo-controlled trials is likely to be more reproducible and correct than any one expensive “standard” evaluation.

    I also think there is a HUGE improvement to cross-analysis consistency when the data comes from storytellers who are not prompted to talk about specific projects, such as is commonly done in formal evaluations. This post shows how to detect that bias in our story samples:
    https://chewychunks.wordpress.com/2012/05/21/the-automatic-self-bias-detector/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s