From survey questions to prescriptive answers

Last week I visited the University of Limpopo in South Africa to help design a survey that would reveal the underlying factors that lead to HIV discrimination or resilience in the face of stigma.

My belief is that within thousands of stories that address discrimination, certain elements in those stories are going to be associated with poor outcomes – intractable HIV stigma forced upon a person – and other story elements will correlate with resilience (where the story resolves to give the HIV patient more control and respect). Here is how I read a few questions that codify the context and use them to construct statements that can prescribe possible interventions – strategies to strengthening resilience in HIV communities.

Codifying the context of social stigma

Q: The person mostly responsible for the stigma was
An authority figure
A family member
An educator
A religious figure
A health worker
A CBO volunteer
A vendor
A friend
A stranger
An acquaintance

Q: What was the response to the stigma?
The person fought back / confronted it immediately on their own
A family effort was organised to fight it / confront it
An effort by friends was organised to fight it / confront it
A community effort was organised to fight it / confront it
They avoided the source of the problem on their own
They avoided the source of the problem with help from family and friends
They tried to ignore it
Did nothing [just accepted it]

Q: The place where the story happened was a
Health centre
Traditional Healer’s house
Prophet’s house or church
Social gathering

Q: Your role in the story
I experienced it
I witnessed it
Neither of the above

These four multiple choice questions (the perpetrator, the response, the location, and your perspective) encode a wide range of possible factors in discrimination. Conservatively, 640 possible combinations if a person’s role in the story is not really important. But possibly 1920 comtinations if it is (8 X 8 X 10 X 3 = 1920). The “neither of the above” answer for your role is not ideal, but we thought that perpetrators who told stories would be much more likely to choose that over “I stigmatized the person” or something direct. Getting the perpetrator’s perspective is difficult.

Still, 640 seems to not be enough. The latest draft added another 5 questions that are similar to these, that I’m not discussing because it is far less clear to me what they define.

The other half of this survey uses triads and polarity questions from the SenseMaker(R) “signification framework” method promoted by Cognitive Edge.

Analysis: Parsing stories to construct Prescriptive Sentences

These multiple choice questions are used as filters to subdivide groups of stories, then look to see if the subgroup is strongly associated with positive or negative outcomes.

These three filters parse stories into either positive (remove stigma) or negative (sustaining stigma) outcomes:

Stories that fight stigma

These involve communities trying to prevent it, encouraging those affected to live openly with HIV or whatever characteristics define the stigma, and are typically about life-changing events for the person affected.

Stories that propagate stigma do the opposite – and will have dots in the opposite corners.

One other triad borrowed from the GlobalGiving Storytelling Project also allows us to understand the nature of the stigma in a more abstract way, that may be important for designing an intervention later:

And this question about the storyteller’s attitude towards the person affected in the story fascinates me:

Analyzing stories from people who think the person deserved it will finally yield the perspective of the perpetrator. I doubt this will have a lot of gray area points, but we don’t know yet.

Results in a sentence

It is possible for a particular combination of answers to appear 3-10 times more often than all other combinations. For example when stigma takes place in a school and an authority figure is the perpretrator and the person’s response is to Avoid the source of the problem by not going to school – these events are always life changing and lead the person to live in the community out of sight. That’s one potential strongly negative combination of the 640 possible ones.

Or alternatively, perhaps we’ll learn that when a person tells a story they experienced about stigma in school from a friend and a family effort was organized to confront it, these stories are life changing but in a positive way. I’ve just described bullying, but the same could apply when it is about HIV.

Prescriptive, not descriptive or conclusive

That’s what I call ‘prescriptive results‘ instead of ‘descriptive’ or ‘conclusive’. Demographers love descriptive data, but I don’t find most of them to be helpful when designing an intervention. What’s more important? That the storyteller makes $20,000 / year and not $35,000, or that they experienced the story and it was life changing? Yeah, thought so.

A randomized control trial can conclude that an effect was strong or weak not non-existent, but a ‘prescriptive’ experiment is designed to reduce hundreds of possible permutations of 4 factors until perhaps just 4 strong combinations emerge. These 4 should be the focus of interventions and additional follow-up research, including RTCs. But we should start with prescriptive data before jumping into a slow and expensive RCT.

Lastly – cognitive overload is a serious problem with all surveys! I think these 9 questions plus the actual story are sufficient to achieve the goal, but 99 out of 100 people who design surveys would be terrified of not including enough questions. We’ve got to stop acting like control freaks with other people’s time! That means shorter surveys and much more re-use of data – the way google is starting to do (whether you gave them permission or not).

Instant preliminary data!

Because GlobalGiving has already collected nearly 40,000 stories about a broad spectrum of community efforts, I was able to extract 546 stories that are exactly the kind this survey was designed to gather. From these, we already know that 70% are about HIV discrimination in East Africa. Less than 20% are about gender discrimination – so we can be quite vague in the story prompting question and still get a lot of HIV focus, which is what the group wants.

Example stories:

  • The story is about a person who was neglected by her sister because she was a HIV positive ann a mother of two is now helpless because her sister wairimu could not live with her because her sister wairimu could not live with ner sister wairimu  could helpless because she was not able to provide her with basic needs and the medical expenses.her daughter who is in form 4 was later sent home due to lack of school fees. the woman looked devastated and cannot even afford a daily meal.the situation was so bad that she needs an urgent help.|
  • Last year saw those affected and infected by HIV/AIDs being visited by certain organization named by Aphia II.This NGO non governmental organization whose aim is to deal with health issues of the people and moreso they have involved themeselves in dealing with HIV/AIDs scourge.As witnessed, those suffering from this disease in my area are poor and had been dying just because of lack of meditation and proper care.Stigma did also contribute to these deaths. In a way of helping them, APHIA II brought basic necessities for them like food , clothing and other physical needs. They went ahead to educated the community on how to treat the victims and further advised not to treat them badly as this could contribute to social stigma.They also provided them with A>R>Vs and they are still doing the same.Atleast the victims of HIV?AIDs have hope and futhermore, something to.|
  • About three weeks ago, there was a problem from one of our pupils. The pupil was a girl. Her father was dead and her mother HIV positive. The girl did not have enough money to purchase for her education. That matter gave her a hard task. Her mother was not able to do anything. For she was the only one left, the girl tried to help she would not come to school as usual. She dropped tremendously from her marks.  All of the pupils in school knew about this but they could not do anything. We tried tooth and nail but our effort ran to a wild goose. We donated some money that could help the girl child. The money helped her at home and made life more easier. She could now help her mother in medical services.  After a while, her mother kicked the bucket. The girl was stranded. She did not have someone to help her. We thought about that and we decided to have a Harambee meeting. We donated some money in which the counciloor gave out two million.  It was such a pleasure. The girl went on with education and passed.|

If you want to know more – contact Chris Burman at the University of Limpopo. Write your name in a comment below and I’m sure he’ll be happy to share the survey and his insights.

8 thoughts on “From survey questions to prescriptive answers

  1. I will continue to visit this post for future reference for my dissertation, Marc. Question: How did you code it when respondent’s said their response to stigma was to pray, or as my subjects word it, “tell it to God?” I think that could include all the family, friends, or community options you listed. There must be an analytic difference between praying at home (the private sphere) versus “carrying it to church” (the public/community sphere). I have that as a response quite a bit when I ask Nigerian communities how they respond to environmental damage. Conceptually, do you think of that as being an active response or a passive one?

  2. We haven’t tested that – but I imagine we could add “pray and hope for divine intervention” or something like that to the question. Most likely those people would choose “ignore it” or “accept it” – but you’re right, neither captures what the person really means… which is that they are acting out, just in a way that we haven’t anticipated.

    In my opinion, prayer does change things when done in the presence of others, who might hear those prayers in a different light than if they were voiced through a secular channel.

    I think praying in private is passive, whereas praying where others are present is active. It falls into community organization response – in my view.

    Chris? Suggestions? add “prayed about it” to the list?

  3. Laine – you might notice that this survey differs from the version you sent me about people speaking to power. Note that the who and where of the story are coded as distinct category answers, but the definition of whether the events were positive or negative are fuzzier and ambiguous.

    Your version had ambiguity in where / how people ascribe responsibility to a problem, but the problem was very distinctly defined, right?

  4. Yes, my version has ambiguity in how people ascribe responsibility but no, there is no ambiguity in the where. The where and the problem are already defined because the oil crisis in that village is what brought me there in the first place. I am asking about the response and ‘your role in the story’ mainly, while the issue of perpetrator is important too but secondary. All of my events would be considered to be negative, e.g. oil spills. I am off to a new village tomorrow so I will let know how it goes.

  5. Prayer – is a very useful thought that we did not think of. i will run it past my colleagues today. Thank you for the suggestion.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s