How to conduct a storytelling evaluation (Part 2 of 2: build a form)

Refer back to Part 1: Analyze stories here

Our process as a series of 9 steps. This tutorial covers the third step:
(1)    Analyze existing data — use the story search and story phrase bubbles tools
(2)    Form a hypothesis — based on what you see
(3)    Design a different kind of survey — story form builder
(4)    Collect more data — attract a group of young people, train them, given to papers, and send them out
(5)    Compare your new data against the existing data — in the same analysis tools as step (1)
(6)    Form a conclusion
(7)    Define the problem — informed by a new perspective
(8)    Change project direction, apply for funding, etc.
(9)    Go to step 3; repeat.

Go to GlobalGiving.org, log into your project leader account, then click on the ‘stories’ button on your dashboard

If you are not currently a project leader you can use this secret link: http://www.globalgiving.org/dy/v2/storyInput/info to collect stories, but you should join officially. The GlobalGiving Organization application page is at http://www.globalgiving.org/non-profits/join-globalgiving/

Read the stories page overview

Understand the “scribes” model for collecting stories before you build your form

(1) Engage a group of local people. We find that girls who have just graduated from high school do the best work, and provide you with a broad sample of both men and women, both young and old.
(2) Invite them to a 1 hour training. You want to end up with about a dozen, but you may have to invite 20 or 30 as some will not be interested.
(3) Have them fill out the one-page story form you are building during the training.
(4) Give them blank printed story forms and a goal: interview 10 people in the next 2 weeks. Ask each one for 2 stories. And return all the papers.
(5) Provide them with an incentive to participate – this can be a small payment (in East Africa we found that 10 cents per story was sufficient) or a privilege, or additional skills training.
(6) Collect the stories and scan them using an Android camera phone or iPhone.
(7) Analyze, and share your findings with the scribes. They will want to know what the community is talking about. Then repeat this every 6-12 months.

Customize your own storytelling form. The first questions are selected for you. They are required.

You will see that two of the required questions are scribe’s phone number / email address and the storyteller’s phone number / email address. These will be kept confidential and are used for tracking how many people are actually filling out the forms, and which stories were told which whom. We prefer that you tell your scribes to choose one kind of data (phone OP email) and ask for that one kind of information for everybody.
If somebody (i.e. a scribe in training) tells his or her own story and there is no scribe involved to write it or collect it, then put the same phone number in both boxes.

The next questions, in yellow, are optional. You may add them to your form by clicking on the checkboxes to the right.

Which questions you choose is up to you. It will NOT allow you to add more than front and back of one page worth of questions.

Each of the optional questions are best used with one or more other questions in the set, depending on the kind of stories that interest your organization.

The choices may seem overwhelming at first, so below are a few examples of an organization with a specific goal and which questions they chose.

Example: Story-based program monitoring for a girls’ after school program

Vijana Amani Pamoja runs the mrembo project in Kamukunji, a Nairobi slum. Mrembo (“beautiful girl” in swahili) aims to give adolescent girls ages 8-15 life skills for deal with boys and other threats in their daily lives. This organization collects stories from girls before and after each 10-week session to keep track of what issues are coming up in their girls’ lives that need to be addressed in future sessions.

I’ve highlighted the optional questions in red. This organization chose to focus on the who, what, and why of the story, along with two questions about power and hierachy.

Example: VSO’s Valuing Volunteering community research project questionnaire

Two VSO volunteers (Jody in Philipinnes and Simon in Kenya) are gathering community feedback about what would make volunteerism more effective in developing countries. They’ve included a lot more of the conflict and power mapping questions, and some less structured open-text questions, like “what else would have made a difference in this story?”

Even though few of these questions overlap with VAP, both will be able to benchmark their answers against the other on these questions and story elements:
1. All words in the actual story — you build a set of relevant stories by searching the text first.
2. topics
3. age
4. sex
5. role
6. authority figures
7. why it happened
8. who benefitted (outcome)
9. types of solutions

That’s nine different criteria even though these two groups have never coordinated on which questions they’ll ask. And there are many more people collecting, so that everyone will eventually have comparable data for every question they could ask.

Choose your questions and submit the form.

You can mouse over the tooltip (?) symbol for more information about the context for each question.

Your form is ready – check your email for instructions

The email will contain unique URLs to your specific story form. There are two links. One is a web form where you can enter data. Our free transcription service will use this when you scan your stories with a camera phone and email them to us (marcmaxson@globalgiving.org or projecthelp@globalgiving.org). The other link is a printable PDF that you should print and photocopy many times so that scribes can collect stories anywhere without needing any technology.

Scanning stories with a smart phone

We have an instructional video to demonstrate this. Basically, there are many android and iPhone apps that do repetitive document scanning faster than a conventional scanner. We recommend you download one of these:

Scan to PDF (free on android)
Genius Scan (free on iPhone, $2 on android)

These apps will create a PDF and attach it to a gmail message for you. If you address it to us at GlobalGiving, we will do the rest and email you back when your group of stories is available online for analysis (at djotjog.com/search/ and djotjog.com/bubbles/)

You can also read a longer description of the method here: Download the 2012 RealBook