This site is not fully supported by Internet Explorer. To fully enjoy this website, please use an alternative browser

Paper was invented in 206 BC….It’s time for a change: Introducing CAPI
Survey design and sampling

(This blog first appeared on the UK Data Service's Data Impact Blog on 24 November 2105)

I recently attended a Global He about the experience of using technology (specifically ACASI) to research violence against adolescent girls in humanitarian situations. During the session, IRC and the Population Council hosted a discussion on the use of technology within an adolescent-girl protection research programme and innovative ways of collecting data in Ethiopia's refugee camps by COMPASS, a research centre, like Young Lives, based in the University of Oxford.

The seminar prompted me to reflect on our own processes and considerations for migrating from administering our questionnaires using paper and pen to using tablets for data collection.

Can CAPI (computer-assisted personal interviewing) solve all your problems?

The simple answer is yes, computer-assisted personal interviewing can solve all your current problems but it will give you some new ones to think about as well! In 2013, Round 4 of our longitudinal survey, we started using tablets to collect our data in the field. Gone were the paper and pens of yore and in were the stylus pens and screens and with this, a myriad of considerations and trade-offs. So here are some reflections on introducing CAPI to teams across four varying countries:

Weighing the benefits of CAPI versus PAPI

Our old process of collecting the data via PAPI (Paper and Pen Interviewing) provided security in that you always had a written record of the respondents answers that you could reference. The problem with PAPI is that it requires a lot of time at the back end building and entering data into a database and then cleaning it for use in analysis. At this point you are hundreds of miles away from your respondents, and months down the road from the interview, so any inconsistencies needed to be checked using the memory of the fieldworkers and supervisors. With CAPI you have the luxury of preloaded data (i.e. household rosters), consistency checks built in to the programme, validation checks and immediate access for preliminary analysis and key findings; as well as saving time in the field by building in skip patterns to the questionnaire program. This resulted in a much cleaner data set that was available shortly after collection.

Introducing Fieldworkers to CAPI

We are fortunate in Young Lives to have a number of dedicated field staff in each country. Many of these fieldworkers have been with us from the first round in 2002 and have formed solid relationships with  our children and families. It was key that the technology support and add to the existing skills of our fieldworkers. Through trial and error we have found that it is best to spend time first training our fieldworkers on the content and meaning of each section using a paper copy of the questionnaire. Then after they have an understanding of the section we move on to the program where they can focus on the skips, codes, answer types and screen/question layouts. In many instances, they were able to suggest meaningful and valuable changes to the program based off their local knowledge of the questionnaires and the context of our children. The user interface of the CAPI software we used meant we could replicate the look of the paper questionnaire and replicate it in a screen format which meant the migration from paper to tablets was relatively seamless for many of our key workers.

 

 

Safety is key

Another of the considerations was ensuring the safety of our fieldworkers. A lot of our sites are located across varying circumstances. Some sites are safer than others and some routes of travel require more consideration. However, our fieldworkers have been visiting these sites since 2002 and know them better than anyone. Much like the paper questionnaires, they carry the tablets in non-descript backpacks and bags. They’re encouraged to use seamstresses to sew tablet covers from local textiles. If faced with theft they instructed not to fight but hand over the tablet. As a note, our fieldworkers have been carrying digital cameras, voice recorders and other equipment for a number of years in our project and were comfortable with carrying tablets. Fortunately, with over 150 tablets across the four countries we only came across a few instances where tablets were stolen from fieldworkers.

Logistics

Our sites are in both rural and urban settings with varying levels of access to electricity and transport. Obviously the ability to charge the tablets at night was a priority. In cases where there was no access we used battery packs to charge our devices. Other extras we purchased were USB sticks for sharing and backing up the data and dongles (USB mobile broadband stick) for accessing the internet to share files. Procuring the tablets in all four countries proved unfeasible. In Peru and India we were able to get devices locally. However, in Ethiopia and Vietnam we had to ship them in from the UK. This needed to be built into the timeline to consider time to import and any customs delays. Proper documentation related to the purchase and handover of the devices needed to be in order as well.

Introducing CAPI and tablets to Young Lives children

Other than the logistics and safety of our fieldworkers, a key worry was if the tablets were going to be a distraction or if we used laptops would the screen form a wall between the interviewer and the respondents. During piloting we found that after the initial novelty wore off the interviews carried on just like they do on paper and people were quite at ease with the technology.

 

 

As it has been widely noted, many people in our study countries have leapfrogged to using smart phones so if anything our tablets/laptops could quickly become obsolete.

You will reap the rewards

It must be mentioned that when moving to CAPI the majority of the work comes before the fieldwork starts. Not only do you have the logistical challenges mentioned above, but you must first design the questionnaire then program this questionnaire into CAPI. In Young Lives our research team develops the questionnaire and then we use an off the shelf software to build the CAPI program. This includes screen layout, answer type (drop down vs radio buttons vs text answers), what data to preload, validation checks, and how the files export. All this must be carefully thought through and planned. Everything must be piloted and checked to ensure there are no bugs, data is displaying correctly and the skip patterns work. After the program is built and ready to be administered in the field the process of checking, handling, sending and backing up the data files must be carefully coordinated and everyone must be trained.

Using CAPI and tablets in the field is without a doubt a costly procedure. Does it cost more than PAPI and data entry? You bet. Do the benefits of preloaded data, built in consistency checks and faster access to cleaner data outweigh the costs? Absolutely.

What's next?

In Round 5, in 2016, we will continue to use tablets and CAPI but will be looking at ACASI (Audio Computer Assisted Self Interview) software for administering our self-administered questionnaire. This questionnaire touches on sexual Methods Guide on how to introduce computer-assisted personal interviewing.

Paper was invented in 206 BC….It’s time for a change: Introducing CAPI
Survey design and sampling

(This blog first appeared on the UK Data Service's Data Impact Blog on 24 November 2105)

I recently attended a Global He about the experience of using technology (specifically ACASI) to research violence against adolescent girls in humanitarian situations. During the session, IRC and the Population Council hosted a discussion on the use of technology within an adolescent-girl protection research programme and innovative ways of collecting data in Ethiopia's refugee camps by COMPASS, a research centre, like Young Lives, based in the University of Oxford.

The seminar prompted me to reflect on our own processes and considerations for migrating from administering our questionnaires using paper and pen to using tablets for data collection.

Can CAPI (computer-assisted personal interviewing) solve all your problems?

The simple answer is yes, computer-assisted personal interviewing can solve all your current problems but it will give you some new ones to think about as well! In 2013, Round 4 of our longitudinal survey, we started using tablets to collect our data in the field. Gone were the paper and pens of yore and in were the stylus pens and screens and with this, a myriad of considerations and trade-offs. So here are some reflections on introducing CAPI to teams across four varying countries:

Weighing the benefits of CAPI versus PAPI

Our old process of collecting the data via PAPI (Paper and Pen Interviewing) provided security in that you always had a written record of the respondents answers that you could reference. The problem with PAPI is that it requires a lot of time at the back end building and entering data into a database and then cleaning it for use in analysis. At this point you are hundreds of miles away from your respondents, and months down the road from the interview, so any inconsistencies needed to be checked using the memory of the fieldworkers and supervisors. With CAPI you have the luxury of preloaded data (i.e. household rosters), consistency checks built in to the programme, validation checks and immediate access for preliminary analysis and key findings; as well as saving time in the field by building in skip patterns to the questionnaire program. This resulted in a much cleaner data set that was available shortly after collection.

Introducing Fieldworkers to CAPI

We are fortunate in Young Lives to have a number of dedicated field staff in each country. Many of these fieldworkers have been with us from the first round in 2002 and have formed solid relationships with  our children and families. It was key that the technology support and add to the existing skills of our fieldworkers. Through trial and error we have found that it is best to spend time first training our fieldworkers on the content and meaning of each section using a paper copy of the questionnaire. Then after they have an understanding of the section we move on to the program where they can focus on the skips, codes, answer types and screen/question layouts. In many instances, they were able to suggest meaningful and valuable changes to the program based off their local knowledge of the questionnaires and the context of our children. The user interface of the CAPI software we used meant we could replicate the look of the paper questionnaire and replicate it in a screen format which meant the migration from paper to tablets was relatively seamless for many of our key workers.

 

 

Safety is key

Another of the considerations was ensuring the safety of our fieldworkers. A lot of our sites are located across varying circumstances. Some sites are safer than others and some routes of travel require more consideration. However, our fieldworkers have been visiting these sites since 2002 and know them better than anyone. Much like the paper questionnaires, they carry the tablets in non-descript backpacks and bags. They’re encouraged to use seamstresses to sew tablet covers from local textiles. If faced with theft they instructed not to fight but hand over the tablet. As a note, our fieldworkers have been carrying digital cameras, voice recorders and other equipment for a number of years in our project and were comfortable with carrying tablets. Fortunately, with over 150 tablets across the four countries we only came across a few instances where tablets were stolen from fieldworkers.

Logistics

Our sites are in both rural and urban settings with varying levels of access to electricity and transport. Obviously the ability to charge the tablets at night was a priority. In cases where there was no access we used battery packs to charge our devices. Other extras we purchased were USB sticks for sharing and backing up the data and dongles (USB mobile broadband stick) for accessing the internet to share files. Procuring the tablets in all four countries proved unfeasible. In Peru and India we were able to get devices locally. However, in Ethiopia and Vietnam we had to ship them in from the UK. This needed to be built into the timeline to consider time to import and any customs delays. Proper documentation related to the purchase and handover of the devices needed to be in order as well.

Introducing CAPI and tablets to Young Lives children

Other than the logistics and safety of our fieldworkers, a key worry was if the tablets were going to be a distraction or if we used laptops would the screen form a wall between the interviewer and the respondents. During piloting we found that after the initial novelty wore off the interviews carried on just like they do on paper and people were quite at ease with the technology.

 

 

As it has been widely noted, many people in our study countries have leapfrogged to using smart phones so if anything our tablets/laptops could quickly become obsolete.

You will reap the rewards

It must be mentioned that when moving to CAPI the majority of the work comes before the fieldwork starts. Not only do you have the logistical challenges mentioned above, but you must first design the questionnaire then program this questionnaire into CAPI. In Young Lives our research team develops the questionnaire and then we use an off the shelf software to build the CAPI program. This includes screen layout, answer type (drop down vs radio buttons vs text answers), what data to preload, validation checks, and how the files export. All this must be carefully thought through and planned. Everything must be piloted and checked to ensure there are no bugs, data is displaying correctly and the skip patterns work. After the program is built and ready to be administered in the field the process of checking, handling, sending and backing up the data files must be carefully coordinated and everyone must be trained.

Using CAPI and tablets in the field is without a doubt a costly procedure. Does it cost more than PAPI and data entry? You bet. Do the benefits of preloaded data, built in consistency checks and faster access to cleaner data outweigh the costs? Absolutely.

What's next?

In Round 5, in 2016, we will continue to use tablets and CAPI but will be looking at ACASI (Audio Computer Assisted Self Interview) software for administering our self-administered questionnaire. This questionnaire touches on sexual Methods Guide on how to introduce computer-assisted personal interviewing.