A Brief Survey Reflection
by Dave Grace -- July 22nd, 2016
Thanks to Dr. Randy Kramer who provided a very pragmatic and helpful course on “Social Science Surveys,” providing a ‘soup to nuts’ crash course on survey methodology for masters students to be able to conduct their own surveys within the established norms of the scientific community.
This was quite helpful for my participation in the present survey on the forest preferences of residents nearby sacred forests of varying urbanization levels in India’s National Capital Region and its periphery.
I am now more aware that data collection by means of survey is a lot of work. The physical work of the survey (n=200) required approximately one month’s time, while survey development and preparation spanned over a year’s time, commencing with exploratory research in the study site last summer.
Once over half of the surveys were complete I felt like I needed a bath, but also felt like we were over the hill.
Image for scale not endorsement.
Here are a few things I picked up from the course:
Surveys are difficult partly because of how deceptively simple they seem. Anyone can ask people questions and record answers. Thus, there is much concern about methods and reliability.
On the one hand, there are statistical questions, dealing with probabilistic sampling procedures and measures of error. Here, much concern is devoted to efficiently and reliably secure the statistical power, in a probabilistic sample, necessary to move beyond description of an isolated sample in order to infer about a larger population from the limited sample. Particularly fascinating is the application of statistical surveys in U.S. presidential polls to predict the votes of a population of over 200 million from a sample of only a thousand or so voters. This is important.
In addition to the statistics, in practice surveys are a structured encounter and exchange between people. Since surveys are interpersonal exchanges, surveys require common socials (and perhaps some uncommon ones such as patience and a curiosity for the mundane). For instance, surveys administered face to face require the enumerator to standardize the way they offer the questions to the respondent without excessively influencing responses. Patience is required because it is easy to anticipate responses and to lead respondents in a certain path.
For instance, words can have multiple meanings in one language and this difficulty in communication increases when translating the survey from one language to another. A less than clear question in my survey asked about whether respondents felt it to be distracting to see others outside of their community while they worship. My intention was for community to be defined geographically whereas the enumerators translated community as caste. This was my mistake both in writing the question and communicating its intent to the enumerators. This simply is a different question than originally intended and will be analyzed as such.
Survey’s also require many mundane organizational details, especially those conducted face-to-face. Who will the enumerators be? What, when, and where will food/transportation/housing be? For researchers who are not from the study location, there are issues of nationality, language, and culture. As an example, there are many social barriers that are not self evident from a satellite view or in the creation of a random sample. For instance, some sample locations in my survey were unavailable due to recent deaths and a isolated geography for mourning. This mourning period was longer than our sampling period so alternative sample locations were selected. One sample location was temporarily unavailable because of a shooting. After the shooting, we simply returned the next day. Some female respondents were unable to complete surveys because males demanded an end to the survey. We complied with this rare demand and only a couple of incomplete surveys resulted.
Despite survey challenges, the 200 surveys are now complete. Showers have been taken and protein will now be required for regaining body mass after this physically and intellectually taxing exercise.
Image for scale not endorsement.