Survey Pointers

SURVEY POINTERS

Julhusin B. Jalisan

"The rightful concern of statisticians is the use of science to

provide relevant information on large populations,

to be made available to decision agents

within a short period of time and

of acceptable quality."

Vicente T. Paterno

I

n this fast changing world, problems in almost all fields of human endeavor have to be solved and resolved on scientific bases. The influence of decisions based on quantitative information has never been more evident than in the present decade especially with the use of modern technology. Indeed, the growing complexities of the activities and functions of the various fields of endeavor have made the use of statistics imperative.

However, one major problem among statisticians, under a given circumstance and budgetary constraints, is the production of reliable data due to built-in biases of the collectors and/or respondents.

In many cases, the main cause of failure in any research undertaking is the wrong choice of method in the collection of data. This is especially true in the case of survey. While the confidence level can be established as basis for accepting the representativeness of the samples, and variability can be computed for estimating probable sampling errors, it is virtually impossible to do the same due to errors introduced through bias either positively or negatively.

Perhaps the most important point to remember is that, despite the formidable arrays of formulae, statements of confidence, and tests of significance, the quality of data collected is the most important aspect of any survey. The information generated by statistical manipulation can only be as good as the data upon which it was based. The adage "Garbage In, Garbage Out" (Smith, 1981), therefore, should be borne in mind when designing surveys and gathering data, so that the processing of the data will bear useful results.

SURVEY DESIGN

Surveys are usually undertaken to provide answers for researchers on problem situations. Sometimes, however, a researcher just wants to establish a baseline on particular aspects of a situation against which to subsequently measure progress.

In any survey work, the first task is to identify the problem. Frequently, the researcher's objective is only half formulated, ambiguous, or a statement of observed symptoms about which he or she is concerned. Often, this is expressed as a question. One must get clear guidance on what to study before acting to develop a questionnaire or a lot of time and effort will go for naught. Once the purpose or problem has been stated in an objective manner, the need for a study will become clearer, and detailed survey questions can be formulated. In developing a concise statement of the problem, the following guide questions can be of help (Anderson, 1952):

 What new knowledge to be obtained?

 What hypothesis to be tested?

 What problem to be solved?

 What types of information should shed new light on the prevailing situation?

Brainstorming is a useful technique for establishing the purpose. Accepting the suggestions of several other interested participants, uncritically, is a good way to get started.

General Guidelines for Questionnaire Design

Discussed below are some important points to remember when preparing a questionnaire.

Single Purpose. Whenever possible, limit the survey to a single purpose. A poor, but frequent, practice is to try to accommodate several different things in one survey, rationalizing that it does not take much longer to ask another question while there, and it is cheaper than running a separate survey, etc. Unfortunately, a "multi-purpose shopping expedition" usually results in a cumbersome document that may never be completely analyzed, but which will effectively hinder the gathering and processing of data for the primary intended purpose. Furthermore, a sample survey that is properly structured to meet a specific need is generally not a suitable vehicle for answering multi-purpose questions from the same sample base. Consequently, even if analyzed, the additional data may be invalid.

Limit the Number of Questions. Each question takes time and costs money to ask, process and analyze. Therefore, be selective. Screen each proposed question carefully and decide whether such answer can be more readily obtained elsewhere. If a questionnaire is too long, it can affect the attention and accuracy of the respondents.

Avoid Leading Questions. Many people respond to please the interviewer. To avoid embarrassment, they tell what they think he/she wants to hear. Others deliberately distort their answers depending on how they perceive the answer may be used. One cannot eliminate all the problems in this area, but the survey can be improved considerably by being careful to phrase the questions as objectively as possible.

Avoid "Memory" Questions. Questions which rely on an individual's recall and cannot be verified in any meaningful way are likely to have high degree of inaccuracy. Also, the longer the period of recall, the more inaccurate the answer is likely to be.

Cross-Check Questions. If there is likely to be a strong element of doubt or distortion in the answer, provide for some probing or objectively verifiable cross-check questions, if possible. It is not, however, necessary to record the responses to probing questions.

Clarity. Pre-test the questions before deciding on the exact wording to be used in the questionnaire. This is absolutely essential. Questions that may appear clear and straightforward to the survey designer may prove to be confusing to the respondent and could elicit irrelevant answers. Make sure the questions are phrased in the appropriate familiar dialect of the respondent to ensure understanding.

In addition to the above guidelines, the following can be of help to facilitate the preparation of a questionnaire:

Identification. Each question and possible response