[Adapted from General Accounting Office (1991). "Using Structured Interviewing Techniques." Washington, DC: Program Evaluation and Methodology Division Report 10.1.5.]
One key element in conducting useful research is gathering reliable information. And the basis for doing that is designing questions and questionnaires that get the kind of information from which the researcher can draw valid conclusions.
When one looks at a completed questionnaire or the results of a valid study, he or she is often tempted to say, "That's not difficult to do," but that temptation quickly passes after a few minutes of closer analysis. That's about all it takes to realize that designing a good question and good questionnaire requires more thought and time than one might originally think. There are an almost infinite variety of things to think about and to do correctly to avoid the kinds of errors that can make scores of hours of work worthless.
This digest looks at some of the basic building blocks of a structured interview, points out potential pitfalls, and suggests ways for the researcher to avoid them, in order to produce a set of questions that have the best possibility of generating reliable, accurate data on the topics of interest.
SOME BASIC TERMS
The first thing that a new researcher needs to know is some standard
terminology. This knowledge helps that person understand other concepts
to be introduced and puts him or her in a position to communicate with
more experienced people in the discipline.
Structured interview--One that uses a DCI to gather data, either by telephone or face to face. In a structured interview, the evaluator asks the same questions of numerous individuals in a precise manner, offering each individual the same set of possible responses. (In contrast, an unstructured interview contains many open-ended questions that are not asked in a precise, structured way.)
Computer-assisted telephone interview (CATI)--A form of telephone interview where the DCI is stored in a computer and the interviewer records responses directly into the computer.
ADVANTAGES AND DISADVANTAGES TO VARIOUS DATA-GATHERING METHODS
This section discusses the relative benefits and disadvantages of the
two primary types of structured interviews and compares them to the mail
- Enables the interviewer to establish rapport with the respondent
- Allows the interviewer to observe as well as listen
- Permits more complex questions to be asked than in other types of data collection
- An effective method of gathering data when the DCI is lengthy
- Some uses: to get before-and-after data about a lesson module or a change in administrative procedure; to gather opinions on a specific learning or teaching technique
- Less costly than personal interview
- Takes less time than a personal interview
- Simplifies recording of data if CATI is used
- Most effective when the number of questions is relatively small and time available to gather data is short
- Some uses: gathering information on parent satisfaction with the educational program; determining awareness of programs and tools available in the school district
Mail Questionnaire (not a structured interview method)
- Least costly but slowest method of collecting data
- Requires precise question design to match reading comprehension of respondents
- Some uses: to get demographic information on district residents; to get opinions on potential new programs, if they can be clearly described in the questionnaire
DESIGNING A STRUCTURED INTERVIEW
The first step in designing the interview is to formulate the broad overall questions that the survey is intended to answer, i.e.,
- Why is the study being undertaken?
- What does the study aim to learn or determine?
- - To describe something that has occurred; e.g., How do parents view the value of computer use as an aid to teaching math in elementary grades?
- - To compare results to some standard (a normative study); e.g., Has the use of computers as a teaching aid matched the goals of the teachers using these tools?
- - To determine if a procedural change has made a difference (a cause-and-effect study); e.g., Has the use of computer-assisted instruction changed student attitudes about learning math?
The second task is to translate the broad overall questions into measurable elements as hypotheses or more precise questions. The descriptive question above, for example, would require developing measures such as parents' relative knowledge about computer use in math instruction, the importance parents place on math learning, whether parents believe their children learned more with or without the use of computer technology, to name a few.
Then, the target population needs to be identified. If computer-assisted math instruction is new only to a certain grade or school, care needs to be taken to interview only the parents whose children have been affected by the program change. It may sound basic, but innumerable studies have proceeded to unsatisfactory completion because of insufficient consideration of this aspect.
Now, the study can proceed to the development of a pool of specific questions designed to elicit the desired information. The number of questions developed should be more than the number to be asked; then, the most appropriate and useful can be selected from those available.
COMPOSING APPROPRIATE QUESTIONS
Three main criteria exist for writing appropriate questions: relevance, selection of the proper respondents, ease of answering.
- Relevance--Questions should be directly related to the purpose of the study and have a good probability of yielding the kind of data desired.
- Selection of respondents--Even though a question may be relevant to the study, it may not be answerable by the people to whom it will be asked.
- Ease of response--Questions need to be relatively easy to answer and should not create embarrassment for or an undue burden on the interviewee.
Among the types of questions that should be avoided are those that require respondents to consult records or other information sources, would make them uncomfortable for any reason, would reflect negatively on them, would make the interview confrontational, or have no specific answer.
SELECTING A QUESTION FORMAT
Important considerations in deciding on the format of questions include how the question is to be delivered (mail, telephone, face to face), the type of information the respondent is expected to provide, and the possible alternative responses. Making these decisions will result in the selection of open-ended, fill-in-the blank, binary-choice, scaled-response, or unscaled-response questions. Of course, depending on the type of information desired, a structured interview questionnaire will generally have a combination of these types of questions.
- Open-ended--Because open-ended questions provide no structure for the answer, they should be tightly focused to elicit the kind of information the researcher wants to get. And, because they require accurate and time-consuming transcription, their use should be limited to initial research where the number of respondents is small and the object is to refine the research direction and determine more precise questions that can be structured another way.
- Fill-in-the-blank--This type of question has a simple answer, usually a name, frequency, or quantity, which is the kind of information these questions are good at obtaining.
- Binary--These are good for obtaining factual information that falls into the yes-no, true-false category of answer.
- Scaled-response--These consist of a list of alternative responses that increase or decrease in intensity in an ordered fashion. These kinds of questions can be further defined as balanced, unbalanced, and rating and ranking.
- Unscaled-response--With this type of question, the respondent is asked to choose one or more options from a list; this is the type of question that should include the ootheroe category so that the responder is not forced to select an answer with which he or she is not completely satisfied.
The reader should be alerted to the fact that this digest provides only an introduction to structured interviewing. Success in developing and conducting structured interviews requires consulting and studying additional references and reliance on the assistance of experts in the field.
REFERENCES AND RECOMMENDED READING
Campion, M.A., Campion, J.E., & Hudson, J.P., Jr. (1994). "Structured Interviewing: A Note on Incremental Validity and Alternative Question Types." Journal of Applied Psychology, 79, 998-1002.
Hollwitz, J. & Wilson, C.E. (1993). "Structured Interviewing in Volunteer Selection." Journal of Applied Communication Research, 21, 41-52.
General Accounting Office. (1991). Using Structured Interviewing Techniques. Washington, D.C.: Program Evaluation and Methodology Division (http://www.gao.gov/policy/10_1_5.pdf).
Pawlas, G.E. (1995). "The Structured Interview: Three Dozen Questions to Ask Prospective Teachers." NASSP Bulletin, 79, 62-65.
Watts, G.E. (1993). "Effective Strategies in Selecting Quality Faculty."
Paper presented at the International Conference for Community College Chairs,
Deans, and Other Instructional Leaders. Phoenix, AZ.
Library Reference Search
Please note that this site is privately owned and is in no way related to any Federal agency or ERIC unit. Further, this site is using a privately owned and located server. This is NOT a government sponsored or government sanctioned site. ERIC is a Service Mark of the U.S. Government. This site exists to provide the text of the public domain ERIC Documents previously produced by ERIC. No new content will ever appear here that would in any way challenge the ERIC Service Mark of the U.S. Government.