CATI: providing best-in-class telephone research
What separates a well-run computer-assisted telephone study from a badly run one, and why the difference rarely shows up in the topline.
CATI is often sold as though the software were the method. It is not. The software removes a category of clerical error and enforces the routing; everything that determines whether the study is any good happens around it.
The script is the product
The screen is a worse reading environment than paper. Long preambles that scan fine in a document become unlistenable when read aloud at interviewing pace, and interviewers under time pressure will compress them — differently, each time, which is precisely the interviewer variance the mode is supposed to control.
Scripts should be written to be spoken. Short stems, response options that a person can hold in their head, and no question that requires the respondent to remember a list of seven items they cannot see.
One structural weakness is worth designing around: CATI systems make going backwards awkward. Interviewers working from paper find it easier to correct an earlier answer, and interviewers who have used both tend to prefer paper for exactly that. If a respondent is likely to revise an answer once they understand where the questionnaire is going, put the question later.
Sample management is where studies are won
A response rate is not a property of the mode; it is a property of how the sample was worked. That means a call schedule that varies time of day and day of week rather than dialling the same list at the same hour, a defined number of attempts before a number is retired, and appointment setting for people who are reachable but busy.
It also means reporting the outcome honestly. Dispositions should be recorded for every number and the response rate computed on a standard definition rather than on whichever denominator flatters the study. The point is not the number itself — low response rates do not automatically mean biased estimates — but that an undisclosed denominator makes the figure meaningless.
Monitoring, not auditing
Quality control that happens after fieldwork tells you what went wrong. Live monitoring, with a supervisor listening to a proportion of calls from day one, tells you in time to fix it. Almost every serious problem in a telephone study — a misread question, a routing error nobody noticed, an interviewer helpfully explaining what a question means — is cheap on day one and expensive on day ten.
The unglamorous summary
Best-in-class telephone research is a well-spoken script, a disciplined call schedule, honest disposition reporting and someone listening. The dialler is table stakes.
