“The Insight is in the Verbatims”
Having worked solely with verbatims over the past year, I have come to believe that the true insight from a quantitative survey is in the verbatim comments. Why is this?
A quantitative survey has basically two types of question – the precoded and the open-ended (to me, Lieckert scales are a variety of precoded question). In the precoded questions, the range of possible answers are defined before anyone is interviewed – and therefore is known prior to the start of the survey. Open-ends, on the other hand, are for the unknown unknowns – you know only that your target sample for your survey will have an opinion on the matter, but not what that opinion will be. And this is what I mean by the insight being in the verbatims – without them, you wouldn’t know if a new, previously unconsidered factor, was affecting decision making.
Management of Verbatims.
This leads on to the importance of proper coding at all stages of a project. If you are going to get the full value and insight from the verbatims collected in a project, you need to be clear with how you will deal with them (and make sure you have the budget). Code-frames must be flexible; they should be allowed to change, wave by wave if necessary, in order to reflect the changing perceptions of the sample being surveyed. If code frames are set, and not allowed to change, you are in effect wasting your time collecting verbatim data; you might as well, after the first wave, set the question up as a pre-coded question, and save time (and money) – because opinions and attitudes will change over time – that is the whole point of doing tracking studies.
If you are doing a large study, with potentially thousands of verbatim answers, think about what you are going to do with the verbatims before you start: are you going to read them all, or are you going to code them, or are you going to analyse them in some other way? How much will that cost, and how long will it take? Is it worth it to you, and to your client?
If you do not have the time or the budget to code the verbatims, or are not prepared to allow the code frame to be flexible, you should ask yourself whether they are necessary before fielding the study; answering verbatims, whether on the telephone or on the web, lengthens the interview, and adds to both drop out rates and the cost of carrying out the survey. If the verbatims are then not going to be used properly (and the valuable insight they contain is going to be lost), you might as well not collect the in the first place.
Having written this post, I just read through it and it seems a bit… didactic. I didn’t mean it to be. We, at Katugas, like to provide a good and flexible coding service to our clients, but when we see valuable insight being thrown away, we get upset.
Please think about your verbatims!