How questionnaires shape answers

Society for Psychotherapy Research Conference, World Chapter
27-30 June 2018, Amsterdam, The Netherlands


How questionnaires shape answers. On validity and performativity
of ‘the data’ in psychotherapeutic research



Background and introduction
In psychotherapeutic research, quantified symptom assessment by means of validated self-report questionnaires is default methodological practice to study pre-post symptom change over a course of treatment. A multitude of problems are voiced in the literature regarding the validity of numerical assessment. Nonetheless, quantified symptom measurement is often regarded as ‘the best we have’. In this paper, we argue that it is not feasible to take the issues for granted in gathering ‘the data’, as questionnaires not only are hard (if not impossible) to interpret straightforwardly and universally, but they also actively change the object of interest, which has severe implications for the epistemic value of psychotherapy research.

We structure our argument around data of two patients who participated in the Ghent Psychotherapy Study (GPS; Meganck et al., 2017). We discuss their scoring on a battery of symptom- and psychological wellbeing measures that were administered repeatedly before, during and after treatment. First, we discuss the interpretation of their quantitative scores given the idiosyncratic context. Second, we use interview data to zoom in onto the processof the scoring and its meaning for these particular participants.

Both cases were seriously affected by having to score the questionnaires, which has a salient impact on their primary symptoms – for one patient positively, for the other negatively – and therefore on the pre-post difference that is obtained to study treatment effect. By means of these case discussions, we reiterate the nature and complexity of numerical representation, but also show how the act of administration creates a surplus that would not have occurred in treatment outside the context of psychotherapy research. We call this the performativityof data collection.

This performativity of questionnaire administration has severe consequences for the ‘object of interest’, which is change due to psychotherapeutic treatment. As the administration of questionnaires affects the pre-post difference scores beyond the treatment effect itself, performativity in principle prohibits straightforward interpretation of numbers that we are used to in psychotherapy research. This is not necessarily bad, as it does provide a wealth of clinically relevant information on patient behaviour and change factors, yet it is crucial to acknowledge that simple straightforward interpretation is simply illusory. We argue that it is necessary to start asking our research questions differently, such that it actually fits ‘the data’ gathered in psychotherapy research.

Comments are closed.