What we can lose (and learn) when we exclude
Monday, August 17, 2020
Posted by: SPR
What we can lose (and learn) when we exclude:
Rethinking SCR-based Exclusions in Human Fear Conditioning Research
Decisions about whether to exclude participants from research are complicated. This is because there are many legitimate reasons to exclude a participant or set of participants from a dataset, and there are typically no established criteria by which to decide whether a participant should or should not be excluded as this will heavily depend on the specific study. This ambiguity is common to virtually all areas of research and, importantly, may lead to substantial methodological heterogeneity as well as missed opportunities to examine potentially meaningful variation in our samples.
Research concerned with how we learn to fear specific stimuli in our environment often involves training participants to associate threat (i.e., unconditioned stimulus/US) with some neutral stimuli (i.e., conditioned stimulus/CS+), and not others (CS-). To quantify this learning, researchers commonly use skin conductance response (SCR), an index of sympathetic nervous system activation that is larger in response to fear conditioned stimuli (CS+) compared to neutral stimuli (CS-). However, it is often the case that some participants’ SCRs do not discriminate between stimuli that predict threat (CS+) and those that predict safety (CS-), nor do some participants respond with measurable (i.e., non-zero) or consistent SCRs to the conditioned stimuli (CS+ and/or CS-). These physiological ‘non-learner’ and ‘non-responder’ participants are often excluded from subsequent analyses. Importantly, the ways in which ‘non-learner’ and ‘non-responder’ exclusions are defined significantly varies within and across labs, potentially resulting in substantial variability in exclusion criteria.
In a recent paper published in elife, lead author Dr. Tina Lonsdorf reviewed the human fear conditioning literature published in the past-six months for differences in SCR-based exclusion criteria. “The topic of procedural and methodological heterogeneity has been bothering me for a long time. What had been bothering me was that criteria provided for “non-learning” were often ambiguous, if specified at all. Also, I have often been asked by reviewers to exclude “non-learners” from my own data analyses, but I had always been hesitant and reluctant to do so for a number of reasons outlined in the elife paper”, says Dr. Tina Lonsdorf.
In her literature review, Dr. Lonsdorf and her colleagues found that 40% of studies excluded ‘non-learner’ and ‘non-responder’ participants, with the percentage of participants excluded for being a ‘non-learner’ ranging anywhere from 2% to 74% of the sample. This range in participant exclusions was also accompanied by substantial differences in how studies qualified ‘non-learner’ and ‘non-responder’ participants. For example, criteria used to define ‘non-learners’ ranged in terms of the number of trials included, as well as the magnitude to which participants’ SCRs differed between threat and safety stimuli (i.e., CS+/CS- discrimination), with different discrimination cut-offs used. Criteria for defining ‘non-responders’ also varied, with definitions ranging in terms of the minimum SCR amplitude to qualify as a valid response, as well as the number of ‘non-responses’ considered for exclusion. “I did of course expect substantial heterogeneity in definitions, but I was actually surprised to see how much variance there was in definitions of ‘non-learning’ and ‘non-responding’”, says Dr. Lonsdorf.
Apart from this variability in defining ‘non-learner’ and ‘non-responder’ participants, other potential problems may arise when using SCRs as a decision point for participant exclusions. SCRs are influenced by many different factors, including demographic and genetic factors, as well as clinical symptoms and disorders. Therefore, the exclusion of people who show low (or absent) fear conditioned responses as measured by SCRs may lead to missed opportunities to examine important individual differences that modulate these processes. For example, a recent study published in Psychophysiology showed that healthy participants who would be classified as SCR ‘non-learners’ and ‘non-responders’ showed reduced activation in brain regions typically associated with fear learning and response (e.g., amygdala, insular cortex) compared to people who showed evidence of learning and/or responding as indicated by their SCRs. This suggests that these types of SCR-based exclusions may limit our ability to understand why and how people vary in their ability to acquire conditioned fear responses.
These types of exclusions also raise the risk of ‘selecting out’ people from samples with particular characteristics and symptoms that are of interest to the study question. For instance, anxious individuals often show overgeneralization of fear responding—that is, reduced ability to distinguish between threat (CS+) and safety (CS-) stimuli, pointing to the possibility that these individuals may be identified as ‘non-learners’ when inclusion criteria require high SCR discrimination. To illustrate this, Dr. Lonsdorf and her colleagues re-analyzed existing data using the different CS+/CS- discrimination cut-offs identified in their literature review and examined whether these exclusion criteria induced sample bias by excluding highly anxious individuals. When higher cut-offs were used (i.e., participants were required to show larger SCR differences between the CS+ and CS-), anxious individuals were more likely to be excluded. Dr. Lonsdorf explains, “[These results] imply that we would exclude individuals that display a discrimination pattern similar to that observed in clinical or high-risk samples from our experimental work. Importantly, these are the individuals that we would like to make inferences on from our experimental work. These SCR-based exclusions may, therefore, pose severe limitations on potential clinical translation of basic research”.
This emerging research on SCR-based exclusions highlights what we can learn from people who show more extreme variation in fear learning and response, as well as what we can lose from our samples by using these types of exclusions. Importantly, these findings demonstrate the need to carefully consider the potential implications of using SCRs when making decisions about participant exclusions, and point to ways in which we can better make these decisions. For instance, decisions about participant exclusions should not necessarily be based on SCR results alone. Rather, these decisions could consider SCR alongside other proxies of fear conditioning and response, such as fear-potentiated startle, or subjective ratings of fear. This may move us closer to better defining ‘non-learning’ and ‘non-responding’. Furthermore, clear and detailed reporting of why and how participants were excluded in our studies may support more robust and replicable research findings. To guide this endeavor, Dr. Lonsdorf and her colleagues include a list of recommended reporting details that can be found in their elife paper. This list can be annotated online by other researchers and students who are interested in providing suggestions.
Dr. Lonsdorf welcomes researchers to add and edit this list.
To learn more about Dr. Lonsdorf’s research, click here. If you are interested in learning more about methodological issues related to fear conditioning research, check out recent work published by members of the EIFEL-ROF, an interdisciplinary network of European fear conditioning researchers.