How comprehensible are scientific experts? A multi-method comparison of linguistic analyses, surveys, and real-time audience responses

Abstract

The comprehensibility of scientific experts is fundamentally important but presents a challenge for experts and their audiences. The definition and evaluation of comprehensibility are central to developing approaches for improvement. On the one hand, comprehensibility can be indicated by linguistic measures; on the other hand, audience assessments represent comprehensibility perceptions. However, the extent of overlap between these perspectives remains unknown. We conducted two studies to address this gap by analyzing five debates on various scientific topics, each featuring three experts. Our approach involved an integration of computational linguistic analyses, surveys, and real-time response measurements. The findings demonstrate that content and linguistic complexity appear complementary in their relationships with audience ratings for comprehensibility. Interestingly, more complex expert statements corresponded to higher overall debate ratings, hinting at the potential influence of human factors. Therefore, recognizing this influence is critical for improving the communication between experts and laypeople.