A new version of this entry is available:
Loading...
Article
2025
Idea evaluation for solutions to specialized problems: leveraging the potential of crowds and Large Language Models
Idea evaluation for solutions to specialized problems: leveraging the potential of crowds and Large Language Models
Abstract (English)
Complex problems such as climate change pose severe challenges to societies worldwide. To overcome these challenges, digital innovation contests have emerged as a promising tool for idea generation. However, assessing idea quality in innovation contests is becoming increasingly problematic in domains where specialized knowledge is needed. Traditionally, expert juries are responsible for idea evaluation in such contests. However, experts are a substantial bottleneck as they are often scarce and expensive. To assess whether expert juries could be replaced, we consider two approaches. We leverage crowdsourcing and a Large Language Model (LLM) to evaluate ideas, two approaches that are similar in terms of the aggregation of collective knowledge and could therefore be close to expert knowledge. We compare expert jury evaluations from innovation contests on climate change with crowdsourced and LLM’s evaluations and assess performance differences. Results indicate that crowds and LLMs have the ability to evaluate ideas in the complex problem domain while contest specialization—the degree to which a contest relates to a knowledge-intensive domain rather than a broad field of interest—is an inhibitor of crowd evaluation performance but does not influence the evaluation performance of LLMs. Our contribution lies with demonstrating that crowds and LLMs (as opposed to traditional expert juries) are suitable for idea evaluation and allows innovation contest operators to integrate the knowledge of crowds and LLMs to reduce the resource bottleneck of expert juries.
File is subject to an embargo until
This is a correction to:
A correction to this entry is available:
This is a new version of:
Other version
Notes
Publication license
Publication series
Published in
Group decision and negotiation, 34 (2025), 4, 903-932.
https://doi.org/10.1007/s10726-025-09935-y.
ISSN: 1572-9907
Other version
Faculty
Institute
Examination date
Supervisor
Cite this publication
Gimpel, H., Laubacher, R., Probost, F., Schäfer, R., & Schoch, M. (2025). Idea evaluation for solutions to specialized problems: leveraging the potential of crowds and Large Language Models. Group decision and negotiation, 34(4). https://doi.org/10.1007/s10726-025-09935-y
Edition / version
Citation
DOI
ISSN
ISBN
Language
English
Publisher
Publisher place
Classification (DDC)
650 Management and public relations
Collections
Original object
University bibliography
Standardized keywords (GND)
Sustainable Development Goals
BibTeX
@article{Gimpel2025,
doi = {10.1007/s10726-025-09935-y},
url = {1https://hohpublica.uni-hohenheim.de/handle/123456789/18211},
author = {Gimpel, Henner and Laubacher, Robert and Probost, Fabian et al.},
title = {Idea evaluation for solutions to specialized problems: leveraging the potential of crowds and Large Language Models},
journal = {Group decision and negotiation},
year = {2025},
volume = {34},
number = {4},
pages = {903--932},
}