The evaluation of creative ideas – analysing the differences between expert and novice judges

Abstract

The evaluation of creative ideas is a special case of judgment and decision making. It is difficult to objectively evaluate creative products because most people possess an internalised model of creativity which is usually neither verbalised nor explicitly defined. Also, one of the main assessment dimensions of creativity, originality varies as a function of the evaluator’s previous experience. For these reasons, previous research has provided practical rather than theoretical grounds for studying the evaluation process. The present thesis examines the conceptual basis on which people evaluate creative ideas. The aim is to identify factors and conditions which enhance the detection of creative ideas. A novel paradigm was created to test how creativity-related features influence the assessment of creativity. In six experiments, experts’ and non-experts’ judgment was examined regarding urban design. Two experiments established the expert ratings of the stimuli. Further two experiments explored the extent to which non-experts relied on four features (originality, utility, scalability, and riskiness) for judging the creativity of novel project ideas while the level of motivation was controlled. Overall, the findings show that non-experts’ creativity judgment relied on all four characteristic features. Their ratings of the features predicted a substantial part of the variance in the creativity ratings. In another experiment, the effect of providing explicit task-related information was tested. Such information did not make a solid difference in the creativity ratings. A final experiment assessed the differences between making relative and absolute judgments about creativity. There was a large overlap between the selection of best and worst ideas regardless of which way the judgment was made. In conclusion, non-experts were found to possess a robust internal model of creativity and not to make random choices. Experts and non-experts were found to judge creativity vastly differently, they only agreed that utility is the most important criterion.