Blinding idea evaluation? New experiment reveals little bias in the innovation process

Innovation and implementation of new ideas can directly influence organizational outcomes. Thus, organizations often dedicate significant time to the assessment of countless new ideas through myriad idea evaluation systems. However, the evaluation process is often distorted by various biases arising from hierarchy, sequence, and nepotism. Data on evaluation studies show that evaluators can be biased toward specific idea proposers.

In a new study published in the Strategic Management Journal, a research team sought evidence for the bias claim by employing a blinding approach for evaluating ideas and diagnosing biases in the process. The study focused on three potential biases. It tested whether lower evaluations are given to ideas coming from: (1) female idea proposers; (2) idea proposers from other office units; and (3) idea proposers from other locations.

Authors Linus Dahlander, Arne Thomas, Martin W. Wallin, and Rebecka C. Angstrom conducted a field experiment at a top multinational technology corporation. Innovation managers were asked to evaluate 48 random ideas from the company’s idea management system, doing both blind and non-blind evaluations. While the managers received no information about the idea proposer during blind evaluations, they were provided with the proposer’s name, unit, and location in the non-blind evaluations.

Much to their own surprise, the researchers could not find bias in idea evaluation due to gender, location, or organizational structure. This shows that evaluators separate ideas from proposers, and hiding the identity of idea proposers from evaluators might not necessarily improve idea evaluation.

“A follow-up online experiment with a larger sample size and more statistical capacity replicated the results of the field experiment. Moreover, an additional study showed that people overestimated the magnitude of the biases,” Dr. Thomas said.

The Strategic Management Journal, published by the Strategic Management Society, is the world’s leading mass impact journal for research in strategic management.