@InProceedings{lehman:iec-amt,
    author = "Lehman, Joel and Miikkulainen, Risto",
    title = "Leveraging Human Computation Markets for Interactive Evolution",
    booktitle = "Machine Learning Meets Crowdsourcing Workshop at ICML 2013",
    year = "2013",
    site = "http://joellehman.com/lehman_icmlworkshop.pdf",
    abstract = "Leveraging human input for selection in an evolutionary algorithm, i.e. interactive evolution, is effective when an appropriate domain fitness function is hard to quantify, but where solution quality is easily recognizable by humans. However, single-user applications of interactive evolution are limited by user fatigue: Humans become bored with monotonous fitness evaluations. This paper shows that user fatigue can potentially be bypassed through human computation markets that enable directly paying for human input. Experiments evolving images show that purchased human input can be leveraged more economically when evolution is seeded with products from a purely-computational aesthetic measure. Further experiments in the same domain validate a system feature, demonstrating how human computation can help guide interactive evolution system design. Finally, experiments in an image composition domain show how the approach can facilitate large-scale interactive evolution in tasks that are not inherently enjoyable. In this way, combining human computation markets with interactive evolution facilitates mechanical application of a powerful form of selection pressure."
}