Revisiting Machine Learning from Crowds a Mixture Model for Grouping Annotations

This work addresses learning from crowds, where supervised learning systems rely on labels provided by multiple subjective or inexperienced annotators. The proposed mixture model learns patterns of annotation noise by grouping similar labeling behaviors rather than modeling a separate noise pattern for each annotator. By using a fixed-size mixture model, the approach requires fewer parameters, handles sparse annotation distributions, and scales more effectively to large datasets. Experiments on both real and simulated datasets demonstrate the advantages of the proposed method over existing approaches, showing its ability to capture common annotation patterns while providing a more scalable framework for learning from noisy and heterogeneous crowd-sourced labels.

Recommended citation: Mena, Francisco, and Ricardo Ñanculef. Revisiting Machine Learning from Crowds a Mixture Model for Grouping Annotations. Iberoamerican Congress on Pattern Recognition. Cham: Springer International Publishing, 2019.
Download Slides