Supplementary material for the paper 'Learning to classify software defects from crowds: A novel approach':