Fig. 1

In this 2 × 2 table, the labels for the rows and columns refer to the predictions made by Machine A and Machine B on all the training data. The zeroes and ones in the four cells are known outcomes in the data, and the optimal crowd directs the analysis to these outcomes