One criterion, two merits: A single-criterion-based sample selection method for informativeness and diversity

In streaming batch-mode active learning process for data, sample selection typically involves two stages: informativeness measurement and similarity measurement. By analyzing the expression of model performance improvement induced by new samples, we identify a linear relationship between the performance gradient and the sample's vectors. Based on this finding, we propose a streaming batch active learning sample selection method, named One Criterion Two Merits (OCTM), which integrates informativeness and diversity measurement using a single criterion—the model improvement gradient. First, the model update gradient is computed for each incoming sample. Then, the magnitude of this gradient is used as an informativeness measure. Finally, the minimum angle between the new sample and buffer samples is calculated to quantify diversity. The threshold used for real-time decisions is critical in data stream scenarios, which traditionally relies on the assumption of a known threshold distribution. To address this issue, we propose a distribution-free threshold estimation method that determines the threshold based on the distribution of labeled samples. By sorting the measurement values and setting a confidence level, the threshold can be effectively computed.