Add 'platt' to calib options in EMQ? Allow n_prevpoints in APP to be specified by a user-defined grid? Add the fix suggested by Alexander: For a more general application, I would maybe first establish a per-class threshold value of plausible prevalence based on the number of actual positives and the required sample size; e.g., for sample_size=100 and actual positives [10, 100, 500] -> [0.1, 1.0, 1.0], meaning that class 0 can be sampled at most at 0.1 prevalence, while the others can be sampled up to 1. prevalence. Then, when a prevalence value is requested, e.g., [0.33, 0.33, 0.33], we may either clip each value and normalize (as you suggest for the extreme case, e.g., [0.1, 0.33, 0.33]/sum) or scale each value by per-class thresholds, i.e., [0.33*0.1, 0.33*1, 0.33*1]/sum. - This affects LabelledCollection - This functionality should be accessible via sampling protocols and evaluation functions - [TODO] Test the return_type="index" in protocols and finish the "distributing_samples.py" example - [TODO] add ensemble methods SC-MQ, MC-SQ, MC-MQ - [TODO] add CDE-iteration and Bayes-CDE methods - [TODO] add Friedman's method and DeBias - [TODO] check ignore warning stuff check https://docs.python.org/3/library/warnings.html#temporarily-suppressing-warnings - [TODO] nmd and md are not selectable from qp.evaluation.evaluate as a string - [TODO] add https://www.kaggle.com/datasets/Cornell-University/arxiv dataset? in the paper "Online Adaptation to Label Distribution Shift" they say it provides a natural label shift over time - [TODO] add Bhattacharyya distance (https://en.wikipedia.org/wiki/Bhattacharyya_distance) (which is actually not a proper distance)