The paper analyzes the expected size of conformal prediction sets.
problem Lack of finite-sample analysis and guarantees for prediction set sizes.
method Theoretical quantification and empirical computation of expected set size.
result Derives point estimates and high-probability interval bounds for prediction set size.
Automates size normalization for fashion items.
problem Reduce merchandise returns in e-commerce.
method Uses sales data to automate size mapping.
result Automated size mappings comparable to human-generated ones.
Exact distribution of split conformal prediction coverage found.
problem Determining the reliability of prediction sets in batch mode.
method Analysis of exchangeable data to find universal distribution of empirical coverage.
result Exact distribution of empirical coverage is universal and determined by nominal miscoverage level and calibration sample size.
Backward Conformal Prediction offers flexible control over prediction set sizes while ensuring coverage guarantees.
problem Providing reliable prediction sets with controlled sizes in applications like medical diagnosis.
method Defines a rule that constrains prediction set sizes based on observed data, adapting coverage levels.
result Maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.
New method predicts and optimizes matrix recovery from noisy measurements.
problem Recovering rank-1 matrices from Gaussian measurements with noise.
method Stochastic prox-linear iterative algorithm with trajectory predictions.
result The method converges linearly with accurate predictions of error.
TIDBD adapts step sizes online for better robotic predictions.
problem Choosing appropriate learning parameters for online prediction-learning.
method Temporal-Difference Incremental Delta-Bar-Delta (TIDBD) for step-size adaptation.
result TIDBD performs comparably to classic TD learning and detects sensor failures.
Conformal Prediction is a machine learning methodology that produces valid prediction regions under mild conditions. In this paper, we explore the application of making predictions over multiple data sources of different sizes without disclosing data between the sources. We propose that each data source applies a trans…
Improved online prediction with guaranteed coverage.
problem Creating reliable online predictions for arbitrary sequences.
method Online conformal prediction with decaying step sizes.
result Substantially improved practical properties, including close coverage at every time point.
Study finds that only a fraction of data is needed for accurate patient-level prediction models.
problem Developing predictive models for patient-level outcomes using large observational data.
method Empirical assessment of sample size effects on model performance and complexity using learning curves.
result A median reduction of 9.5% to 78.5% in the number of observations and 8.6% to 68.3% in the number of predictors can be achieved with adequate sample size.
DPSM minimizes prediction set size by integrating conformal principles into deep classifier training.
problem Large prediction sets from standard conformal methods are impractical.
method Formulates conformal training as bilevel optimization, proposing DPSM algorithm.
result Significantly reduces prediction set size compared to prior methods.
Machine learning outperforms statistical methods with larger data sets.
problem Lower predictive performance of machine learning methods compared to statistical methods under low sample size.
method Learning curve method to analyze predictive performance across different sample sizes.
result Machine learning methods improve their predictive performance as sample size increases.
pmsims R package uses Gaussian process for flexible sample size estimation in clinical models.
problem Determining adequate sample size for clinical prediction models.
method Simulation-based Gaussian process search for flexible sample size estimation.
result Gaussian process-based method produces more stable sample size estimates, especially in challenging settings.
This work introduces COLA, a strategy to aggregate conformal prediction sets efficiently.
problem Efficiently combining multiple conformity scores to reduce prediction set size.
method Introduces COnfidence-Level Allocation (COLA) to optimally allocate confidence levels across sets.
result COLA achieves smaller prediction sets than state-of-the-art methods while maintaining valid coverage.
New analysis reveals batch size effects on stochastic conditional gradient methods.
problem Understanding the role of batch size in stochastic conditional gradient methods.
method Deriving a new analysis focusing on momentum-based stochastic conditional gradient algorithms (e.g., Scion).
result Increasing batch size initially improves optimization accuracy but can degrade performance beyond a critical threshold.
Support vector regression (SVR) has been widely used to reduce the high computational cost of computer simulation. SVR assumes the input parameters have equal sample sizes, but unequal sample sizes are often encountered in engineering practices. To solve this issue, a new prediction approach based on SVR, namely as hig…
New model learns multisets to predict containment and sizes of differences.
problem Learning permutation invariant representations for flexible containment.
method Formalize multisets, propose training on predicting symmetric difference sizes.
result Model outperforms DeepSets on predicting containment and sizes of symmetric differences.
This paper uses supervised learning to predict optimal chunk-size for parallel linear algebra operations.
problem Finding the optimal chunk-size for parallel linear algebra operations.
method The paper uses supervised learning models (logistic regression, neural networks, decision trees) to predict the optimal chunk-size for multiple linear algebra operations.
result The custom decision tree model outperforms classical decision trees and other models in predicting optimal chunk-size for linear algebra operations.
Paper proposes a cost-sensitive conformal training method with provably controllable learning bounds.
problem Uncertainty quantification and learning bounds in conformal prediction.
method Cost-sensitive conformal training algorithm that minimizes the expected size of prediction sets using rank weighting.
result Theoretical analysis shows tightness between weighted objective and expected size of conformal prediction sets.
Deep learning predicts fit for fashion e-commerce.
problem Predicting correct fit for customer satisfaction and cost reduction.
method Deep learning content-collaborative approach using customer and article embeddings.
result Significant improvement over state-of-the-art methods.
New bounds on efficiency for conformalized regression methods.
problem Efficiency of conformal prediction in regression models.
method Non-asymptotic bounds on prediction set length for conformalized quantile and median regression.
result Identifies phase transitions in convergence rates across different regimes of miscoverage level.
Downsampling can improve generalization in ridgeless linear regression, especially with optimal sketching size.
problem Improving generalization in ridgeless linear regression with limited data.
method Investigating the effects of downsampling on the sketched ridgeless least square estimator in the proportional regime.
result Optimal sketching size minimizes out-of-sample prediction risks and stabilizes risk curves.
Paper develops an online learning algorithm for functional data models.
problem Recovering slope functions or predictors in functional data models.
method Online regularized learning algorithm in reproducing kernel Hilbert spaces with polynomially decaying step-size.
result Established fast convergence rates for estimation error without capacity assumption.
Bayesian model predicts online activity participation.
problem Predicting the number of new users initiating an activity.
method Simple Bayesian approach for online activity sample sizes.
result Effective in predicting sample size for online experiments.
Learn2Evaluate uses learning curves to estimate high-dimensional prediction performance.
problem Estimating test performance in high-dimensional data settings is challenging.
method Learn2Evaluate uses learning curves to estimate test performance at the total sample size.
result Learn2Evaluate provides a lower confidence bound for performance estimation.
In the information-based paradigm of inference, model selection is performed by selecting the candidate model with the best estimated predictive performance. The success of this approach depends on the accuracy of the estimate of the predictive complexity. In the large-sample-size limit of a regular model, the predicti…
Increasing the batch size is a popular way to speed up neural network training, but beyond some critical batch size, larger batch sizes yield diminishing returns. In this work, we study how the critical batch size changes based on properties of the optimization algorithm, including acceleration and preconditioning, thr…
In this paper, we introduce a method for adapting the step-sizes of temporal difference (TD) learning. The performance of TD methods often depends on well chosen step-sizes, yet few algorithms have been developed for setting the step-size automatically for TD learning. An important limitation of current methods is that…
New model predicts financial tail events using RIA-EVT-Copula.
problem Predicting financial tail events for risk management.
method RIA-EVT-Copula framework combining POT, RIA, and copulas.
result Improved accuracy in predicting financial extremes.
Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.
problem Characterizing and optimizing ridge ensembles in proportional feature-to-sample size regimes.
method Proportional asymptotics analysis, GCV for tuning, proving risk equivalence.
result Risk of optimal full ridgeless ensemble matches optimal ridge predictor's risk.
JUCAL jointly calibrates aleatoric and epistemic uncertainties in classifier ensembles.
problem Misrepresentation of predictive uncertainty due to unbalanced aleatoric and epistemic uncertainties.
method Joint Uncertainty Calibration (JUCAL) that jointly calibrates two constants to weight and scale uncertainties.
result Significantly outperforms state-of-the-art calibration methods across various text classification tasks.
Predicts optimal training dataset sizes per class for machine learning models.
problem Optimizing training dataset sizes for class-specific machine learning models.
method Algorithm based on space-filling design of experiments, models like powerlaw curves and generalized linear models.
result The algorithm predicts optimal training dataset sizes per class for improved model performance.
Model predicts neural network performance scaling laws across various factors.
problem Understanding the performance of neural networks across different training factors.
method Random feature model trained with gradient descent, analyzing compute-optimal scaling laws.
result Predicts asymmetric compute-optimal scaling rule and behavior of training and test loss gap.
Revisits granular models explaining firm growth rates and sizes.
problem Understanding the relationship between firm size and growth rate statistics.
method Developed new theoretical insights linking firm size and growth rate statistics within granular models.
result Growth volatility distribution is size-independent but fat-tailed, challenging granular models.
This work investigates power laws in deep neural network ensembles and predicts their performance.
problem Understanding the performance of deep neural network ensembles and their optimal structure.
method Investigated the behavior of negative log-likelihood (CNLL) of a deep ensemble as a function of ensemble size and member network size, identifying power law dependencies.
result One large network may perform worse than an ensemble of several medium-size networks, known as a memory split.
Develops a new model to predict training dynamics of large language models.
problem Lack of mechanistic understanding of training dynamics in large language models.
method A first-principles reduced-order model of training dynamics, predicting group-size invariance and stability thresholds.
result Closed-form model predicts training dynamics with high accuracy and provides new diagnostics.
Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on e…
This paper investigates different vector step-size adaptation approaches for non-stationary online, continual prediction problems. Vanilla stochastic gradient descent can be considerably improved by scaling the update with a vector of appropriately chosen step-sizes. Many methods, including AdaGrad, RMSProp, and AMSGra…
A new scaling law predicts optimal batch size for training models.
problem Finding the optimal batch size for training models efficiently.
method Proposed a three-term scaling law that considers model size, training data, training steps, and batch size.
result The three-term law accurately recovers the optimal batch size and can be robustly fit with fewer training runs.
Two methods reduce BN and DNN complexity, balancing size and accuracy.
problem Balancing model size and prediction accuracy in Bayesian networks and deep neural networks.
method Quantization-aware training and tree-augmented naive Bayes structure learning extension.
result Pareto optimal models found for small-scale scenarios.
The paper analyzes bagging in overparameterized learning, deriving risk properties and optimal subsample sizes.
problem Characterizing the risk of bagged predictors in overparameterized settings.
method General strategy using classical results on simple random sampling, specialized for ridge and ridgeless predictors.
result Derives exact asymptotic risk of bagged ridge and ridgeless predictors under various conditions.
New approach to fairness in machine learning models using conformal prediction.
problem Fairness in machine learning models' downstream decision-making.
method Theoretical derivation and empirical evaluation of label-clustered conformal prediction.
result Label-clustered conformal prediction often provides a favorable balance between utility and substantive fairness.
Behavioral theories posit that investor sentiment exhibits predictive power for stock returns, whereas there is little study have investigated the relationship between the time horizon of the predictive effect of investor sentiment and the firm characteristics. To this end, by using a Granger causality analysis in the …
ECV method optimizes ensemble parameters for randomized ensembles.
problem Efficient tuning of ensemble parameters in randomized ensembles.
method ECV (Extrapolated Cross-Validation) method for tuning ensemble and subsample sizes.
result ECV yields δ-optimal ensembles for squared prediction risk.
Machine learning predicts dam-break flood wave behavior accurately.
problem Predicting long-term wave behavior in dam-break floods.
method Solved Saint-Venant equations using Lax-Wendroff scheme, trained RC-ESN with flow depth data.
result RC-ESN model predicts 286 time-steps ahead with RMSE < 0.01, outperforming LSTM.
We reveal a model rank that predicts successful recovery of target functions at overparameterization.
problem Understanding the mysterious good generalization performance of overparameterized nonlinear models.
method Rank stratification and linear stability theory for general nonlinear models.
result Linearly stable functions are preferred by nonlinear training, and model rank predicts minimal training data size.
Derives a size premium from automated market makers in decentralized AI subnets.
problem Determining the profitability and risk of decentralized AI subnets.
method Analyzes daily data on 128 subnets, tests the size premium, and calculates transaction costs.
result The size premium is reduced by a halving of token emissions but remains profitable only below a certain asset threshold.
Bob predicts a future observation based on a sample of size one. Alice can draw a sample of any size before issuing her prediction. How much better can she do than Bob? Perhaps surprisingly, under a large class of loss functions, which we refer to as the Cover-Hart family, the best Alice can do is to halve Bob's risk. …
A new convex loss function optimizes set predictions with balanced size and coverage.
problem Optimizing set predictions with balanced size and coverage.
method Proposes a convex loss function using Choquet integrals for nondecreasing subset-valued functions.
result Optimal trade-offs between conditional probabilistic coverage and set size.