Optimal algorithms for online convex optimization with random order.
problem Online convex optimization with random order and non-convex loss functions.
method Stochastic gradient descent and algorithmic stability analysis.
result Achieves optimal bounds and significantly outperforms previous methods.
Transforms offline algorithms to online with low regret in random order model.
problem Developing online algorithms with low approximate regret from offline approximation algorithms.
method General reduction theorem and coreset construction method.
result Achieves polylogarithmic ε-approximate regret for various online problems.
New method achieves small-loss regret bounds in random-order model.
problem Online learning with adversarial loss functions in random order.
method Extending batch-to-online transformation, using average sensitivity and stability.
result Small-loss regret bounds of order ildeO(φ⋆(OPTT)). MDMs train to decode tokens in a random order, which affects performance; we show they can be optimized for a favorable order.
problem Performance of MDMs is affected by the random order in which tokens are decoded.
method We show that MDMs can be optimized for a favorable decoding order by equipping their continuous-time variational objective with multivariate noise schedules.
result MDMs can be decomposed into a weighted auto-regressive losses over orders, making them auto-regressive models with learnable orders.
Self-directed learners can minimize mistakes in online classification.
problem Minimizing mistakes in online classification with adaptive prediction order.
method Designing efficient self-directed learners for linear classification.
result Strong separation between worst-order and random-order learning for linear classification.
New algorithm for clustering data streams with no substitutions.
problem Clustering sequential data with no point substitution.
method Careful risk estimation, linear bin division, multiscale center selection.
result First constant approximation algorithm for random-order k-median clustering.
Improved rates for continual learning using SGD and last-iterate analysis.
problem Forgetting in overparameterized models after fitting multiple tasks.
method Developed novel SGD upper bounds for continual linear models and analyzed their performance.
result Established universal forgetting rates for continual learning.
We define a stochastic model of a two-sided limit order book in terms of its key quantities \textit{best bid [ask] price} and the \textit{standing buy [sell] volume density}. For a simple scaling of the discreteness parameters, that keeps the expected volume rate over the considered price interval invariant, we prove a…
We use standard physics techniques to model trading and price formation in a market under the assumption that order arrival and cancellations are Poisson random processes. This model makes testable predictions for the most basic properties of a market, such as the diffusion rate of prices, which is the standard measure…
New algorithms reduce regret for online submodular maximization under various conditions.
problem Online optimization of submodular functions with adversarial or random utilities.
method Characterized strongly DR-submodular functions and derived bounds for different utility classes.
result Logarithmic regret bounds for adversarial strongly DR-submodular functions and submodular functions with random order.
Optimizes trade execution with reinforcement learning for limit orders.
problem Maximizing revenue in a limit order book with market and limit orders.
method Formulated as a dynamic allocation task, uses multivariate logistic-normal distributions for efficient training.
result Outperforms traditional strategies in simulated environments.
We propose a model for price formation in financial markets based on clearing of a standard call auction with random orders, and verify its validity for prediction of the daily closing price distribution statistically. The model considers random buy and sell orders, placed following demand- and supply-side valuation di…
Most deep learning models are based on deep neural networks with multiple layers between input and output. The parameters defining these layers are initialized using random values and are "learned" from data, typically using stochastic gradient descent based algorithms. These algorithms rely on data being randomly shuf…
Offline k-means clustering was studied extensively, and algorithms with a constant approximation are available. However, online clustering is still uncharted. New factors come into play: the ordering of the dataset and whether the number of points, n, is known in advance or not. Their exact effects are unknown. In this…
Most modern financial markets use a continuous double auction mechanism to store and match orders and facilitate trading. In this paper we develop a microscopic dynamical statistical model for the continuous double auction under the assumption of IID random order flow, and analyze it using simulation, dimensional analy…
Many tasks in machine learning and data mining, such as data diversification, non-parametric learning, kernel machines, clustering etc., require extracting a small but representative summary from a massive dataset. Often, such problems can be posed as maximizing a submodular set function subject to a cardinality constr…
New algorithms achieve optimal regret in sliding window model with limited memory.
problem Experts problem in the sliding window model with limited information.
method 2 queries, polylog(nT) memory, exponential improvement on memory.
result Achieve optimal regret of sqrt(nW)polylog(nT) with 2 queries and polylog(nT) memory.
New formulas for feature importance tests in regression models.
problem Identifying important features in regression models.
method Established formulas for AUC criteria and proposed alternative metrics.
result Integrated Gradients (IG) performs nearly as well as Kernel SHAP (KS) but is faster.
User surveys for Quality of Experience (QoE) are a critical source of information. In addition to the common "star rating" used to estimate Mean Opinion Score (MOS), more detailed survey questions (problem tokens) about specific areas provide valuable insight into the factors impacting QoE. This paper explores two aspe…
Study shows visual feedback and monetary incentives reduce plugload energy consumption in commercial buildings.
problem Mitigating energy consumption in commercial buildings through occupant plugload control.
method Field experiments with visual feedback and monetary incentives in government and university buildings.
result Mean energy reduction of ~9.52% in office environments and ~21.61% in university environments with visual feedback.
New findings on tensor decomposition complexity, showing polynomial functions can estimate the largest component under certain conditions.
problem The complexity of tensor decomposition, especially for low-degree polynomials.
method Modeling a slightly larger component in a random tensor decomposition and using polynomial functions to estimate it.
result Polynomial functions can accurately estimate the largest component when r≪n3/2 but fail when r≫n3/2. Curriculum learning shows marginal benefits over random ordering on standard datasets.
problem Investigating the benefits of curriculum learning over random ordering in training neural networks.
method Experiments with curriculum, anti-curriculum, and random-curriculum orderings on various datasets and conditions.
result Curriculum learning only marginally improves performance on standard datasets and is outperformed by random ordering.
An online framework optimizes efficiency in conformal prediction with a target miscoverage rate.
problem Achieving coverage and minimizing interval length in a sequential, online setting.
method Optimizes efficiency by directly optimizing the average length of intervals while maintaining coverage.
result Shows a gap between optimal performance for exchangeable and arbitrary sequences, and provides a matching algorithm for the Pareto-optimal settings.
The paper introduces BCART models for aggregate claim amount, improving frequency-severity and joint modeling.
problem Modeling aggregate claim amount with frequency-severity and joint dependencies.
method Developed three types of BCART models: frequency-severity, sequential, and joint models. Used various distributions for claim severity data.
result Weibull distribution outperforms gamma and lognormal for right-skewed, heavy-tailed claim severity data.
The paper uses model-based trees to create interpretable surrogate models for complex machine learning models.
problem Interpreting complex machine learning models.
method Using model-based trees to partition feature space and create interpretable models.
result Model-based trees generate optimal surrogate models that balance interpretability and performance.
Gauge Flow Models use a learnable Gauge Field in Generative Flow Models.
problem Improving generative model performance.
method Integrates a learnable Gauge Field into Flow ODEs.
result Gauge Flow Models outperform traditional Flow Models in Flow Matching experiments.
The study examines how model predictions hold up under model extensions.
problem Model predictions may not be robust under model extensions, limiting their applicability.
method The study uses causal ordering to assess robustness of qualitative model predictions and characterizes model extensions that preserve predictions.
result Conditions and techniques are provided to assess robustness of model predictions under model extensions.
Revises Bayesian model averaging for foundation models.
problem Ensemble pre-trained and lightly-finetuned foundation models for improved classification performance.
method Introduces trainable linear classifiers and computationally cheaper model averaging scheme (OMA).
result Ensembled models can better predict on various datasets.
Paper introduces symmetric divergence link models for probability distributions.
problem Symmetric divergence measures for probability distributions.
method Two general classes of link models: one for survival functions and another for cumulative probability distribution functions.
result Advantages of symmetric divergence measures over asymmetric measures for model averaging and feature assessment.
New method to handle credit portfolio model uncertainties.
problem Model risk in credit portfolio models.
method Demonstrates comprehensive yet easy-to-implement approach to uncertainty in model parameters.
result Comprehensive method to deal with model uncertainties.
The paper tests stock return models and uses LSTM to predict stock returns.
problem Validating stock return models and predicting stock returns.
method Used Fama-French three-factor, four-factor, and five-factor models; also used LSTM model.
result Fama-French five-factor model shows better validity for stock returns.
Researchers review challenges in interpreting additive models, especially neural additive models.
problem Challenges in interpreting additive models, particularly neural additive models.
method Review of generalized additive models and discussion of nonidentifiability.
result Challenges in claiming interpretability or suitability for safety-critical applications of additive models.
Novel hybrid modeling combines ML and physics for real-time diagnosis.
problem Real-time diagnosis of complex systems.
method Combines machine learning and physics-based models to create reduced-order models.
result Generated models are two orders of magnitude simpler, improving efficiency.
CRS model improves ranking data modeling with theoretical guarantees.
problem Lack of rich, multimodal models for ranking data.
method Contextual Repeated Selection (CRS) model for multimodal ranking data.
result CRS model significantly outperforms existing methods in various ranking contexts.
Sigma models linked to Gross-Neveu models via quiver varieties.
problem Understanding the relationship between sigma models and Gross-Neveu models.
method Exploring the mathematical correspondence between sigma models and Gross-Neveu models, including their geometric and trigonometric/elliptic deformations.
result Sigma models are mathematically equivalent to Gross-Neveu models under certain conditions.
Interpretable machine learning has become a strong competitor for traditional black-box models. However, the possible loss of the predictive performance for gaining interpretability is often inevitable, putting practitioners in a dilemma of choosing between high accuracy (black-box models) and interpretability (interpr…
Simple models are preferred over complex models, but over-simplistic models could lead to erroneous interpretations. The classical approach is to start with a simple model, whose shortcomings are assessed in residual-based model diagnostics. Eventually, one increases the complexity of this initial overly simple model a…
Matryoshka hides secret models in a carrier model, achieving high capacity and robustness.
problem Stealing functionality of private ML data by hiding models in a carrier model.
method Parameter sharing approach exploiting the learning capacity of the carrier model.
result Hides a 26x larger secret model or 8 secret models in the carrier model.
Eigen-stratified models reduce model size and improve performance.
problem Large model size in Laplacian-regularized stratified models.
method Formulate eigen-stratified models with linear combinations of bottom eigenvectors of the graph Laplacian.
result Significant reduction in model size with eigen-stratified models.
Seq2Seq models speed up epidemic model predictions.
problem Complex epidemic models are computationally expensive.
method Used deep seq2seq models as surrogates for complex models.
result Surrogates predict scenarios up to several thousand times faster.
This work develops scalable model selection methods with fast update and selection.
problem Efficient model selection for large pools of candidate models.
method Isolated model embedding, which supports asymptotically fast update and selection.
result Standardized Embedder achieves competitive model selection performances.
Paper proposes BMPO to optimize policies using bidirectional models.
problem Model-based reinforcement learning's reliance on forward model accuracy.
method Develops BMPO using both forward and backward models for policy optimization.
result BMPO outperforms state-of-the-art methods in sample efficiency and asymptotic performance.
Copulas outperform marginal models in multivariate risk forecasting, reducing model risk by narrowing down the set of models.
problem Model risk in multivariate risk forecasting, especially during crises.
method Comprehensive empirical study comparing Copula-GARCH models with fixed marginals, copulas, or neither.
result Model risk is almost entirely due to copula choice, not marginal models.
BayesBlend blends multiple models' predictions for better insurance loss predictions.
problem Improving insurance loss predictions by combining multiple models.
method Pseudo-Bayesian model averaging, stacking, and hierarchical stacking.
result BayesBlend provides a user-friendly way to blend model predictions and estimate weights.
The paper identifies when larger models improve predictions and proposes a switcher model.
problem Understanding when larger models benefit from added complexity.
method Numerical studies on T5 architecture to analyze predictive uncertainty and model performance.
result Large models improve on examples where small models are uncertain, but not on certain examples.
Improved diffusion model generation speed with speculative sampling.
problem Generating samples from computationally expensive diffusion models.
method Extending speculative sampling to diffusion models, using fast draft models for candidate token generation.
result Significant speedup in generation, halving the number of function evaluations.
We propose a generalization of neural network sequence models. Instead of predicting one symbol at a time, our multi-scale model makes predictions over multiple, potentially overlapping multi-symbol tokens. A variation of the byte-pair encoding (BPE) compression algorithm is used to learn the dictionary of tokens that …
The paper extends statistical inference methods for black-box generative models.
problem Understanding and validating black-box generative models without access to their internal data.
method Develops model-level statistical inference tasks using generative model representations.
result Model-level representations are effective for multiple inference tasks.