Study optimizes step size for Metropolis algorithm in non-identifiable cases.
problem Optimizing step size for Metropolis algorithm in non-identifiable models.
method Analytical derivation of average acceptance rate for non-identifiable cases.
result Developed optimization principle for step size based on average acceptance rate.
INNs improve acceptance rates in electron spectra analysis.
problem Analyzing electron spectra from near-critical laser-plasmas.
method Invertible Neural Networks (INNs) for forward and inverse modeling.
result INNs significantly increase acceptance rates up to a factor of 10.
The Kelly rule fails to maximize growth in a time-changed return setting.
problem Performance of the Kelly rule in a time-changed return process.
method Investigated the Kelly rule in a semi-martingale setting with a time change process.
result The Kelly rule does not maximize average growth rate in a non-normal log-return setting.
This paper tackles worst-class error rate in classification tasks.
problem Minimizing worst-class error rate in classification tasks, especially in medical image classification.
method Designing a boosting approach to bound the worst-class error rate using Deep Neural Networks (DNNs).
result The proposed boosting approach lowers worst-class test error rates while avoiding overfitting.
Improved sampling for Bayesian neural networks reduces vanishing acceptance rates and increases predictive accuracy.
problem Sampling inefficiency in Bayesian neural networks, especially with deep architectures and large datasets.
method Approximate blocked Gibbs sampling to partition and sample subgroups of parameters.
result Increased predictive accuracy and quantification of predictive uncertainty in classification tasks.
Indices of acceptability are well suited to frame the axiomatic features of many performance measures, associated to terminal random cash flows.We extend this notion to classes of càdlàg processes modelling cash flows over a fixed investment horizon.We provide a representation result for bounded paths. We suggest an ac…
Study reveals bias in machine learning conference reviews.
problem Bias in machine learning conference review process.
method Comprehensive analysis of ICLR papers from 2017-2020.
result Strong institutional bias in accept/reject decisions.
Measures financial resilience using BSDEs and their properties.
problem Measuring financial resilience in dynamic risk environments.
method Developed stochastic calculus for BSDEs with jumps, revealing resilience rate as expectation of generator.
result Resilience rate can be represented as expectation of BSDE generator, revealing properties of dynamic risk measures.
Model uses Preisach hysteresis to predict gig worker acceptance, reducing costs and improving fill rates.
problem Predicting and optimizing gig worker acceptance in labor markets.
method Preisach hysteresis model applied to neural network and XGBoost classifier for binary transaction outcomes.
result Model reduces total wage bill by 21.3% and increases expected fill rate by 9.7 pp.
Conventional Monte Carlo simulations are stochastic in the sense that the acceptance of a trial move is decided by comparing a computed acceptance probability with a random number, uniformly distributed between 0 and 1. Here we consider the case that the weight determining the acceptance probability itself is fluctuati…
Cactus improves auto-regressive decoding speed without sacrificing quality.
problem Accelerating auto-regressive decoding while maintaining output quality.
method Formalizes speculative sampling as constrained optimization and proposes Cactus for controlled divergence from the verifier distribution.
result Empirically validated effectiveness across various benchmarks.
Research examines motivations and factors influencing retailers' payment method choices.
problem Understanding motivations and factors affecting retailers' payment method choices.
method Qualitative and quantitative analysis of various factors including regulatory constraints, merchant service providers, and demographic variables.
result Lower interchange fees and regulatory constraints make card payment adoption financially feasible for merchants.
New algorithm MTMC reduces MCMC evaluation costs.
problem High-dimensional sampling with intractable posterior evaluations.
method Iteratively updated approximation of posterior distribution for acceptance rate.
result Approximation converges to true posterior as iterations increase.
Markov chain Monte Carlo (MCMC) algorithms are widely used to sample from complicated distributions, especially to sample from the posterior distribution in Bayesian inference. However, MCMC is not directly applicable when facing the doubly intractable problem. In this paper, we discussed and compared two existing solu…
Large learning rates lead to optimal generalization if chosen carefully.
problem Understanding the optimal range of large learning rates for neural network training.
method Empirical study focusing on two questions: optimal initial LR range and differences between models trained with different LRs.
result Optimal initial learning rates slightly above the convergence threshold lead to optimal results after fine-tuning with a small LR or weight averaging.
Proposes a method to estimate acceptance regions for many classes, including new ones.
problem Lack of methods to handle new classes in set-valued classification.
method Generalized Prediction Set (GPS) approach to estimate acceptance regions.
result Achieves a good balance between accuracy, efficiency, and anomaly detection.
Optimal scaling for proximal MALA in high dimensions confirmed.
problem Optimizing sampling efficiency in high-dimensional target densities.
method Introduced and analyzed the proximal MALA algorithm, showing it maintains optimal scaling.
result Proximal MALA achieves optimal scaling in high dimensions with an average acceptance probability of 0.574.
Accelerating Speculative Diffusions via Block Verification
problem Adapting speculative decoding for continuous diffusion models
method Introducing a novel speculative sampling mechanism for diffusion models
result Improves acceptance rate and speeds up inference
Modern deep neural network training is typically based on mini-batch stochastic gradient optimization. While the use of large mini-batches increases the available computational parallelism, small batch training has been shown to provide improved generalization performance and allows a significantly smaller memory footp…
The study proposes a framework to accept OOD data based on competence scores.
problem Silent failures in Domain Generalization where models reject OOD data without proper justification.
method A learning to reject framework using proxy incompetence scores to predict trustworthiness.
result Increasing incompetence scores are predictive of reduced accuracy, but not always favorable for accuracy/rejection trade-off.
The principle of peer review is central to the evaluation of research, by ensuring that only high-quality items are funded or published. But peer review has also received criticism, as the selection of reviewers may introduce biases in the system. In 2014, the organizers of the ``Neural Information Processing Systems\r…
Online method selects candidates from data streams, ensuring irreversible decisions.
problem Conformal selection's incompatibility with irreversible decisions in online scenarios.
method Online Conformal Selection with Accept-to-Reject Changes (OCS-ARC) incorporating online Benjamini-Hochberg procedure.
result OCS-ARC controls FDR at or below nominal level, improving selection power.
Paper predicts embryo implantation probability from IVF time-lapse imaging.
problem Manual embryo selection in IVF has low success rates.
method Data-driven system trained on time-lapse imaging videos.
result Algorithm improves positive and negative predictive values.
Bayesian mixture models are widely applied for unsupervised learning and exploratory data analysis. Markov chain Monte Carlo based on Gibbs sampling and split-merge moves are widely used for inference in these models. However, both methods are restricted to limited types of transitions and suffer from torpid mixing and…
A method for making predictions with a reject option using conformal prediction.
problem Uncertainty in machine learning predictions, especially when models are unsure.
method Formalizing ML with reject option, using conformal prediction for distribution-free error guarantees.
result Theoretical guarantees on error rate for prediction sets with distribution-free validity.
Study controls error rates of binary classifiers using hypothesis testing.
problem Traditional binary classifiers have uncontrolled error rates.
method Combines binary classification with statistical hypothesis testing.
result Trained classifiers can be made to meet target error rate thresholds.
Unordered feature sets are a nonstandard data structure that traditional neural networks are incapable of addressing in a principled manner. Providing a concatenation of features in an arbitrary order may lead to the learning of spurious patterns or biases that do not actually exist. Another complication is introduced …
We address the M-best-arm identification problem in multi-armed bandits. A player has a limited budget to explore K arms (M<K), and once pulled, each arm yields a reward drawn (independently) from a fixed, unknown distribution. The goal is to find the top M arms in the sense of expected reward. We develop an algorithm …
Averaged SGD achieves optimal convergence rate for neural networks in the NTK regime.
problem Convergence analysis of averaged stochastic gradient descent for neural networks.
method Analyzed convergence of averaged stochastic gradient descent for overparameterized two-layer neural networks.
result Achieved minimax optimal convergence rate with global convergence guarantee.
New star-shaped acceptability indexes generalize existing methods.
problem Generalizing existing acceptability measures.
method Characterizing acceptability indexes through star-shaped risk measures and sets.
result Introducing concrete examples linked to various financial measures.
The study introduces anytime learning schedules for large language models without fixed horizons.
problem Training large language models without knowing the total training horizon.
method Theoretical analysis and weight averaging to create anytime learning schedules.
result Theoretical and empirical evidence shows that weight averaging with simple step sizes can achieve comparable final loss to well-tuned cosine schedules.
We introduce an autoregressive-type model with self-modulation effects for a foreign exchange rate by separating the foreign exchange rate into a moving average rate and an uncorrelated noise. From this model we indicate that traders are mainly using strategies with weighted feedbacks of the past rates in the exchange …
Sample-Rank simplifies MO recommendations by sampling and ranking, improving revenue with stable conversion rates.
problem Multi-objective recommendations in online food ordering systems.
method Multi-goal sampling followed by ranking, reducing MO problem to LTR model.
result Significant lift in revenue (2.64%) with stable conversion rates, no drop in last-mile traversal.
Improved learning rates with new smoothness measure.
problem Learning with noisy data and unknown function class.
method Generalized Hölder smoothness to average smoothness, proving upper and lower bounds.
result Achieved nearly optimal learning rates in realizable and agnostic settings.
Studies acceptable bundles on a partially punctured polydisk.
problem Understanding acceptable bundles in Simpson--Mochizuki theory.
method Expository study with new arguments.
result New arguments differ from Mochizuki's.
In this paper we investigate the scaling behavior of the average daily exchange rate returns of the Indian Rupee against four foreign currencies namely US Dollar, Euro, Great Britain Pound and Japanese Yen. Average daily exchange rate return of the Indian Rupee against US Dollar is found to exhibit a persistent scaling…
Study on acceptable bundles on a punctured disk.
problem Understanding acceptable bundles in Simpson--Mochizuki theory.
method Detailed study and introduction of a new invariant.
result Arguments differ from Simpson and Mochizuki's.
Improved TD learning with tail averaging and regularization achieves optimal convergence rates.
problem Convergence analysis of TD learning with linear function approximation.
method Tail-averaging and regularization applied to TD learning algorithm.
result Achieves optimal O(1/t) convergence rate in expectation and with high probability. SGDM accelerates faster than SGD with large batch sizes and permits broader learning rates.
problem Understanding the role of momentum in SGDM and its convergence rates.
method Analysis of SGDM convergence rates under strongly convex settings, including finite-sample rates and asymptotic normality of the averaged estimator.
result SGDM converges faster than SGD with large batch sizes and permits broader learning rates.
Many problems in machine learning and game theory can be formulated as saddle-point problems, for which various first-order methods have been developed and proven efficient in practice. Under the general convex-concave assumption, most first-order methods only guarantee an ergodic convergence rate, that is, the uniform…
ABC method uses machine learning for likelihood-free inference.
problem Statistical inference in simulator-based models with intractable likelihoods.
method Direct comparison of empirical distributions via KL divergence estimator and contrastive learning.
result Asymptotic normality of ABC posterior distributions with properly scaled exponential kernel.
Simple conditions for comonotonic additive risk measures from acceptance sets.
problem Conditions for comonotonic additive risk measures from acceptance sets.
method Conditions on acceptance sets for induced comonotonic additive risk measures.
result Acceptance sets induce comonotonic additive risk measures if and only if the acceptance sets and their complements are stable under convex combinations of comonotonic random variables.
We introduce a simple algorithm, True Asymptotic Natural Gradient Optimization (TANGO), that converges to a true natural gradient descent in the limit of small learning rates, without explicit Fisher matrix estimation. For quadratic models the algorithm is also an instance of averaged stochastic gradient, where the par…
The paper develops a theory for speculative decoding acceptance criteria.
problem Speculative decoding's acceptance criteria and their rejection regions.
method Characterization of rejection regions as lower level sets of the target distribution, derivation of exact and margin-based certificates.
result Relaxed and tree-based acceptance criteria substantially enlarge the region of certified acceptance.
Enhances RJMCMC efficiency with non-linear transport-based proposals.
problem Designing efficient RJMCMC proposals for complex models.
method Applies non-linear transport-based approach to construct efficient transdimensional jumps.
result Acceptance probability depends only on model probabilities when exact transports are used.
SGD (Stochastic Gradient Descent) is a popular algorithm for large scale optimization problems due to its low iterative cost. However, SGD can not achieve linear convergence rate as FGD (Full Gradient Descent) because of the inherent gradient variance. To attack the problem, mini-batch SGD was proposed to get a trade-o…
We formulate and study a general family of (continuous-time) stochastic dynamics for accelerated first-order minimization of smooth convex functions. Building on an averaging formulation of accelerated mirror descent, we propose a stochastic variant in which the gradient is contaminated by noise, and study the resultin…
Simplified analysis of SGD for linear regression with weight averaging.
problem Understanding SGD optimization in linear regression models.
method Simplified analysis using linear algebra tools, bypassing complex operator manipulations.
result Recovery of bias and variance bounds for SGD in linear regression.