Random permutations can offer faster convergence than with-replacement sampling for some functions.
problem Understanding when and how random permutations outperform with-replacement sampling in SGD convergence.
method Analyzing convergence rates for different function classes (1D strongly convex, general strongly convex, quadratic strongly convex).
result The optimal convergence gap between random and permutation-based SGD varies from exponential to nonexistent, depending on the function class.
New method optimizes hyperparameters for randomized algorithms like random feature regression.
problem Optimizing hyperparameters in randomized algorithms is challenging due to their stochastic nature.
method Introduced a random objective function and used ensemble Kalman inversion (EKI) for gradient-free optimization.
result Demonstrated successful optimization of hyperparameters in various randomized algorithms.
Random exploration optimizes Bayesian optimization with optimal error rates and computational efficiency.
problem Optimizing Gaussian Process models in Bayesian optimization.
method Random sampling from a distribution in an infinite dimensional Hilbert space, with domain shrinking and order-optimal regret guarantees.
result Achieves optimal error rates and computational efficiency in both noise-free and noisy settings.
A new Randomized-Hyperopt method improves XGBoost hyperparameter tuning.
problem Improving the performance of XGBoost through hyperparameter optimization.
method Proposes Randomized-Hyperopt for XGBoost hyperparameter tuning.
result Randomized-Hyperopt outperforms other methods in terms of accuracy and execution time.
Improved neural network training in low-dimensional random bases.
problem Inefficient optimization in large-scale neural networks.
method Re-draw random subspace at each training step, apply independent projections to different network parts.
result Significantly better optimization performance and efficiency.
Derives explicit investment strategy with random endowment.
problem Optimal investment with random endowment in a market.
method Duality arguments to derive explicit expression for optimal strategy.
result Explicit expression for optimal trading strategy exists.
Randomization is minimax-optimal for variance in experimental design, even with structure.
problem Designing optimal randomized experiments for variance minimization.
method Analyzing permutation symmetric and non-symmetric sets of outcomes, proposing inference-constrained MSOD.
result Randomization is minimax-optimal for variance, even with structure, and requires uniformity constraints for Fisher's exact test.
Random Function Descent improves optimization in high dimensions.
problem Lack of effective optimization methods in high-dimensional spaces.
method Introducing a 'random function' framework to optimize classical optimization problems.
result Random Function Descent (RFD) is a scalable optimization method that bridges Bayesian and classical optimization.
The paper solves MMV and MV problems with random coefficients and finds shared optimal strategies.
problem Optimal trading strategies with random market coefficients.
method Backward stochastic differential equations (BSDEs) to find optimal strategies.
result MMV and MV problems share the same optimal portfolio and value under random coefficients.
New random forest method provides optimal rates and confidence bands.
problem Improving random forest regression rates and constructing confidence bands.
method Proposed Ehrenfest centered purely random forests achieve optimal rates; used Gaussian approximation for supremum of empirical processes.
result Explicit asymptotic uniform confidence bands constructed for both random forest types.
Two-stage nonconvex algorithm and convex relaxation both achieve optimal accuracy in noisy blind deconvolution.
problem Solving bilinear systems of equations with random noise under different designs.
method Two-stage nonconvex algorithm and convex relaxation.
result Both methods achieve minimax-optimal accuracy in the presence of random noise.
Random hyperbolic surfaces have nearly optimal spectral gaps.
problem Proving the nearly optimal spectral gap conjecture for random Belyi surfaces.
method Using the Brooks-Makover model, the authors show a spectral gap greater than 1/4 - c/log(n).
result A random hyperbolic surface in the Brooks-Makover model has a spectral gap greater than 1/4 - c/log(n).
Empirical study shows Randomized Signature Methods improve portfolio optimization in financial markets.
problem Drift estimation in non-linear, non-parametric financial markets is challenging.
method Applied Randomized Signature Methods for non-linear, non-parametric drift estimation in multi-variate financial markets.
result Randomized Signature Methods provide features on the same scale and improve portfolio optimization in real-world settings.
A new random forest algorithm improves tree construction for optimal performance.
problem Improving the performance of random forests, especially in complex and smooth scenarios.
method Adaptive split-balancing method using permutation-based splitting criterion.
result Achieves minimax optimality under various Lipschitz and Hölder classes.
Develops a new random forest method for clustered data with improved prediction and inference.
problem Improving prediction and inference accuracy for clustered data with within-cluster dependence.
method Clustered Random Forests, using weighted least squares estimators for leaf predictions.
result Optimal prediction and inference weights vary under covariate shift, necessitating user-chosen weights.
Optimal algorithms for online convex optimization with random order.
problem Online convex optimization with random order and non-convex loss functions.
method Stochastic gradient descent and algorithmic stability analysis.
result Achieves optimal bounds and significantly outperforms previous methods.
Extends RL to random stopping times, improving optimization.
problem Real-world applications with random stopping times.
method Extended RL framework to random stopping times, derived new formulas.
result Improves optimization convergence with new formulas.
We propose random hinge forests, a simple, efficient, and novel variant of decision forests. Importantly, random hinge forests can be readily incorporated as a general component within arbitrary computation graphs that are optimized end-to-end with stochastic gradient descent or variants thereof. We derive random hinge…
Exact optimality achieved in distributed mean estimation with shared randomness.
problem Achieving optimal communication, privacy, and utility tradeoffs in distributed mean estimation.
method Utilization of a rotationally symmetric shared random codebook and a k-closest encoding mechanism. result Proposed mechanism achieves exact optimality for randomly rotated simplex codebook.
Study shows randomized strategies can't be Nash equilibria in markets with transient price impact.
problem Existence of pure Nash equilibria in markets with transient price impact.
method Considered randomized strategies and showed that they cannot be Nash equilibria.
result Nash equilibria cannot contain randomized strategies.
New algorithm selects robust martingale for optimal stopping problems.
problem Optimal stopping problems in stochastic processes.
method Randomized dual martingale minimization algorithm.
result Efficiently selects Doob martingale as close as possible.
We propose accelerated randomized coordinate descent algorithms for stochastic optimization and online learning. Our algorithms have significantly less per-iteration complexity than the known accelerated gradient algorithms. The proposed algorithms for online learning have better regret performance than the known rando…
Study optimal hedging for claims with random weights in discrete time.
problem Optimal hedging for claims with random weights in discrete time.
method Explicit recursive representation of optimal hedging strategy, without ND condition.
result Obtained explicit optimal hedging strategy in a recursive form.
Unified theory of optimal transport for random measures.
problem Statistical uncertainty in optimal transport.
method Constructing L2 over Wasserstein space for random probability measures. result Unified treatment of random optimal transport and principled inference.
The thesis presents a new perspective on high-dimensional optimization.
problem The failure point of classical optimization methods in high dimensions.
method A distributional view of optimization, focusing on random objective functions and Bayesian Optimization.
result The distributional view explains predictable progress in high-dimensional optimization and provides insights into optimal step size control.
New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
Bayes-optimal learning of deep random networks with Gaussian weights is studied.
problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.
result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.
Bayesian optimization outperforms other methods in hyperparameter tuning for reinforcement learning.
problem Finding optimal hyperparameters that generalize across random seeds in reinforcement learning.
method Benchmarked Successive Halving, Random Search, and Bayesian Optimization with and without repetitions on PPO2 algorithms for Cartpole and Inverted Pendulum tasks.
result Bayesian optimization with noise robust acquisition function is the best choice.
We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …
This paper analyzes how randomizing rewards in MBRL can improve performance without being overly optimistic.
problem The gap between theoretical worst-case regret analysis and empirical performance in MBRL.
method Reward randomization in model-based reinforcement learning (MBRL) with kernelized linear regulator (KNR) model.
result Reward randomization guarantees partial optimism and near-optimal worst-case regret.
We systematically investigate the problem of representing Markov chains by families of random maps, and which regularity of these maps can be achieved depending on the properties of the probability measures. Our key idea is to use techniques from optimal transport to select optimal such maps. Optimal transport theory a…
We consider derivative-free black-box global optimization of expensive noisy functions, when most of the randomness in the objective is produced by a few influential scalar random inputs. We present a new Bayesian global optimization algorithm, called Stratified Bayesian Optimization (SBO), which uses this strong depen…
A new tree-based model improves uncertainty estimation in sequential optimization.
problem Improving uncertainty estimation in sequential model-based optimization.
method Proposed a new ensemble of randomized trees (BwO forest) with bagging and oversampling.
result BwO forest outperforms existing tree-based models in various optimization scenarios.
Muons and random optimizers perform similarly, challenging geometric optimization theory.
problem Empirical success of Muon optimizer challenges geometric optimization theory.
method Introducing Freon and Kaon optimizers, demonstrating performance without precise geometric structure.
result Performance of optimizers is controlled by alignment and descent potential, not geometric structure.
Randomized Geometric Algebra for Convex Neural Networks Optimizes Transfer Learning.
problem Training neural networks to global optimality via convex optimization.
method Randomized algorithms in Clifford's Geometric Algebra for hypercomplex vector spaces.
result Convex optimization and geometric algebra improve LLMs' robustness and reliability in transfer learning.
Paper proposes a new financial fraud detection system using improved RF and GBM.
problem Detecting financial fraud with high accuracy and efficiency.
method Introduces GBM-SSRF, a novel model combining GBM and SSRF.
result GBM-SSRF model outperforms traditional methods in fraud detection.
EGORSE optimizes high-dimensional problems using random and supervised embeddings.
problem Efficiently solving computationally expensive high-dimensional optimization problems.
method EGORSE combines random and supervised linear embeddings for adaptive optimization.
result EGORSE outperforms state-of-the-art methods in high-dimensional optimization.
Optimal reinsurance contracts for multiple dependent risks are derived without specific dependency assumptions.
problem Finding optimal reinsurance contracts for multiple dependent risks without assuming their dependency structure.
method Assumes maximal expected utility criterion and independent negotiation of reinsurance for each risk. Derives optimality conditions and shows that under mild assumptions, optimal contracts are classical (non-randomized) type.
result Optimal reinsurance contracts exist and can be classical (non-randomized) type under mild assumptions.
Dictionaries are collections of vectors used for representations of random vectors in Euclidean spaces. Recent research on optimal dictionaries is focused on constructing dictionaries that offer sparse representations, i.e., ℓ0-optimal representations. Here we consider the problem of finding optimal dictionaries …
HF-opt uses Hamiltonian dynamics to optimize functions, achieving accelerated rates with randomized integration time.
problem Optimizing functions efficiently and accelerating convergence rates.
method Randomized Hamiltonian flow (RHF) with accelerated convergence rates.
result RHGD achieves accelerated convergence rates similar to Nesterov's AGD.
Optimal weighted random forests improve prediction accuracy.
problem Unequal prediction performance among random forest trees.
method Proposes 1-step and 2-step optimal weighting algorithms.
result Asymptotically optimal in terms of squared loss and risk.
WHOMP optimizes randomized controlled trials by minimizing subgroup bias.
problem Minimizing subgroup bias in randomized controlled trials.
method Wasserstein Homogeneity Partition (WHOMP) method.
result WHOMP optimally minimizes type I and type II errors in trials.
New random forest variants achieve optimal performance in high dimensions.
problem Handling dependencies between features in high-dimensional data.
method Using oblique splits in random forests with general split directions.
result Achieved minimax optimal convergence rates in arbitrary dimension.
Optimal transport is #P-hard when components are independent, even with approximate solutions.
problem Computational complexity of optimal transport with independent marginals.
method Proved #P-hardness and developed a pseudo-polynomial time approximation algorithm.
result Optimal transport is #P-hard even with independent components and approximate solutions.
We investigate two perturbation approaches to overcome conservatism that optimism based algorithms chronically suffer from in practice. The first approach replaces optimism with a simple randomization when using confidence sets. The second one adds random perturbations to its current estimate before maximizing the expe…
New method for dynamic valuation in markets with random endowments.
problem Dynamic valuation in markets with random endowments.
method Developed new FBSDE systems and established optimality conditions.
result Established necessary and sufficient conditions for optimality.
Study optimal consumption and investment strategies with constraints in a market with random coefficients.
problem Optimal consumption and investment strategies with constraints in a regime switching market with random coefficients.
method Explicit optimal strategies provided via solutions to new BSDE systems.
result Solving new BSDEs to find optimal values and strategies.
Study optimal portfolios for many players in a market model with random coefficients.
problem Optimal portfolio selection for many players under relative performance criteria in a market model with random coefficients.
method Game theory and stochastic optimal control, focusing on CARA and CRRA risk preferences, and extending to continuum of players.
result Existence of forward Nash equilibrium and mean field equilibrium for the n-agent game and corresponding mean field stochastic optimal control problem.