Universal algorithm minimizes adaptive regret for various convex functions.
problem Minimizing adaptive regret in changing environments for multiple convex functions.
method Borrowing MetaGrad's idea of multiple learning rates and using sleeping experts.
result First universal algorithm for minimizing adaptive regret of convex functions.
Paper improves adaptive regret for convex and smooth functions.
problem Online convex optimization in changing environments.
method Develops adaptive algorithms exploiting both convexity and smoothness.
result Regret bounds are comparable to worst-case results but tighter when comparators have small losses.
Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.
problem Optimal rate of convergence and cost of adaptation in distributed nonparametric function estimation.
method Distributed minimax estimation and adaptive estimation under communication constraints for Gaussian sequence model and white noise model.
result Established minimax rate of convergence and exact communication cost for adaptation.
ABUs learn and adapt activation functions for deep neural networks.
problem Lack of a unified theory connecting task and network properties with activation functions.
method Introduce Adaptive Blending Units (ABUs) as a trainable linear combination of activation functions.
result Advantages of ABUs over common activation functions across various network specifications.
Adapts to estimate functions from noisy ERT data.
problem Estimating functions from noisy Exponential Radon Transform data.
method Locally adaptive kernel type estimator for functions of varying smoothness.
result Achieves minimax optimal rate up to a log(n) factor for Sobolev functions.
Adaptive sampling improves convex function learning.
problem Learning convex functions in the L ∞ L_\infty L ∞ norm. method Function-specific complexity measure for adaptive sampling.
result Adaptive sampling nearly achieves optimal error rate.
Research shows minimal communication limits adaptive function estimation rates.
problem Adaptive estimation of a smooth function under minimal communication constraints.
method Investigates the L ∞ L_\infty L ∞ -risk and L 2 L_2 L 2 -risk under different numbers of servers. result For L ∞ L_\infty L ∞ -risk, optimal rates cannot be achieved under minimal communication. For L 2 L_2 L 2 -risk, adaptivity is possible but depends on server number and sample size. This research improves neural network performance with adaptive activation functions in sparse data settings.
problem Limited data availability in scientific and engineering problems.
method Investigation of two types of adaptive activation functions with individual trainable parameters.
result Adaptive activation functions, especially with individual trainable parameters, enhance prediction accuracy and confidence in sparse data settings.
Paper proposes a new adaptive multiscale value function approximation for reinforcement learning.
problem Value function approximation in reinforcement learning with varying complexity.
method Adaptive multiscale approximation using multiresolution analysis and tree approximation.
result Convergence rate of the multiscale approximation is independent of basis function regularity.
A novel adaptive kernel improves RBF neural networks performance.
problem Improving performance of RBF neural networks.
method Adaptive fusion of Euclidean and cosine distance measures using gradient descent.
result The method outperforms manual fusion on three major problems.
Issue found in proof of adaptive submodular function minimization theorem.
problem Proof of adaptive submodular function minimization theorem is incorrect.
method Example provided to show incorrectness of a critical step in a related theorem.
result Critical step in related theorem is found to be incorrect.
Adaptive replication improves stochastic function optimization.
problem Challenges in accurately estimating functions with high variance.
method Trust-region-based Bayesian optimization with adaptive replication.
result Adaptive replication substantially improves solution accuracy and efficiency.
Locally adaptive activation functions boost deep and physics-informed neural networks.
problem Improving the performance and training speed of deep and physics-informed neural networks.
method Layer-wise and neuron-wise locally adaptive activation functions with a slope recovery term.
result The proposed methods accelerate convergence and reduce training cost.
Bayesian nonparametric LABS model adapts to function smoothness in Besov spaces.
problem Estimating functions with unknown smoothness in Besov spaces.
method Lévy Adaptive B-spline (LABS) regression model with automatic smoothness adaptation.
result LABS posterior contracts around true function in Besov classes at nearly minimax-optimal rates.
A nonlinear wave alternative for the standard Black-Scholes option-pricing model is presented. The adaptive-wave model, representing 'controlled Brownian behavior' of financial markets, is formally defined by adaptive nonlinear Schrödinger (NLS) equations, defining the option-pricing wave function in terms of the stock…
Adapts neural network neurons' activation functions for better predictions.
problem Training neural networks with fixed activation functions limits their performance.
method Proposes training over a shape parameter, allowing neurons to adapt their own activation functions.
result Improves prediction accuracy by allowing neurons to tune their activation functions.
Adapted metrics found on complex manifolds.
problem Finding metrics suitable for complex manifolds.
method Characterizing adapted metrics as critical points of a functional.
result Gauduchon metric is adapted on locally conformally product manifolds.
New algorithm adapts to unknown smoothness in contextual bandits.
problem Adapting to unknown smoothness in non-parametric multi-armed bandits.
method Develops a self-similarity condition-based policy to adapt to unknown smoothness.
result Matches known smoothness case's regret rate for differentiable and non-differentiable payoff functions.
Proposes a new method for estimating non-pathwise differentiable functional parameters.
problem Estimating dose-response curves for continuous exposure.
method Targeted Highly Adaptive Lasso (HAL) for non-pathwise differentiable functional parameters.
result The Targeted HAL-MLE achieves dimension-free rates up to log(n) factors and outperforms other methods in simulations.
Fast algorithms developed for adaptive and fully adaptive submodular maximization problems.
problem Maximizing submodular functions subject to constraints in linear time.
method Developed linear-time algorithms for two submodular maximization problems: adaptive and fully adaptive.
result Achieved ( 1 − 1 / e − ε ) (1-1/e-ε) ( 1 − 1/ e − ε ) approximation ratio for adaptive submodular maximization and $ rac{1-1/e-ε}{4-2/e-2ε}$ for fully adaptive submodular maximization. New algorithm adapts to unknown smoothness in stochastic bandits with polynomial cost.
problem Adapting to unknown smoothness in stochastic bandits.
method Reconsidered Locatelli and Carpentier's lower bound, defined admissible rate functions, and developed a new algorithm.
result New algorithm matches minimal rate functions and provides polynomial cost of adaptation.
New method optimizes unknown functions adaptively.
problem Tuning hyperparameters in machine learning.
method Gradually and adaptively optimizing an unknown function using estimated gradients.
result Advantages for tuning high dimensional hyperparameters in machine learning.
LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
This paper explores adaptive neural activation in RNNs for better learning.
problem Fixed neural activation functions limit the performance and adaptability of RNNs.
method Developed a novel parametric family of nonlinear activation functions inspired by biological neurons.
result Adaptive neural activation improves learning speed and performance in RNNs.
A new BO method adapts hyperparameters online and uses a novel kernel for global and local optimization.
problem Expensive black-box optimization problems.
method Online length-scale adaption, mixed-global-local kernel, and adaptive hyperparameters.
result The proposed method outperforms state-of-the-art BO methods on global optimization benchmarks.
Adaptive decision-making for state estimation with partial observations.
problem Stochastic state estimation with partial observations and active diagnosis.
method Weak adaptive submodularity and adaptive greedy policy.
result Adaptive greedy policy achieves near-optimal performance for weakly adaptive submodular reward functions.
Proposes adaptive ridge regression for functional linear models with piecewise shapes.
problem Functional linear regression with unknown coefficient function.
method Adaptive piecewise function template with L 2 L_2 L 2 penalization. result Improves predictive power and interpretability compared to standard methods.
Adaptive tensor modeling preserves continuity in multidimensional data.
problem Discretization of continuous multidimensional data loses important information.
method Functional Tucker decomposition (FTD) with RKHS modeling.
result FTD enables adaptive and expressive tensor modeling.
New analysis proves consistency for adaptive Bayesian quadrature methods.
problem No theoretical guarantees for adaptive Bayesian quadrature methods.
method Introduces weak adaptivity and proves consistency for a broad class of adaptive Bayesian quadrature rules.
result Proves consistency and derives non-tight but informative convergence rates for adaptive Bayesian quadrature methods.
Study on adaptivity to kernel regularity in bandit problems.
problem Adaptation to unknown kernel regularity in continuum-armed bandit problems.
method Derive adaptivity lower bound and verify with minimax non-adaptive kernelised bandit algorithms.
result Impossibility of achieving optimal cumulative regret in different RKHSs with varying regularities.
New method reduces Gibbs partition function estimation complexity.
problem Estimating partition functions of Gibbs distributions.
method Doubly-adaptive MCMC with adaptive cooling schedule and mean estimator.
result Outperforms state-of-the-art algorithms in computational complexity and robustness.
New algorithms optimize private convex optimization with faster rates for functions with κ-growth.
problem Optimizing private convex functions with varying difficulty and growth conditions.
method Adapts inverse sensitivity mechanism and localization techniques to achieve faster rates without knowing growth constant.
result Achieves faster privacy rates ( d / n ε ) f r a c κ κ − 1 ({\sqrt{d}}/{n\varepsilon})^{ fracκ{κ- 1}} ( d / n ε ) f r a c κ κ − 1 for functions with κ-growth. Deep networks adapt to function regularity and data distribution.
problem Understanding deep learning's adaptability to function regularity and data distribution.
method Developed nonparametric approximation and estimation theories for a broad class of functions using deep ReLU networks.
result Deep neural networks are adaptive to different regularity of functions and nonuniform data distributions.
Direct proof shows adaptive gradient descent converges near-linearly for convex functions.
problem Proving near-linear convergence of adaptive gradient descent for convex functions.
method Direct Lyapunov-based argument for convex functions with unique minimizer.
result Direct proof of near-linear convergence for convex functions.
An adaptive dropout approach improves high-dimensional Bayesian optimization.
problem High-dimensional black-box optimization problems.
method Adaptive dropout of variables in the acquisition function.
result AdaDropout effectively tackles high-dimensional challenges and improves solution quality.
A new method for learning function parameters in operators using data-adaptive RKHS.
problem Learning function parameters in operators with robustness to noise and numerical error.
method Data Adaptive RKHS Tikhonov Regularization (DARTR) method.
result DARTR leads to an accurate estimator robust to noise and numerical error, converging at a consistent rate as data refines.
Proposes a new method for non-parametric regression with adaptive truncation.
problem Sparse high-dimensional additive models with varying complexity.
method Convex hierarchical penalty combining finite basis and smoothing penalties.
result Converges at minimax rate for hierarchical function classes and sparse additive models.
Adaptive pricing framework for perpetual contracts using liquidity curves and oracles.
problem Ensuring stable and predictable pricing for perpetual contracts.
method Uses liquidity curves and on-chain oracles with parabolic and sigmoid functions to quote prices and fees.
result Ensures pricing stability and predictability through adaptive pricing framework.
Study adaptive sensing of Cox processes using posterior sampling and positive bases.
problem Adaptive sensing of Cox point processes with intensity function modeling.
method Model intensity function as truncated Gaussian process in positive basis, use Langevin dynamics and posterior sampling.
result Demonstrated improved sensing compared to classical Bayesian experimental design.
New algorithm maximizes non-monotone adaptive submodular functions in linear time.
problem Maximizing non-monotone adaptive submodular functions subject to a cardinality constraint.
method Developed a linear-time algorithm for non-monotone adaptive submodular maximization.
result Achieved a 1 / e − ε 1/e-ε 1/ e − ε approximation ratio with O ( n ε − 2 log ε − 1 ) O(nε^{-2}\log ε^{-1}) O ( n ε − 2 log ε − 1 ) value oracle queries. Paper introduces adaptive parameterization to improve neural network efficiency.
problem Neural networks' limited flexibility due to fixed activation functions.
method Adaptive parameterization of feed-forward layers that learn to adapt based on input.
result Adaptive LSTM achieves state-of-the-art performance with fewer parameters and faster convergence.
SoftAdapt dynamically adjusts loss weights for multi-part functions.
problem Slow convergence and poor weight selection for multi-part loss functions.
method SoftAdapt dynamically changes weights based on live performance statistics.
result Improved convergence and better weight selection for multi-part loss functions.
Generative models learn better with data-adaptive noise.
problem Learning heavy-tailed distributions in flow-based models.
method Data-adaptive latent noise using 1D quantile functions optimized via Wasserstein distance.
result Flexibility and effectiveness in learning heavy-tailed and compactly supported distributions.
Locally adaptive clustering for tree delineation.
problem Tree delineation from distance data.
method Locally adaptive hierarchical cluster termination.
result Multi-scale alternative to conventional termination criteria.
Adaptive RBF-KAN improves KANs by dynamically adjusting kernel parameters.
problem Efficiently approximating multivariate functions using learnable univariate edge functions.
method Integrates LOOCV-based kernel scale estimation with adaptive kernel learning.
result Adaptive RBF-KAN outperforms fixed kernel KANs on various benchmark functions.
Greedy algorithms near-optimal for adaptive optimization with budget constraint.
problem Adaptive optimization with budget constraint in AI and ML.
method Investigates two simple greedy algorithms and a combined algorithm for pointwise submodular and cost-sensitive submodular functions.
result Best greedy algorithm is near-optimal with respect to optimal algorithm using half the budget.
New algorithm reduces TV-denoising to adaptive online learning.
problem Estimating TV-bounded functions from noisy samples.
method Deep connection to Strongly Adaptive online learning; O ( n log n ) O(n \log n) O ( n log n ) time algorithm. result Near minimax optimal rate of O ( n 1 / 3 C n 2 / 3 ) O(n^{1/3}C_n^{2/3}) O ( n 1/3 C n 2/3 ) under squared error loss. Adaptive model selection for RL with unknown function classes.
problem Model selection for RL with unknown function classes.
method Proposed adaptive algorithms that adapt to the smallest function class containing the true model.
result Cumulative regret matches that of an oracle with known function classes.