Designing deterministic denominators for SGLD stabilizes large drifts.
problem Stabilizing large drifts in SGLD
method Using state-dependent envelopes and empirical quantiles for activation thresholds
result Proxy-quantile denominators are close to oracle-score behavior and improve deterministic taming choices
Study on regret minimization in deterministic MDPs.
problem Minimizing regret in deterministic reinforcement learning.
method Logarithmic regret lower bounds, leveraging graph theory and cycles.
result Explicitly quantifies the fundamental limit of performance achievable by any learning algorithm.
Paper develops efficient incomplete U-statistics for degenerate cases.
problem High computational cost and non-standard asymptotic behavior in degenerate U-statistics.
method Characterizes dependence structure using hypergraph theory and combinatorial designs, bypassing traditional Hoeffding decomposition.
result Derives a Berry-Esseen bound for incomplete U-statistics of deterministic designs, enabling Gaussian limiting distributions in degenerate cases.
Paper proposes a new method to stabilize noisy gradient algorithms.
problem Stochastic-gradient Langevin algorithms can introduce bias when taming denominators depend on stochastic-gradient realizations.
method Proposes a structure-preserving framework for designing tamed denominators that avoid unnecessary taming and maintain the stabilizing effect of taming.
result The method avoids stationary bias and explains the stationary error split into bias and remaining error.
Study compares deterministic and probabilistic ML for precise AM component dimensions.
problem Accurately estimate dimensions of additively manufactured parts with variability.
method Employed models integrating continuous and categorical factors, tested deterministic and probabilistic ML methods.
result Gaussian Process Regression and Bayesian Neural Networks provide strong predictive performance and uncertainty quantification.
New algorithms reduce regret in both stochastic and deterministic environments.
problem Designing algorithms that perform well in both types of MDPs.
method Proposed new environment norms and algorithms with variance-dependent regret bounds.
result First algorithm with simultaneously optimal bounds for both stochastic and deterministic MDPs.
Unified approach to experimental design using interlacing polynomials.
problem Experimental design problems, especially D/A/E-design and E-design.
method Unified deterministic approach using interlacing polynomials.
result Improved approximation guarantees for various experimental design objectives.
A new method for high-dimensional RBDO using stochastic emulators.
problem Efficient RBDO in high-dimensional settings.
method Unified stochastic representation, stochastic emulators, deterministic mapping.
result Significant computational gains in high-dimensional settings.
Electrostatics method samples complex distributions deterministically.
problem Sampling and inference of complex, high-dimensional distributions.
method Electrostatics-based particle system with Newton mechanics principles.
result Method achieves comparable performance to other methods in benchmark tasks.
Paper bounds PAC RL sample complexity in deterministic MDPs.
problem Identify ε-optimal policy with high probability.
method Proposes nearly matching upper and lower bounds on sample complexity, introduces deterministic return gap, uses graph-theoretical concepts and maximum-coverage exploration.
result First nearly matching upper and lower bounds on sample complexity for PAC RL in deterministic MDPs.
New algorithm improves online binary classification with constant time complexity.
problem Online binary classification with rebalancing.
method Non-iteratively reweighted recursive least-squares.
result Exacts converges to batch formulation and outperforms existing algorithms.
This paper presents a stochastic logic time delay reservoir design. The reservoir is analyzed using a number of metrics, such as kernel quality, generalization rank, performance on simple benchmarks, and is also compared to a deterministic design. A novel re-seeding method is introduced to reduce the adverse effects of…
Bayesian surrogate models reduce uncertainty in high-dimensional design optimisation problems.
problem Uncertainty in high-dimensional inputs for complex computational models.
method Variational Bayesian inference for constructing statistical surrogates with Gaussian process priors and KL divergence for approximation.
result The RDVGP surrogate provides accurate and versatile approximations for robust structural optimisation.
Paper introduces deterministic EM approximations for non-convex likelihood functions.
problem Deterministic approximations for the E-step of EM algorithm are lacking.
method Developed a theoretical framework for deterministic approximations, analyzed Riemann sums and tempered EM.
result Proved convergence guarantees for deterministic approximations and new non-trivial temperature profiles.
This paper presents a convergence analysis of kernel-based quadrature rules in misspecified settings, focusing on deterministic quadrature in Sobolev spaces. In particular, we deal with misspecified settings where a test integrand is less smooth than a Sobolev RKHS based on which a quadrature rule is constructed. We pr…
We consider the problem of online linear regression on arbitrary deterministic sequences when the ambient dimension d can be much larger than the number of time rounds T. We introduce the notion of sparsity regret bound, which is a deterministic online counterpart of recent risk bounds derived in the stochastic setting…
The total variation distance is a core statistical distance between probability measures that satisfies the metric axioms, with value always falling in [0,1]. This distance plays a fundamental role in machine learning and signal processing: It is a member of the broader class of f-divergences, and it is related to …
Bayesian attention improves model performance and robustness.
problem Limited exploration of stochastic attention in neural networks.
method Introduces Bayesian attention belief networks using gamma and Weibull distributions.
result Outperforms deterministic and stochastic attention methods in accuracy and robustness.
This paper considers the problem of high dimensional signal detection in a large distributed network whose nodes can collaborate with their one-hop neighboring nodes (spatial collaboration). We assume that only a small subset of nodes communicate with the Fusion Center (FC). We design optimal collaboration strategies w…
Proposes a new policy gradient algorithm to improve reinforcement learning efficiency and stability.
problem Inefficiency and instability of DDPG in practical applications, and difficulty in controlling Q estimation bias and variance.
method Introduces a Regularly Updated Deterministic (RUD) policy gradient algorithm.
result The RUD algorithm makes better use of new data and has lower Q value variance, leading to improved performance.
ZDPG learns model-free policies without critics, improving on PG.
problem Model-free policy learning in complex dynamic problems.
method Approximates policy-reward gradients via two-point stochastic evaluations of the Q-function.
result Restores true model-free policy learning without critics, with improved stability and efficiency.
Bayesian method for estimating inputs leading to specific probability outputs.
problem Estimating inputs for specific probability outputs of uncertain functions.
method Bayesian strategy using Gaussian process modeling and SUR principle.
result Surpassed performance of existing methods through numerical experiments.
New algorithm solves stochastic optimization problems with unknown gradients.
problem Solving nonlinear optimization problems with stochastic objectives and deterministic constraints.
method Adaptive SQP with differentiable exact augmented Lagrangian and stochastic line search.
result Global convergence established for both non-adaptive and adaptive SQP methods.
Convex regression is a promising area for bridging statistical estimation and deterministic convex optimization. New piecewise linear convex regression methods are fast and scalable, but can have instability when used to approximate constraints or objective functions for optimization. Ensemble methods, like bagging, sm…
Generative models use Riemannian manifolds to improve latent space interpretation.
problem Generative models often bias latent space interpretations.
method Use Riemannian manifolds to define latent space paths that respect ambient geometry.
result Improves interpretability of learned representations for both stochastic and deterministic generators.
A new attack for probabilistic classifiers adapts to noise levels.
problem Adversarial examples can mislead probabilistic classifiers.
method Adapts HopSkipJump attack for probabilistic classifiers, adjusting queries based on noise levels.
result Decision-based attacks are effective against probabilistic classifiers, even with noise.
Adaptive batching improves Gaussian process surrogates for noisy level set estimation.
problem Learning the level set of noisy simulator responses.
method Developed four novel adaptive batching schemes for Gaussian process metamodels.
result Adaptive batching brings significant computational speed-ups with minimal loss of modeling fidelity.
This article provides a novel framework to evaluate limit order tactics that highlights expected fill price, adverse price selection cost, and opportunity cost. We formulate the problem of optimal execution of market orders with nonlinear market impact, power law decay kernel, and stochastic and deterministic liquidity…
Paper accelerates diffusion models, improving sampling speed.
problem Low sampling speed in score-based diffusion models.
method Design of novel training-free algorithms for deterministic and stochastic samplers.
result Accelerated samplers converge faster with improved rates.
Estimates funding impact from an algorithmic relief rule, finding little effect on hospital activities.
problem Evaluating the impact of algorithmic policy decisions.
method Developed a treatment-effect estimator using algorithmic decisions as instruments.
result Funding from an algorithmic relief rule had little effect on COVID-19-related hospital activities.
Bayesian regression underestimates parameter uncertainties in noisy models.
problem Parameter uncertainties are underestimated in Bayesian regression for imperfect models.
method Analyzed and designed an ansatz to correct for misspecification in near-deterministic surrogate models.
result Posterior distributions must cover all training points to avoid divergent generalization error.
Stochastic neural networks with infinite width become deterministic, reducing training variance.
problem Understanding how stochasticity in neural networks affects learning and regularization.
method Theoretical analysis of stochastic neural networks with infinite width.
result As the width of an optimized stochastic neural network increases, its predictive variance on the training set decreases to zero.
We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies wit…
In this paper, we discuss the statistical properties of the ℓq optimization methods (0<q≤1), including the ℓq minimization method and the ℓq regularization method, for estimating a sparse parameter from noisy observations in high-dimensional linear regression with either a deterministic or rando…
We present a probabilistic modeling framework and adaptive sampling algorithm wherein unsupervised generative models are combined with black box predictive models to tackle the problem of input design. In input design, one is given one or more stochastic "oracle" predictive functions, each of which maps from the input …
The performance of Orthogonal Matching Pursuit (OMP) for variable selection is analyzed for random designs. When contrasted with the deterministic case, since the performance is here measured after averaging over the distribution of the design matrix, one can have far less stringent sparsity constraints on the coeffici…
We investigate the high-dimensional regression problem using adjacency matrices of unbalanced expander graphs. In this frame, we prove that the ℓ2-prediction error and the ℓ1-risk of the lasso and the Dantzig selector are optimal up to an explicit multiplicative constant. Thus we can estimate a high-dim…
EHVI outperforms scalarized EI in MOBO for molecule design.
problem Benchmarking MOBO strategies for molecule design.
method Compared EHVI against fixed-weight scalarized EI in MOBO.
result EHVI consistently outperforms scalarized EI in molecular optimization tasks.
This paper highlights new opportunities for designing large-scale machine learning systems as a consequence of blurring traditional boundaries that have allowed algorithm designers and application-level practitioners to stay -- for the most part -- oblivious to the details of the underlying hardware-level implementatio…
We propose a conditional non-autoregressive neural sequence model based on iterative refinement. The proposed model is designed based on the principles of latent variable models and denoising autoencoders, and is generally applicable to any sequence generation task. We extensively evaluate the proposed model on machine…
New method designs fairer transport plans with uncertainty.
problem Designing fair and balanced mass transport plans.
method Hierarchical fully probabilistic design (HFPD) for transport plans.
result Optimal hyperprior for transport plans with uncertain marginals.
In data-limited settings, stochastic policies can outperform deterministic ones in bandit problems.
problem Making reliable decisions with limited data in bandit problems.
method Designing TRUST, an algorithm that uses localization laws and relative pessimism.
result TRUST achieves comparable sample complexity to LCB on minimax problems but is significantly lower on few-sample problems.
We address the challenge of designing optimal adversarial noise algorithms for settings where a learner has access to multiple classifiers. We demonstrate how this problem can be framed as finding strategies at equilibrium in a two-player, zero-sum game between a learner and an adversary. In doing so, we illustrate the…
Paper explores stability, regularization, and gradient flows for stochastic inverse problems.
problem Recovering random probability distributions from measurements.
method Direct inversion, variational formulation with regularization, and optimization via gradient flows.
result The choice of metric impacts stability and properties of the optimizer.
Despite the numerous advances, reinforcement learning remains away from widespread acceptance for autonomous controller design as compared to classical methods due to lack of ability to effectively tackle the reality gap. The reliance on absolute or deterministic reward as a metric for optimization process renders rein…
Developed mlf-core for deterministic machine learning.
problem Ensuring machine learning models are deterministic for verification.
method Formulated requirements, developed mlf-core ecosystem, tested various models.
result Demonstrated deterministic models in biomedical fields.
Aims to optimize complex multivariate systems with constraints.
problem Optimizing force-field systems in physics with large-scale simulations.
method Combines machine learning and experimental design to find feasible input combinations.
result Locates multiple good regions in the input space.
Deep Deterministic Policy Gradient (DDPG) has been proved to be a successful reinforcement learning (RL) algorithm for continuous control tasks. However, DDPG still suffers from data insufficiency and training inefficiency, especially in computationally complex environments. In this paper, we propose Asynchronous Episo…