New infill criterion identifies local optima in multimodal models.
problem Identify challenging test scenarios for physical systems.
method Model-based optimization with efficient global optimization and infill criterion.
result New infill criterion outperforms existing methods in identifying local optima.
InSphereNet uses infilling spheres for 3D object classification, improving accuracy with fewer parameters.
problem 3D object classification using points, voxels, or images.
method Constructs infilling spheres from signed distance field (SDF) for classification.
result InSphereNet achieves superior accuracy with fewer inputs and parameters.
This paper tackles text infilling, a task of filling missing text portions, and presents a self-attention model that outperforms other methods.
problem The task of filling missing text portions, especially when the number and length of missing portions are unknown.
method A self-attention model with segment-aware position encoding and bidirectional context modeling, trained on extensive supervised data.
result The self-attention model significantly outperforms other approaches, setting a strong baseline for future research.
ICP improves text infilling and POS tagging with valid confidence sets.
problem Statistical reliability of machine learning predictions.
method Inductive conformal prediction algorithms for text infilling and POS tagging.
result Valid set-valued predictions with small size for real-world applications.
RaMViD uses diffusion models for video prediction and infilling.
problem Predicting and infilling missing information in videos.
method Extends image diffusion models to videos using 3D convolutions and a new conditioning technique.
result Achieves state-of-the-art results on video prediction benchmarks.
We present mlrMBO, a flexible and comprehensive R toolbox for model-based optimization (MBO), also known as Bayesian optimization, which addresses the problem of expensive black-box optimization by approximating the given objective function through a surrogate regression model. It is designed for both single- and multi…
Optimal method detects jumps in jump-diffusion processes.
problem Detecting jumps in jump-diffusion processes with improved finite-sample performance.
method Iterative threshold-kernel method to optimally select threshold parameter.
result Approximate optimal threshold depends on spot volatility, jump intensity, and jump density.
Anticipatory model generates music with control over events.
problem Controlling symbolic music generation.
method Interleaving event and control sequences to predict future events.
result Anticipatory model matches autoregressive models in performance and can infill control tasks.
Model infers mineral locations from geospatial data, improving predictions with auxiliary data.
problem Challenges in characterizing hidden mineral deposits underground.
method Generative modeling approach using masked and infilled geospatial maps.
result Models achieve Dice coefficients of 0.31 and recalls of 0.22 at 1×1 mi² resolution.
Unified perspective unites Bayesian optimization and active learning for efficient goal-oriented optimization.
problem Efficiently optimize expensive engineering and scientific problems with limited data.
method Unified framework linking Bayesian infill criteria and active learning criteria.
result Unified approach formalizes Bayesian infill criteria and active learning criteria.
A new method for high-dimensional Bayesian optimization.
problem Challenges in extending BO to high dimensions.
method Expected Coordinate Improvement (ECI) criterion for high-dimensional Bayesian optimization.
result Significantly better results than standard BO and competitive results with state-of-the-art methods.
Study improves accuracy of weather data for real-time building simulations.
problem Anomalous and missing weather data affect real-time building energy simulations.
method Introduces a framework for quality control of measured weather data using anomaly detection and neural network infilling.
result Neural Networks enhance the accuracy of data imputation compared to traditional methods.
Sparse Gaussian process quantile regression tackles computational challenges in Bayesian quantile regression.
problem Nonconjugacy and computational cost in Gaussian process quantile regression.
method Sparse Gaussian process framework with Laplace approximation, adaptive inducing-input placement, and sequential data acquisition.
result Accuracy of Laplace approximation and effectiveness of adaptive mechanisms in reducing predictive uncertainty.
We use SMC with twist functions to improve probabilistic inference in LLMs.
problem Improving probabilistic inference in large language models.
method We use Sequential Monte Carlo with learned twist functions to estimate expected future values and focus inference on promising sequences.
result Twisted SMC improves the accuracy of language model inference and evaluation.
Paper introduces TtT, market-implied transition time, from greenium term structure.
problem Estimating market-implied transition time to a low-carbon economy.
method Develops inference theory for TtT, introduces two stochastic models.
result Combines two-layer analysis for consistent estimation of diffusion parameters.
Bayesian Optimization tackles hidden constraints in architecture optimization.
problem Optimizing system architectures with hidden constraints using expensive physics-based simulations.
method Surrogate-based optimization with Gaussian Process models, including strategies for handling failed evaluations.
result Best performance achieved with a mixed-discrete GP predicting Probability of Viability (PoV) and minimum PoV threshold selection.
Develops an SSBO algorithm for global optimization of expensive models.
problem Global optimization of expensive black-box models.
method Asynchronous hybrid-criterion with interval reduction.
result Improves global search ability and local search efficiency.
This paper improves volatility estimation for noisy multivariate data.
problem Nonparametric inference for nonlinear volatility functionals of multivariate Itô semimartingales.
method Pre-averaging and truncation techniques to handle noise and jumps; second-order expansion for bias correction; stable central limit theorems for asymptotic results.
result Achieves optimal convergence rate and stable central limit theorems with estimable asymptotic covariance matrices.
Diffusion models generate music sequences without autoregressive loops.
problem Generating music sequences from symbolic data using diffusion models.
method Parameterize discrete symbolic data in continuous latent space, train diffusion model, generate sequences through reverse process.
result Strong unconditional generation and post-hoc conditional infilling compared to autoregressive models.
The study evaluates 15 scalarizing functions in Bayesian multiobjective optimization.
problem Using scalarizing functions in computationally expensive multi- and many-objective optimization.
method 15 scalarizing functions were studied and compared using Gaussian process models and expected improvement as infill criterion.
result Different scalarizing functions have varying performance on benchmark problems with different numbers of objectives.
New GAN model deblends galaxy images with high accuracy and speed.
problem Deblending blended galaxy images in dense regions of the universe.
method Branched generative adversarial network (GAN) to produce images of deblended galaxies.
result High peak signal-to-noise ratio and structural similarity scores compared to ground truth images.
A scalable portfolio approach speeds up Bayesian optimization for noisy functions.
problem Efficiently selecting multiple designs in parallel for noisy, expensive black-box optimization.
method A portfolio approach that balances exploration and exploitation, using a scalable allocation strategy.
result Significant speed improvements over existing methods, with similar or better performance.
New scalarizing functions improve multi-objective Bayesian optimisation.
problem Improving multi-objective Bayesian optimisation efficiency.
method Comparing two infill criteria based on hypervolume improvement.
result Effective scalarizing functions enhance hypervolume maximisation.
New method boosts performance of diffusion models on discrete data like natural language.
problem Performance of diffusion models on discrete data like natural language is poor.
method Proposes score entropy, a novel loss that extends score matching to discrete spaces.
result Significantly boosts performance on language modeling tasks.
Bayesian optimal design of experiments (BODE) has been successful in acquiring information about a quantity of interest (QoI) which depends on a black-box function. BODE is characterized by sequentially querying the function at specific designs selected by an infill-sampling criterion. However, most current BODE method…
New CH covariance class improves spatial statistics by balancing differentiability and tail behavior.
problem Lack of control over mean-square differentiability and tail behavior in Matérn covariance functions.
method Developed a new Confluent Hypergeometric (CH) covariance class using a scale mixture of Matérn and polynomial covariances.
result The CH class offers improved theoretical properties and better performance in extrapolative settings.
An efficient algorithm calculates exact EHVI values for multi-objective optimization problems.
problem Efficient computation of EHVI values for multi-objective optimization problems.
method Partitioning the integration volume into axis-parallel slices and using a new hyperbox decomposition technique.
result Theoretical time complexity improved to Θ(nlogn), asymptotically optimal. Proposes a method to improve surrogate modeling and design optimization using latent variables.
problem Improving efficiency in multi-fidelity adaptive sampling without hierarchical assumptions.
method A framework using a latent variable Gaussian process to capture correlations between different fidelity models and optimize adaptive sampling.
result Demonstrates superior performance in convergence rate and robustness compared to existing methods.
Paper connects neural network score approximation to reverse diffusion model distribution approximation.
problem Quantifying the relationship between neural network score approximation and the distribution generated by reverse diffusion models.
method Combines Hornik's universal approximation theorem, Girsanov's theorem, and data processing inequality.
result Neural network score approximation guarantees distribution approximation in reverse diffusion models.
The study provides conditions for approximating Riemannian manifolds with polyhedral metrics.
problem Approximating Riemannian manifolds with polyhedral metrics.
method Conditions on curvature tensors for Lipschitz and local polyhedral approximations.
result Conditions are sufficient for local polyhedral approximations, conjectured to be sufficient for global approximations.
Paper proposes a new adaptive multiscale value function approximation for reinforcement learning.
problem Value function approximation in reinforcement learning with varying complexity.
method Adaptive multiscale approximation using multiresolution analysis and tree approximation.
result Convergence rate of the multiscale approximation is independent of basis function regularity.
Optimal function approximation with Relu neural networks achieves minimal error.
problem Finding the minimal error in approximating convex functions with Relu networks.
method Established necessary and sufficient conditions for optimal approximations, presented neural network architectures, and proposed an algorithm for convergence.
result Proved the convergence of the proposed algorithm and validated it with experimental results.
Geometric Gaussian approximations capture any distribution.
problem Approximating complex probability distributions.
method Geometric Gaussian approximations through diffeomorphisms or exponential maps.
result Geometric Gaussian approximations are universal, capturing any distribution.
Paper proposes MCMA architecture for neural approximate computing with higher invocation rate and energy savings.
problem Limited invocation rate of neural approximators leading to suboptimal energy efficiency.
method Introduces MCMA architecture with a multiclass classifier and multiple approximators, sharing hardware resources and efficiently swapping approximators.
result Significantly higher invocation rate and energy savings compared to existing methods.
Deep learning networks are approximated using dynamical systems theory.
problem Understanding the approximation capabilities of deep learning networks.
method Modeling deep residual networks as continuous-time dynamical systems and using approximation theories in Lp. result Established general sufficient conditions for universal approximation of deep residual networks.
Method approximates Riemannian barycenter on manifolds.
problem Computing the exact Riemannian barycenter is computationally expensive.
method Uses under- and over-approximations of Riemannian distance to compute an approximate barycenter.
result Approximation method is more efficient than exact methods and steepest descent.
Efficiently reduces tensor ranks using mean-field approximation.
problem Low-rank approximation of non-negative tensors.
method Mean-field approximation of tensor rank reduction.
result Our algorithm achieves faster and competitive tensor rank reduction.
Study approximates unknown function levels with queries.
problem Approximating unknown function levels through sequential queries.
method Introduce Bisect and Approximate algorithms to reduce to local function approximation.
result Rate-optimal sample complexity guarantees for H{ö}lder functions.
We study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such …
Softmax attention approximates complex functions and subsumes many known universal approximators.
problem Universal approximation of continuous sequence-to-sequence functions.
method Interpolation-based analysis of attention's internal mechanism, showing its ability to approximate ReLU functions.
result Softmax attention is a universal approximator for continuous sequence-to-sequence functions.
Deviation inequalities for stochastic approximation methods.
problem Establishing bounds on the deviation of stochastic approximation methods.
method Martingale approximation method for separately Lipschitz functions.
result Established various deviation inequalities for stochastic approximation by averaging and minimization.
Improved matrix approximation using randomized algorithms.
problem Finding better approximations of given matrices.
method Randomized algorithms to compute (HT) as an improved approximation. result Computed (HT) provides a better approximation than given F∗. Approximate symmetries of geodesic equations on 2-spheres are studied. These are the symmetries of the perturbed geodesic equations which represent approximate path of a particle rather than exact path. After giving the exact symmetries of the geodesic equations, two different approaches to study the approximate symmet…
We are concerned with an approximation problem for a symmetric positive semidefinite matrix due to motivation from a class of nonlinear machine learning methods. We discuss an approximation approach that we call {matrix ridge approximation}. In particular, we define the matrix ridge approximation as an incomplete matri…
Transformers use ReLUs to approximate softmax efficiently.
problem Analyzing resource usage in softmax transformer models.
method Translating ReLU approximation results to softmax attention mechanisms.
result Economic resource bounds for softmax attention mechanisms.
Approximating complex curves with simple parametric curves is widely used in CAGD, CG, and CNC. This paper presents an algorithm to compute a certified approximation to a given parametric space curve with cubic B-spline curves. By certified, we mean that the approximation can approximate the given curve to any given pr…
Recently, variational approximations such as the mean field approximation have received much interest. We extend the standard mean field method by using an approximating distribution that factorises into cluster potentials. This includes undirected graphs, directed acyclic graphs and junction trees. We derive generaliz…
Adaptive approximations improve variational inference for complex models.
problem Efficiently approximate marginal distributions and partition functions in complex probabilistic models.
method Two classes of adaptive approximations that include Bethe, tree-reweighted, and convex free energies.
result Proposed approximations automatically adapt to a given model and outperform existing methods.